00000100:00100000:3.0:1788619896.877248:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d00700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571197184:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 279us (1825us total) trans 4294967336 rc 0/0 00000100:00100000:3.0:1788619896.877268:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 146 00002000:00010000:3.0:1788619896.877272:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:3.0:1788619896.877274:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6d00700 with x1875503571197184 ext(0->18446744073709551615) 00002000:00010000:3.0:1788619896.877279:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6d00700. 00000100:00100000:3.0:1788619896.883697:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619896.886811:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619896.886851:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619896.894970:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619896.895051:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619896.906322:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571198080 00000100:00100000:3.0:1788619896.906511:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571198080 00000100:00100000:3.0:1788619896.925449:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619896.925503:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619896.928463:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503588977920:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619896.928500:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001b62f519/0x1cd7fe5368732d7e lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a4ec expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619896.928520:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001b62f519/0x1cd7fe5368732d7e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a4ec expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619896.928530:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505ce00, rc 0) 00000100:00100000:3.0:1788619896.928577:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571198336:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2797us (3137us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619896.928586:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 253 00000100:00100000:3.0:1788619896.931766:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571198592 00000100:00100000:3.0:1788619896.931787:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571198592 00000100:00100000:3.0:1788619896.931851:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 254 00000100:00100000:3.0:1788619896.931862:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571198592:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619896.931881:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619896.931899:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b0fcf257/0x1cd7fe5368732d85 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1a4fa expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619896.932180:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c685b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503588978048:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619896.932301:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48ccc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619896.946581:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571198848 00000100:00100000:3.0:1788619896.946591:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571198848 00000100:00100000:3.0:1788619896.946741:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 255 00000100:00100000:3.0:1788619896.946749:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571198848:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619896.946756:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619896.946766:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000039c2d152/0x1cd7fe5368732d9a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1a508 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619896.946798:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c685b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503588978432:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619896.947263:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c540 going to sleep for 11 seconds 00000100:00100000:3.0:1788619896.949663:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c685b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503588978432:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619896.949682:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000039c2d152/0x1cd7fe5368732d9a lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a508 expref: 7 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619896.949696:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000039c2d152/0x1cd7fe5368732d9a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a508 expref: 7 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619896.949718:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5048600, rc 0) 00000100:00100000:3.0:1788619896.950239:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571198848:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3489us (3707us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619896.953669:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 255 00000100:00100000:3.0:1788619896.962078:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571199360 00000100:00100000:3.0:1788619896.962086:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571199360 00000100:00100000:3.0:1788619896.962145:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 257 00000100:00100000:3.0:1788619896.962154:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b15f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571199360:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619896.962173:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619896.962194:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000574a4e4c/0x1cd7fe5368732daf lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1a524 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619896.962252:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c685b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503588978816:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619896.962357:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c540 going to sleep for 11 seconds 00000100:00100000:3.0:1788619896.965252:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c685b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503588978816:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619896.965290:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000574a4e4c/0x1cd7fe5368732daf lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a524 expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619896.965314:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000574a4e4c/0x1cd7fe5368732daf lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a524 expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619896.965396:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5048e00, rc 0) 00000100:00100000:3.0:1788619896.965444:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b15f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571199360:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3290us (3560us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619896.965471:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 257 00000100:00100000:3.0:1788619896.977904:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571199488 00000100:00100000:3.0:1788619896.977914:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571199488 00000100:00100000:3.0:1788619896.977976:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 438 00000100:00100000:3.0:1788619896.977992:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b16680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2352:x1875503571199488:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619896.978013:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619896.978032:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048e00/0x1cd7fe5368732dbd lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1a52b expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619896.978067:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368732d31 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a4ad expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619896.978108:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c685b480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503588979072:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619896.978189:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c540 going to sleep for 11 seconds 00000100:00100000:3.0:1788619896.984516:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c685b480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503588979072:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619896.984532:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 807243 00010000:00010000:3.0:1788619896.984545:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048e00/0x1cd7fe5368732dbd lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a52b expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619896.984561:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619896.984565:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619896.984570:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048e00/0x1cd7fe5368732dbd lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a52b expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619896.984581:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5048e00, rc 0) 00000100:00100000:3.0:1788619896.985176:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b16680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2352:x1875503571199488:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6639us (6794us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619896.985195:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 438 00000100:00100000:3.0:1788619897.039494:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571200384 00000100:00100000:3.0:1788619897.039502:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571200384 00010000:00010000:3.0:1788619897.055634:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368732d54 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a4c2 expref: 12 pid: 11397 timeout: 489 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619897.055651:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6d02d80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503588979840:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619897.055674:0:5845:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6201e00/0x1cd7fe5368732de7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.055684:0:5845:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6201e00/0x1cd7fe5368732de7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.055690:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6201e00/0x1cd7fe5368732de7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788619897.055796:0:5845:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967495 00000004:00010000:3.0:1788619897.055836:0:5845:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 807243 -> 807243 00000004:00010000:3.0:1788619897.055839:0:5845:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619896 -> 1788619896 00000004:00010000:3.0:1788619897.055842:0:5845:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619896 -> 1788619896 00000004:00010000:3.0:1788619897.055845:0:5845:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619896 -> 1788619896 00000004:00010000:3.0:1788619897.055847:0:5845:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 774 -> 774 00000004:00010000:3.0:1788619897.055854:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000ed0710b6 reply state 00000000f53c1b3d transno 4294967496 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6201e00/0x1cd7fe5368732de7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619897.055864:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788619897.055935:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b15880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+19:14908:x1875503571200640:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 8009us (8361us total) trans 4294967496 rc 0/0 00000100:00100000:3.0:1788619897.055953:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 442 00000020:00080000:3.0:1788619897.055957:0:5845:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1788619897.059599:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619897.071363:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571201024 00000100:00100000:3.0:1788619897.071375:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571201024 00010000:00010000:3.0:1788619897.071437:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368732dd9 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a547 expref: 12 pid: 9417 timeout: 489 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619897.071479:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 155 00000100:00100000:3.0:1788619897.071488:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7431c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571201024:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619897.071503:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619897.071532:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368732dd9 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a547 expref: 12 pid: 9417 timeout: 489 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.071546:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368732dd9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a547 expref: 12 pid: 9417 timeout: 489 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.071567:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368732dd9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a547 expref: 12 pid: 9417 timeout: 489 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.071583:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619897.071589:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368732dee lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a555 expref: 17 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.071600:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368732dee lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a555 expref: 17 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.071628:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368732dee lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a555 expref: 17 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.071668:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368732dee lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a555 expref: 17 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619897.071682:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c74f6d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503588980096:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619897.072165:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c540 going to sleep for 11 seconds 00000100:00100000:3.0:1788619897.077831:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c74f6d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503588980096:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619897.077860:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619897.077916:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7431c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571201024:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 6427us (7806us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619897.077927:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 155 00000100:00100000:3.0:1788619897.086318:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619897.115765:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571201664 00000100:00100000:3.0:1788619897.115775:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571201664 00000100:00100000:3.0:1788619897.115827:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 260 00000100:00100000:3.0:1788619897.115837:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6531880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+10:2353:x1875503571201664:12345-192.168.201.5@tcp:16:fsx.0 00000100:00100000:3.0:1788619897.171655:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571201792 00000100:00100000:3.0:1788619897.171667:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571201792 00000100:00100000:3.0:1788619897.171769:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 444 00000100:00100000:3.0:1788619897.171781:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d03800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+18:14908:x1875503571201792:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619897.171803:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619897.171824:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048200/0x1cd7fe5368732df5 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1a55c expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.171946:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049e00/0x1cd7fe5368732dfc lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.171963:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049e00/0x1cd7fe5368732dfc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.171976:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049e00/0x1cd7fe5368732dfc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788619897.172006:0:5846:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 2046448 00000004:00200000:3.0:1788619897.172010:0:5846:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788619897.172027:0:5846:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049e00/0x1cd7fe5368732dfc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.172055:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048200/0x1cd7fe5368732df5 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a55c expref: 20 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.172074:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049e00/0x1cd7fe5368732dfc lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a55c expref: 19 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.172082:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619897.172086:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788619897.172089:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5049e00, rc 0) 00000100:00100000:3.0:1788619897.172167:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d03800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+19:14908:x1875503571201792:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 388us (632us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619897.172175:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 444 00000100:00100000:3.0:1788619897.175082:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571201920 00000100:00100000:3.0:1788619897.175091:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571201920 00000100:00100000:3.0:1788619897.175168:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 445 00000100:00100000:3.0:1788619897.175178:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d00000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571201920:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619897.175197:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619897.175221:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049c00/0x1cd7fe5368732e03 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1a563 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.175267:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368732dee lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a555 expref: 18 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.175282:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049c00/0x1cd7fe5368732e03 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a563 expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.175317:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368732dee lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a555 expref: 18 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.175363:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368732dee lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a555 expref: 18 pid: 9417 timeout: 489 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619897.175379:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c74f4000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503588980224:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619897.184115:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619897.217806:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571202560 00000100:00100000:3.0:1788619897.217816:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571202560 00000100:00100000:3.0:1788619897.217877:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 447 00000100:00100000:3.0:1788619897.217888:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d01880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2354:x1875503571202560:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619897.217920:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619897.217942:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048200/0x1cd7fe5368732e11 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1a571 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.218147:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049c00/0x1cd7fe5368732e03 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a563 expref: 11 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619897.218187:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c74f7100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503588980608:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619897.218268:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c240 going to sleep for 11 seconds 00000100:00100000:3.0:1788619897.220696:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c74f7100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503588980608:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619897.220722:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 807243 00010000:00010000:3.0:1788619897.220741:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048200/0x1cd7fe5368732e11 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a571 expref: 19 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.220763:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619897.220766:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619897.220773:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048200/0x1cd7fe5368732e11 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a571 expref: 19 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.220787:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5048200, rc 0) 00000100:00100000:3.0:1788619897.220844:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d01880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2354:x1875503571202560:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2957us (3136us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619897.220853:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 447 00000100:00100000:3.0:1788619897.231699:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571202816 00000100:00100000:3.0:1788619897.231709:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571202816 00000100:00100000:3.0:1788619897.231769:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 449 00000100:00100000:3.0:1788619897.231779:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7431500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2354:x1875503571202816:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619897.231799:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619897.231819:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049a00/0x1cd7fe5368732e1f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1a57f expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.231873:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049c00/0x1cd7fe5368732e03 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a563 expref: 11 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619897.232068:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c74f7100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503588980864:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619897.232188:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c240 going to sleep for 11 seconds 00000100:00100000:3.0:1788619897.234559:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c74f7100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503588980864:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619897.234586:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 807243 00010000:00010000:3.0:1788619897.234608:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049a00/0x1cd7fe5368732e1f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a57f expref: 19 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.234629:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619897.234633:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619897.234639:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049a00/0x1cd7fe5368732e1f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a57f expref: 19 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.234655:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5049a00, rc 0) 00000100:00100000:3.0:1788619897.234707:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7431500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2354:x1875503571202816:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2929us (3107us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619897.234716:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 449 00000100:00100000:3.0:1788619897.244103:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571202944 00000100:00100000:3.0:1788619897.244115:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571202944 00000100:00100000:3.0:1788619897.244312:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 450 00000100:00100000:3.0:1788619897.244346:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d03b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2355:x1875503571202944:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619897.244369:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619897.244392:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048200/0x1cd7fe5368732e26 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1a586 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.244459:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049c00/0x1cd7fe5368732e03 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a563 expref: 11 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619897.244502:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c74f7100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503588980992:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619897.244591:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c240 going to sleep for 11 seconds 00000100:00100000:3.0:1788619897.280052:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619897.327263:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619897.334428:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619897.338711:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619897.341911:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619897.345830:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571203968 00000100:00100000:3.0:1788619897.345838:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571203968 00000100:00100000:3.0:1788619897.345929:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 182 00000100:00100000:3.0:1788619897.345937:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6530a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2354:x1875503571203968:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:3.0:1788619897.346266:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 807243 -> 807243 00000004:00010000:3.0:1788619897.346270:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619897 -> 1788619897 00000004:00010000:3.0:1788619897.346273:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619896 -> 1788619896 00000004:00010000:3.0:1788619897.346275:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619897 -> 1788619897 00000004:00010000:3.0:1788619897.346276:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 774 -> 774 00000100:00100000:3.0:1788619897.346367:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6530a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2354:x1875503571203968:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 429us (589us total) trans 4294967498 rc 0/0 00000100:00100000:3.0:1788619897.346374:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 182 00002000:00010000:3.0:1788619897.354399:0:7654:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime: 1788619896 -> 1788619897 00002000:00010000:3.0:1788619897.354408:0:7654:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619894 -> 1788619897 00002000:00010000:3.0:1788619897.354410:0:7654:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime: 1788619896 -> 1788619897 00000100:00100000:3.0:1788619897.354447:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c7432680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503588981760:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619897.354489:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ad29727f/0x1cd7fe5368732e57 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a5b0 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.354515:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad29727f/0x1cd7fe5368732e57 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a5b0 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.354528:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c000, rc 0) 00000100:00100000:3.0:1788619897.354591:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6531f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571204352:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3118us (3219us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619897.354600:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 262 00000100:00100000:3.0:1788619897.368933:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619897.378509:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619897.393010:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c7433100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503588982784:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619897.393053:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000095a50c3e/0x1cd7fe5368732e8f lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a5e8 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.393080:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000095a50c3e/0x1cd7fe5368732e8f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a5e8 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.393093:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d600, rc 0) 00000100:00100000:3.0:1788619897.393175:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8594380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571205376:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6345us (6484us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619897.393184:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 266 00000100:00100000:3.0:1788619897.398740:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619897.406898:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571205888 00000100:00100000:3.0:1788619897.406919:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571205888 00000100:00100000:3.0:1788619897.406985:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 268 00000100:00100000:3.0:1788619897.406995:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6533100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571205888:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619897.407014:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619897.407033:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008101b10a/0x1cd7fe5368732ea4 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1a604 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619897.407095:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fe300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503588983168:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619897.407189:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619897.411243:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fe300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503588983168:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619897.411293:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008101b10a/0x1cd7fe5368732ea4 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a604 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.411319:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008101b10a/0x1cd7fe5368732ea4 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a604 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.411361:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5049a00, rc 0) 00000100:00100000:3.0:1788619897.411435:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6533100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571205888:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4439us (4662us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619897.411443:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 268 00000100:00100000:3.0:1788619897.412485:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c7432300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503588983296:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619897.412509:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 807243 00010000:00010000:3.0:1788619897.412526:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368732eab lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a5fd expref: 15 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.412564:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619897.412567:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619897.412572:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368732eab lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a5fd expref: 15 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.412585:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d400, rc 0) 00000100:00100000:3.0:1788619897.412627:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6530700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+14:2355:x1875503571205760:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4801us (5797us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619897.412635:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 461 00000100:00100000:3.0:1788619897.435001:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619897.438544:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6532300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503588983808:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619897.438585:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000078ef3fe/0x1cd7fe5368732ec7 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a620 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.438614:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000078ef3fe/0x1cd7fe5368732ec7 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a620 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.438628:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b51600, rc 0) 00000100:00100000:3.0:1788619897.438688:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010fe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571206528:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3400us (3695us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619897.438698:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 270 00000100:00100000:3.0:1788619897.442860:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571206656 00000100:00100000:3.0:1788619897.442869:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571206656 00000100:00100000:3.0:1788619897.442929:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 464 00000100:00100000:3.0:1788619897.442965:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7431880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+14:2355:x1875503571206656:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619897.442987:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619897.443007:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049a00/0x1cd7fe5368732ece lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1a627 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.443041:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6200600/0x1cd7fe5368732e3b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a59b expref: 20 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619897.443083:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fc380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503588983936:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619897.443182:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619897.447919:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571206784 00000100:00100000:3.0:1788619897.447929:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571206784 00000100:00100000:3.0:1788619897.447991:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 271 00000100:00100000:3.0:1788619897.448001:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6532300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571206784:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619897.448016:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619897.448032:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000057586f14/0x1cd7fe5368732ed5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1a62e expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619897.448086:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503588984064:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619897.448177:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619897.451053:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503588984064:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619897.451091:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000057586f14/0x1cd7fe5368732ed5 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a62e expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.451115:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000057586f14/0x1cd7fe5368732ed5 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a62e expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.451153:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5048000, rc 0) 00000100:00100000:3.0:1788619897.451208:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6532300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571206784:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3208us (4489us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619897.451217:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 271 00000100:00100000:3.0:1788619897.456849:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571207040 00000100:00100000:3.0:1788619897.456859:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571207040 00000100:00100000:3.0:1788619897.456914:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 272 00000100:00100000:3.0:1788619897.456924:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6002680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571207040:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619897.456943:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619897.456971:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005613e3c8/0x1cd7fe5368732edc lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1a63c expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619897.457036:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503588984192:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619897.457140:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619897.461803:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503588984192:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619897.461843:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005613e3c8/0x1cd7fe5368732edc lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a63c expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.461867:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005613e3c8/0x1cd7fe5368732edc lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a63c expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.461880:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5048c00, rc 0) 00000100:00100000:3.0:1788619897.461941:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6002680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571207040:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5018us (5371us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619897.461973:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 272 00000100:00100000:3.0:1788619897.487553:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571207680 00000100:00100000:3.0:1788619897.487561:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571207680 00000100:00100000:3.0:1788619897.487630:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 184 00000100:00100000:3.0:1788619897.487641:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6001f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+15:2355:x1875503571207680:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:3.0:1788619897.487879:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 807243 -> 807243 00000004:00010000:3.0:1788619897.487883:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619897 -> 1788619897 00000004:00010000:3.0:1788619897.487886:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619897 -> 1788619897 00000004:00010000:3.0:1788619897.487888:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619897 -> 1788619897 00000004:00010000:3.0:1788619897.487890:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 774 -> 774 00000100:00100000:3.0:1788619897.488152:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6001f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571207680:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 512us (728us total) trans 4294967500 rc 0/0 00000100:00100000:3.0:1788619897.488161:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 184 00000100:00100000:3.0:1788619897.496661:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571207808 00000100:00100000:3.0:1788619897.496671:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571207808 00000100:00100000:3.0:1788619897.496731:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 274 00000100:00100000:3.0:1788619897.496743:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6000000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571207808:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619897.496765:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619897.496784:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005613e3c8/0x1cd7fe5368732eff lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1a658 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619897.496850:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503588984960:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619897.496941:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619897.499797:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503588984960:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619897.499853:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005613e3c8/0x1cd7fe5368732eff lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a658 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.499879:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005613e3c8/0x1cd7fe5368732eff lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a658 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.499892:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5048c00, rc 0) 00000100:00100000:3.0:1788619897.499952:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6000000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571207808:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3210us (3384us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619897.499961:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 274 00000100:00100000:3.0:1788619897.503576:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571207936 00000100:00100000:3.0:1788619897.503587:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571207936 00000100:00100000:3.0:1788619897.503645:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 275 00000100:00100000:3.0:1788619897.503656:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7430000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571207936:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619897.503675:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619897.503693:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000057586f14/0x1cd7fe5368732f06 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1a65f expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619897.503751:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503588985088:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619897.503840:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619897.506370:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503588985088:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619897.506422:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000057586f14/0x1cd7fe5368732f06 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a65f expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.506450:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000057586f14/0x1cd7fe5368732f06 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a65f expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.506477:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5048000, rc 0) 00000100:00100000:3.0:1788619897.506542:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7430000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571207936:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2887us (3058us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619897.506550:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 275 00000100:00100000:3.0:1788619897.510168:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571208064 00000100:00100000:3.0:1788619897.510179:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571208064 00000100:00100000:3.0:1788619897.510234:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 276 00000100:00100000:3.0:1788619897.510244:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6000380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571208064:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619897.510270:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619897.510288:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005613e3c8/0x1cd7fe5368732f0d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1a666 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619897.510381:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503588985216:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619897.510470:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619897.544057:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571208448 00000100:00100000:3.0:1788619897.544067:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571208448 00000100:00100000:3.0:1788619897.544133:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 279 00000100:00100000:3.0:1788619897.544146:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7432300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571208448:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619897.544166:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619897.544185:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000057586f14/0x1cd7fe5368732f22 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1a67b expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619897.544263:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fd180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503588985600:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619897.544383:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c780 going to sleep for 11 seconds 00000100:00100000:3.0:1788619897.557052:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571208704 00000100:00100000:3.0:1788619897.557059:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571208704 00010000:00010000:3.0:1788619897.557095:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368732ef8 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a651 expref: 16 pid: 5844 timeout: 490 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619897.557142:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 159 00000100:00100000:3.0:1788619897.557152:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c80eb850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571208704:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619897.557165:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619897.557191:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368732ef8 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a651 expref: 16 pid: 5844 timeout: 490 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.557205:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368732ef8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a651 expref: 16 pid: 5844 timeout: 490 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.557237:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368732ef8 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a651 expref: 16 pid: 5844 timeout: 490 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.557254:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619897.557636:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368732f29 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a682 expref: 20 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.557649:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368732f29 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a682 expref: 20 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.557660:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368732f29 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a682 expref: 20 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.557702:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368732f29 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a682 expref: 20 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619897.557731:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fdf80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503588985856:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619897.557835:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48ca80 going to sleep for 11 seconds 00000100:00100000:3.0:1788619897.559943:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fdf80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503588985856:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619897.559969:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619897.560059:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c80eb850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+15:2355:x1875503571208704:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 2907us (3081us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619897.560071:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 159 00000100:00100000:3.0:1788619897.604513:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571209088 00000100:00100000:3.0:1788619897.604524:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571209088 00000100:00100000:3.0:1788619897.604581:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 280 00000100:00100000:3.0:1788619897.604592:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8595500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571209088:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619897.604611:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619897.604628:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b0fcf257/0x1cd7fe5368732f30 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1a690 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619897.604689:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503588985984:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619897.604789:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619897.609660:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503588985984:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619897.609703:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b0fcf257/0x1cd7fe5368732f30 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a690 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.609748:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b0fcf257/0x1cd7fe5368732f30 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a690 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.609782:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5049200, rc 0) 00000100:00100000:3.0:1788619897.609851:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8595500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571209088:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5260us (5476us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619897.609860:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 280 00000100:00100000:3.0:1788619897.645975:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571209728 00000100:00100000:3.0:1788619897.645987:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571209728 00010000:00010000:3.0:1788619897.646031:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049e00/0x1cd7fe5368732dfc lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a55c expref: 21 pid: 5846 timeout: 490 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619897.646070:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 160 00000100:00100000:3.0:1788619897.646080:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cd3c8700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2355:x1875503571209728:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619897.646096:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619897.646106:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049e00/0x1cd7fe5368732dfc lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a55c expref: 21 pid: 5846 timeout: 490 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.646132:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049e00/0x1cd7fe5368732dfc lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a55c expref: 21 pid: 5846 timeout: 490 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.646156:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619897.646161:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368732f4c lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.646172:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368732f4c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.646180:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368732f4c lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.646219:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619897.646271:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cd3c8700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2355:x1875503571209728:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 192us (472us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619897.646278:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 160 00000100:00100000:3.0:1788619897.656844:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571209984 00000100:00100000:3.0:1788619897.656855:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571209984 00002000:00100000:3.0:1788619897.656918:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78cd3ca680 x1875503571209984/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:748/0 lens 504/0 e 0 to 0 dl 1788619908 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619897.656929:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78cd3ca680 with x1875503571209984 00010000:00010000:3.0:1788619897.656937:0:5858:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368732f29 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a682 expref: 20 pid: 5846 timeout: 490 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.656961:0:5858:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368732f29 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a682 expref: 20 pid: 5846 timeout: 490 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788619897.656969:0:5858:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e78cd3ca680 00000100:00100000:3.0:1788619897.656998:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 12 00000100:00100000:3.0:1788619897.657007:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cd3ca680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2354:x1875503571209984:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:3.0:1788619897.668755:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 807243 -> 807243 00000004:00010000:3.0:1788619897.668765:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619897 -> 1788619897 00000004:00010000:3.0:1788619897.668768:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619897 -> 1788619897 00000004:00010000:3.0:1788619897.668770:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619897 -> 1788619897 00000004:00010000:3.0:1788619897.668772:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 774 -> 774 00000100:00100000:3.0:1788619897.670301:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cd3ca680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2354:x1875503571209984:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 13282us (13638us total) trans 4294967503 rc 0/0 00000100:00100000:3.0:1788619897.670347:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 12 00002000:00100000:3.0:1788619897.670363:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78cd3ca680 x1875503571209984/t4294967503(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:748/0 lens 504/448 e 0 to 0 dl 1788619908 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619897.670374:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78cd3ca680 with x1875503571209984 00010000:00010000:3.0:1788619897.670397:0:5858:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368732f29 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a682 expref: 22 pid: 5846 timeout: 490 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.670417:0:5858:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368732f29 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a682 expref: 22 pid: 5846 timeout: 490 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788619897.670425:0:5858:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e78cd3ca680 00000100:00100000:3.0:1788619897.674629:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571210112 00000100:00100000:3.0:1788619897.674640:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571210112 00010000:00010000:3.0:1788619897.674678:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368732f29 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a682 expref: 22 pid: 5846 timeout: 490 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619897.674729:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 161 00000100:00100000:3.0:1788619897.674741:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cd3c8e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2355:x1875503571210112:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619897.674755:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619897.674781:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368732f29 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a682 expref: 22 pid: 5846 timeout: 490 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.674795:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368732f29 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a682 expref: 22 pid: 5846 timeout: 490 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.674816:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368732f29 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a682 expref: 22 pid: 5846 timeout: 490 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.674846:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619897.674853:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50a00/0x1cd7fe5368732f53 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a6a5 expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.674865:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50a00/0x1cd7fe5368732f53 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a6a5 expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.674873:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50a00/0x1cd7fe5368732f53 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a6a5 expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.674913:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50a00/0x1cd7fe5368732f53 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a6a5 expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619897.674927:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fdf80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503588986752:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619897.675011:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619897.678635:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fdf80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503588986752:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619897.678661:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619897.678731:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cd3c8e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2355:x1875503571210112:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3992us (4264us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619897.678742:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 161 00000100:00100000:3.0:1788619897.689895:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571210368 00000100:00100000:3.0:1788619897.689906:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571210368 00010000:00010000:3.0:1788619897.689944:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000028304b66/0x1cd7fe5368732d38 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a4b4 expref: 6 pid: 7654 timeout: 490 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619897.690012:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 162 00000100:00100000:3.0:1788619897.690022:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6001880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571210368:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619897.690049:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619897.690072:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000028304b66/0x1cd7fe5368732d38 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a4b4 expref: 6 pid: 7654 timeout: 490 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.690086:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000028304b66/0x1cd7fe5368732d38 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a4b4 expref: 6 pid: 7654 timeout: 490 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.690109:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000028304b66/0x1cd7fe5368732d38 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a4b4 expref: 6 pid: 7654 timeout: 490 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.690135:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(5 12) 00010000:00010000:3.0:1788619897.690140:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(5 12) 00010000:00010000:3.0:1788619897.690149:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000003b84bbc9/0x1cd7fe5368732f5a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a6ac expref: 5 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.690160:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003b84bbc9/0x1cd7fe5368732f5a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a6ac expref: 5 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.690199:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000003b84bbc9/0x1cd7fe5368732f5a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a6ac expref: 5 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619897.690225:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fdf80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503588987008:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619897.690668:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619897.695085:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fdf80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503588987008:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619897.695111:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619897.695179:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6001880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571210368:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 5158us (5582us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619897.695190:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 162 00000100:00100000:3.0:1788619897.706761:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cd3c9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+22:14908:x1875503571210624:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3683us (3974us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619897.706775:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 473 00000100:00100000:3.0:1788619897.732890:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571211136 00000100:00100000:3.0:1788619897.732900:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571211136 00000100:00100000:3.0:1788619897.732967:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 284 00000100:00100000:3.0:1788619897.732977:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cd3ca300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571211136:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619897.732996:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619897.733014:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b0fcf257/0x1cd7fe5368732f84 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1a6cf expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619897.733072:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503588987520:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619897.733174:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619897.741158:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503588987520:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619897.741206:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b0fcf257/0x1cd7fe5368732f84 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a6cf expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.741260:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b0fcf257/0x1cd7fe5368732f84 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a6cf expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.741276:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5049200, rc 0) 00000100:00100000:3.0:1788619897.741363:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cd3ca300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571211136:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8386us (8687us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619897.741373:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 284 00000100:00100000:3.0:1788619897.747014:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571211392 00000100:00100000:3.0:1788619897.747025:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571211392 00000100:00100000:3.0:1788619897.747159:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 476 00000100:00100000:3.0:1788619897.747170:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cd3c8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+22:14908:x1875503571211392:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:3.0:1788619897.747224:0:5844:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619897.747252:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368732f68 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a6b3 expref: 22 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.747270:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619897.747275:0:5844:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788619897.747805:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048a00/0x1cd7fe5368732f8b lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.747822:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048a00/0x1cd7fe5368732f8b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.747832:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048a00/0x1cd7fe5368732f8b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788619897.747923:0:5844:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967504 00000004:00010000:3.0:1788619897.747949:0:5844:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 807243 -> 807243 00000004:00010000:3.0:1788619897.747953:0:5844:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619897 -> 1788619897 00000004:00010000:3.0:1788619897.747955:0:5844:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619897 -> 1788619897 00000004:00010000:3.0:1788619897.747957:0:5844:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619897 -> 1788619897 00000004:00010000:3.0:1788619897.747960:0:5844:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 774 -> 774 00000004:00010000:3.0:1788619897.747966:0:5844:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000007a7f7959 reply state 000000004b681bc5 transno 4294967505 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048a00/0x1cd7fe5368732f8b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619897.747975:0:5844:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788619897.748320:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cd3c8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+24:14908:x1875503571211392:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 1149us (1449us total) trans 4294967505 rc 0/0 00000100:00100000:3.0:1788619897.748348:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 476 00000020:00080000:3.0:1788619897.748353:0:5844:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:3.0:1788619897.752193:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048a00/0x1cd7fe5368732f8b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.752207:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048a00/0x1cd7fe5368732f8b lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.752212:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048a00/0x1cd7fe5368732f8b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.752216:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c5048a00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:3.0:1788619897.752220:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048a00/0x1cd7fe5368732f8b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.752225:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048a00/0x1cd7fe5368732f8b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.752255:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619897.752257:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788619897.752261:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048a00/0x1cd7fe5368732f8b lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.752265:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048a00/0x1cd7fe5368732f8b lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619897.764517:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571211648 00000100:00100000:3.0:1788619897.764526:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571211648 00010000:00010000:3.0:1788619897.764565:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50a00/0x1cd7fe5368732f53 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a6a5 expref: 17 pid: 5845 timeout: 490 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619897.764608:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 163 00000100:00100000:3.0:1788619897.764617:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe002680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+17:2355:x1875503571211648:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619897.764630:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619897.764655:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50a00/0x1cd7fe5368732f53 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a6a5 expref: 17 pid: 5845 timeout: 490 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.764668:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50a00/0x1cd7fe5368732f53 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a6a5 expref: 17 pid: 5845 timeout: 490 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.764688:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50a00/0x1cd7fe5368732f53 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a6a5 expref: 17 pid: 5845 timeout: 490 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.764702:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619897.764706:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50e00/0x1cd7fe5368732f92 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a6d6 expref: 23 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.764730:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50e00/0x1cd7fe5368732f92 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a6d6 expref: 23 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.764739:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50e00/0x1cd7fe5368732f92 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a6d6 expref: 23 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.764774:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50e00/0x1cd7fe5368732f92 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a6d6 expref: 23 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619897.764788:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fdf80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503588987776:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619897.764852:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619897.769086:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fdf80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503588987776:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619897.769130:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619897.769199:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe002680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571211648:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 4582us (5748us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619897.769211:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 163 00000100:00100000:3.0:1788619897.773724:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571211776 00000100:00100000:3.0:1788619897.773734:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571211776 00000100:00100000:3.0:1788619897.773795:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 285 00000100:00100000:3.0:1788619897.773806:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6000e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:14908:x1875503571211776:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619897.773824:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619897.773842:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bf930735/0x1cd7fe5368732f99 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1a6dd expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.773875:0:6696:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000003b84bbc9/0x1cd7fe5368732f5a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a6ac expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.773899:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(5 12) 00010000:00010000:3.0:1788619897.773910:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(6 12) 00010000:00010000:3.0:1788619897.773916:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bf930735/0x1cd7fe5368732f99 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a6dd expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.773965:0:6696:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000003b84bbc9/0x1cd7fe5368732f5a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a6ac expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.773979:0:6696:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000003b84bbc9/0x1cd7fe5368732f5a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a6ac expref: 6 pid: 7654 timeout: 490 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619897.773994:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503588987904:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619897.774083:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619897.778446:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503588987904:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619897.778490:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bf930735/0x1cd7fe5368732f99 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a6dd expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.778512:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5048a00, rc 0) 00000100:00100000:3.0:1788619897.778921:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6000e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:14908:x1875503571211776:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5114us (5355us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619897.778931:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 285 00000100:00100000:3.0:1788619897.808561:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571212416 00000100:00100000:3.0:1788619897.808572:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571212416 00002000:00100000:3.0:1788619897.808646:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1765376->1769471) req@ffff8e78fe001180 x1875503571212416/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:748/0 lens 488/0 e 0 to 0 dl 1788619908 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619897.808659:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fe001180 with x1875503571212416 ext(1765376->1769471) 00002000:00010000:3.0:1788619897.808664:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fe001180 00000100:00100000:3.0:1788619897.808689:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 155 00000100:00100000:3.0:1788619897.808697:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe001180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571212416:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619897.808951:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe001180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571212416:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 246us (539us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619897.808959:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 155 00002000:00100000:3.0:1788619897.808966:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1765376->1769471) req@ffff8e78fe001180 x1875503571212416/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:748/0 lens 488/4536 e 0 to 0 dl 1788619908 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619897.808973:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fe001180 with x1875503571212416 ext(1765376->1769471) 00002000:00010000:3.0:1788619897.808976:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fe001180 00000100:00100000:3.0:1788619897.819386:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619897.833541:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571212800 00000100:00100000:3.0:1788619897.833551:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571212800 00002000:00100000:3.0:1788619897.833616:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1777664->1781759) req@ffff8e78df19f480 x1875503571212800/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:748/0 lens 488/0 e 0 to 0 dl 1788619908 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619897.833627:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df19f480 with x1875503571212800 ext(1777664->1781759) 00002000:00010000:3.0:1788619897.833633:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df19f480 00000100:00100000:3.0:1788619897.833660:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 158 00000100:00100000:3.0:1788619897.833671:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2353:x1875503571212800:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619897.833942:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2353:x1875503571212800:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 273us (557us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619897.833967:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 158 00002000:00100000:3.0:1788619897.833975:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1777664->1781759) req@ffff8e78df19f480 x1875503571212800/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:748/0 lens 488/4536 e 0 to 0 dl 1788619908 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619897.833984:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df19f480 with x1875503571212800 ext(1777664->1781759) 00002000:00010000:3.0:1788619897.833987:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df19f480 00000100:00100000:3.0:1788619897.841782:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619897.921681:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe002d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571213312:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 53032us (54167us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619897.921693:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 286 00000100:00100000:3.0:1788619897.941275:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571213696 00000100:00100000:3.0:1788619897.941286:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571213696 00000100:00100000:3.0:1788619897.957955:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571214208 00000100:00100000:3.0:1788619897.957965:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571214208 00000100:00100000:3.0:1788619897.958023:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 288 00000100:00100000:3.0:1788619897.958043:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571214208:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619897.958065:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619897.958085:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b0fcf257/0x1cd7fe5368732fca lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1a707 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619897.958158:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503588988416:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619897.958246:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619897.963970:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503588988416:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619897.964006:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b0fcf257/0x1cd7fe5368732fca lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a707 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.964053:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b0fcf257/0x1cd7fe5368732fca lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a707 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.964063:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5049200, rc 0) 00000100:00100000:3.0:1788619897.964134:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571214208:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6100us (6273us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619897.964142:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 288 00000100:00100000:3.0:1788619897.978489:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571214464 00000100:00100000:3.0:1788619897.978504:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571214464 00000100:00100000:3.0:1788619897.978568:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 289 00000100:00100000:3.0:1788619897.978579:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe13cc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571214464:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619897.978793:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619897.978814:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b0fcf257/0x1cd7fe5368732fdf lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1a715 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619897.978881:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503588988800:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619897.979033:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619897.980310:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c7430a80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503588988672:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619897.980368:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 807243 00010000:00010000:3.0:1788619897.980398:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368732fd8 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a70e expref: 18 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.980419:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619897.980423:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619897.980445:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368732fd8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a70e expref: 18 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.980461:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683de00, rc 0) 00000100:00100000:3.0:1788619897.980520:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+17:2354:x1875503571214336:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5932us (6198us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619897.980529:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 482 00000100:00100000:3.0:1788619897.984889:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503588988800:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619897.984946:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b0fcf257/0x1cd7fe5368732fdf lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a715 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.984972:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b0fcf257/0x1cd7fe5368732fdf lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a715 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619897.984986:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5049200, rc 0) 00000100:00100000:3.0:1788619897.985055:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe13cc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571214464:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6476us (10836us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619897.985064:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 289 00000100:00100000:3.0:1788619898.001172:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571214848 00000100:00100000:3.0:1788619898.001180:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571214848 00002000:00100000:3.0:1788619898.001302:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df19c380 x1875503571214848/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:749/0 lens 488/0 e 0 to 0 dl 1788619909 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619898.001314:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df19c380 with x1875503571214848 00000100:00100000:3.0:1788619898.001393:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 187 00000100:00100000:3.0:1788619898.001405:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+18:2354:x1875503571214848:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619898.001631:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+18:2354:x1875503571214848:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 228us (558us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619898.001639:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 187 00002000:00100000:3.0:1788619898.001645:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df19c380 x1875503571214848/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:749/0 lens 488/4536 e 0 to 0 dl 1788619909 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619898.001653:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df19c380 with x1875503571214848 00000100:00100000:3.0:1788619898.014074:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571215232 00000100:00100000:3.0:1788619898.014084:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571215232 00002000:00100000:3.0:1788619898.014141:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df8b2a00 x1875503571215232/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:749/0 lens 488/0 e 0 to 0 dl 1788619909 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619898.014152:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df8b2a00 with x1875503571215232 00000100:00100000:3.0:1788619898.014193:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 190 00000100:00100000:3.0:1788619898.014202:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+18:2355:x1875503571215232:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619898.014493:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+18:2355:x1875503571215232:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 291us (519us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619898.014502:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 190 00002000:00100000:3.0:1788619898.014509:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df8b2a00 x1875503571215232/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:749/0 lens 488/4536 e 0 to 0 dl 1788619909 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619898.014516:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df8b2a00 with x1875503571215232 00000100:00100000:3.0:1788619898.030566:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571215616 00000100:00100000:3.0:1788619898.030576:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571215616 00000100:00080000:3.0:1788619898.054985:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1788619898.078815:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571216512 00000100:00100000:3.0:1788619898.078824:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571216512 00000100:00100000:3.0:1788619898.078902:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 485 00000100:00100000:3.0:1788619898.078930:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+25:2353:x1875503571216512:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619898.079061:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619898.079081:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049200/0x1cd7fe5368733002 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1a738 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.079131:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4bc00/0x1cd7fe5368732ffb lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a731 expref: 18 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619898.079170:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fe300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503588989440:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619898.079255:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c180 going to sleep for 11 seconds 00000100:00100000:3.0:1788619898.081862:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fe300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503588989440:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619898.081887:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 807243 00010000:00010000:3.0:1788619898.081904:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049200/0x1cd7fe5368733002 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a738 expref: 26 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.081924:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619898.081927:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619898.081932:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049200/0x1cd7fe5368733002 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a738 expref: 26 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.081959:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5049200, rc 0) 00000100:00100000:3.0:1788619898.082014:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+25:2353:x1875503571216512:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3087us (3286us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619898.082022:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 485 00000100:00100000:3.0:1788619898.105696:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571217024 00000100:00100000:3.0:1788619898.105704:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571217024 00010000:00010000:3.0:1788619898.105745:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51800/0x1cd7fe5368732fa7 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a6e4 expref: 25 pid: 5845 timeout: 490 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619898.105806:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 166 00000100:00100000:3.0:1788619898.105818:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c80e9050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+25:2355:x1875503571217024:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619898.105839:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619898.105846:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51800/0x1cd7fe5368732fa7 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a6e4 expref: 25 pid: 5845 timeout: 490 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.105858:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51800/0x1cd7fe5368732fa7 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a6e4 expref: 25 pid: 5845 timeout: 490 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.105881:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619898.105899:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368733010 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.105920:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368733010 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.105928:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368733010 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.105953:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619898.106008:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c80e9050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+25:2355:x1875503571217024:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 192us (411us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619898.106015:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 166 00000100:00100000:3.0:1788619898.112942:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571217152 00000100:00100000:3.0:1788619898.112952:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571217152 00000100:00100000:3.0:1788619898.113010:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 292 00000100:00100000:3.0:1788619898.113021:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:14908:x1875503571217152:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619898.113053:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619898.113071:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b0fcf257/0x1cd7fe5368733017 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1a746 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.113102:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000bf930735/0x1cd7fe5368732f99 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a6dd expref: 5 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.113125:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(6 12) 00010000:00010000:3.0:1788619898.113129:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(7 12) 00010000:00010000:3.0:1788619898.113136:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b0fcf257/0x1cd7fe5368733017 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a746 expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.113174:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000bf930735/0x1cd7fe5368732f99 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a6dd expref: 5 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.113188:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000bf930735/0x1cd7fe5368732f99 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a6dd expref: 5 pid: 6696 timeout: 490 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619898.113204:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fe300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503588989824:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619898.113294:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c300 going to sleep for 11 seconds 00010000:00010000:3.0:1788619898.118312:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bf930735/0x1cd7fe5368732f99 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a6dd expref: 6 pid: 6696 timeout: 490 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619898.118367:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fe300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503588989824:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619898.118404:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b0fcf257/0x1cd7fe5368733017 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a746 expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.118426:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5049200, rc 0) 00000100:00100000:3.0:1788619898.118459:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:14908:x1875503571217152:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5440us (5803us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619898.118466:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 292 00000100:00100000:3.0:1788619898.121960:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900e03100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503588989952:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619898.121998:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619898.122073:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571217280:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 4273us (4607us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619898.122084:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 167 00000100:00100000:3.0:1788619898.125569:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571217408 00000100:00100000:3.0:1788619898.125580:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571217408 00002000:00010000:3.0:1788619898.125625:0:5858:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:3.0:1788619898.125630:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df8b3800 with x1875503571217408 00000100:00100000:3.0:1788619898.125670:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 197 00000100:00100000:3.0:1788619898.125679:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+20:2355:x1875503571217408:12345-192.168.201.5@tcp:10:fsx.0 00000004:00010000:3.0:1788619898.126070:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 807243 -> 410771 00000004:00010000:3.0:1788619898.126074:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619897 -> 1788619897 00000004:00010000:3.0:1788619898.126077:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619897 -> 1788619897 00000004:00010000:3.0:1788619898.126079:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619897 -> 1788619897 00000004:00010000:3.0:1788619898.126081:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 774 -> 774 00000100:00100000:3.0:1788619898.126386:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+21:2355:x1875503571217408:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 705us (927us total) trans 4294967510 rc 0/0 00000100:00100000:3.0:1788619898.126395:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 197 00002000:00010000:3.0:1788619898.126406:0:5858:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:3.0:1788619898.126410:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df8b3800 with x1875503571217408 00000100:00100000:3.0:1788619898.132686:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571217664 00000100:00100000:3.0:1788619898.132694:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571217664 00000100:00100000:3.0:1788619898.132745:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 488 00000100:00100000:3.0:1788619898.132754:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e00700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+24:14908:x1875503571217664:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619898.132772:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619898.132790:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048a00/0x1cd7fe536873301e lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1a74d expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.132863:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048800/0x1cd7fe5368733025 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.132877:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048800/0x1cd7fe5368733025 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.132887:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048800/0x1cd7fe5368733025 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788619898.132911:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 410771 00000004:00200000:3.0:1788619898.132915:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788619898.132932:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048800/0x1cd7fe5368733025 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.132967:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048a00/0x1cd7fe536873301e lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a74d expref: 26 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.132985:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048800/0x1cd7fe5368733025 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a74d expref: 25 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.133012:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619898.133015:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788619898.133019:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5048800, rc 0) 00000100:00100000:3.0:1788619898.133079:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e00700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+25:14908:x1875503571217664:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 326us (530us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619898.133086:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 488 00000100:00100000:3.0:1788619898.142287:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571217920 00000100:00100000:3.0:1788619898.142297:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571217920 00000100:00100000:3.0:1788619898.142397:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 293 00000100:00100000:3.0:1788619898.142408:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571217920:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619898.142425:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619898.142443:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bf930735/0x1cd7fe5368733033 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1a75b expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619898.142505:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fe300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503588990208:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619898.142633:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c300 going to sleep for 11 seconds 00000100:00100000:3.0:1788619898.167318:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c7430000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503588990336:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619898.167448:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe536873303a lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a769 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.167471:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe536873303a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a769 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.167482:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c200, rc 0) 00000100:00100000:3.0:1788619898.167536:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7430700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571218176:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7615us (7706us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619898.167544:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 294 00000100:00100000:3.0:1788619898.173727:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571218432 00000100:00100000:3.0:1788619898.173737:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571218432 00000100:00100000:3.0:1788619898.173793:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 295 00000100:00100000:3.0:1788619898.173805:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7430a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571218432:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619898.173828:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619898.173847:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3e0ac8/0x1cd7fe5368733048 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1a777 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619898.173919:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503588990592:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619898.174015:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c000 going to sleep for 11 seconds 00002000:00010000:3.0:1788619898.177044:0:6698:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size: 0 -> 1217579 00002000:00010000:3.0:1788619898.177053:0:6698:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime: 1788619897 -> 1788619898 00002000:00010000:3.0:1788619898.177057:0:6698:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime: 1788619897 -> 1788619898 00000100:00100000:3.0:1788619898.177084:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503588990592:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619898.177130:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005c3e0ac8/0x1cd7fe5368733048 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a777 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.177155:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3e0ac8/0x1cd7fe5368733048 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a777 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.177169:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5048400, rc 0) 00000100:00100000:3.0:1788619898.177220:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7430a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571218432:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3417us (3575us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619898.177248:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 295 00000100:00100000:3.0:1788619898.184691:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571218688 00000100:00100000:3.0:1788619898.184701:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571218688 00000100:00100000:3.0:1788619898.185889:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c7432680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503588990848:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619898.185931:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 410771 00010000:00010000:3.0:1788619898.185957:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368733056 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a77e expref: 26 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.185976:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619898.185979:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619898.185984:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368733056 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a77e expref: 26 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.185997:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683ce00, rc 0) 00000100:00100000:3.0:1788619898.186061:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+25:2353:x1875503571218560:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2707us (2987us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619898.186070:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 492 00000100:00100000:3.0:1788619898.234965:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619898.253828:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571219584 00000100:00100000:3.0:1788619898.253842:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571219584 00000100:00100000:3.0:1788619898.254121:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 495 00000100:00100000:3.0:1788619898.254135:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7431c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+21:14908:x1875503571219584:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619898.254158:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619898.254180:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048400/0x1cd7fe5368733080 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1a7a8 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.254276:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048c00/0x1cd7fe5368733087 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.254300:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048c00/0x1cd7fe5368733087 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.254313:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048c00/0x1cd7fe5368733087 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788619898.254396:0:5846:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 410771 00000004:00200000:3.0:1788619898.254401:0:5846:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788619898.254422:0:5846:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048c00/0x1cd7fe5368733087 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.254450:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048400/0x1cd7fe5368733080 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a7a8 expref: 23 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.254469:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048c00/0x1cd7fe5368733087 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a7a8 expref: 22 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.254478:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619898.254480:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788619898.254483:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5048c00, rc 0) 00000100:00100000:3.0:1788619898.254538:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7431c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+22:14908:x1875503571219584:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 407us (870us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619898.254546:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 495 00000100:00100000:3.0:1788619898.261607:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571219840 00000100:00100000:3.0:1788619898.261620:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571219840 00000100:00100000:3.0:1788619898.282588:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571220224 00000100:00100000:3.0:1788619898.282601:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571220224 00000100:00100000:3.0:1788619898.282673:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 498 00000100:00100000:3.0:1788619898.282684:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e01500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+25:2352:x1875503571220224:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619898.282724:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619898.282747:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048000/0x1cd7fe53687330aa lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1a7cb expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.282791:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4bc00/0x1cd7fe5368732ffb lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a731 expref: 21 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619898.282830:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fd880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503588992128:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619898.282909:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c000 going to sleep for 11 seconds 00000100:00100000:3.0:1788619898.287075:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fd880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503588992128:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619898.287091:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 410771 00010000:00010000:3.0:1788619898.287105:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048000/0x1cd7fe53687330aa lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a7cb expref: 26 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.287130:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619898.287133:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619898.287138:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048000/0x1cd7fe53687330aa lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a7cb expref: 26 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.287149:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5048000, rc 0) 00000100:00100000:3.0:1788619898.287173:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e01500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+25:2352:x1875503571220224:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4492us (4761us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619898.287180:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 498 00000100:00100000:3.0:1788619898.297533:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619898.297564:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619898.303300:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619898.306099:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619898.314501:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571221376 00000100:00100000:3.0:1788619898.314510:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571221376 00000100:00100000:3.0:1788619898.314555:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 305 00000100:00100000:3.0:1788619898.314565:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e03480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571221376:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619898.314582:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619898.314599:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000057586f14/0x1cd7fe53687330e2 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1a80a expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619898.314653:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503588993152:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619898.315345:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c000 going to sleep for 11 seconds 00000100:00100000:3.0:1788619898.318139:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503588993152:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619898.318180:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000057586f14/0x1cd7fe53687330e2 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a80a expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.318206:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000057586f14/0x1cd7fe53687330e2 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a80a expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.318218:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5048000, rc 0) 00000100:00100000:3.0:1788619898.318313:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e03480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571221376:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3749us (3865us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619898.318322:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 305 00000100:00100000:3.0:1788619898.345477:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571222016 00000100:00100000:3.0:1788619898.345486:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571222016 00000100:00100000:3.0:1788619898.345537:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 505 00000100:00100000:3.0:1788619898.345546:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+25:2354:x1875503571222016:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619898.345582:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619898.345601:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048000/0x1cd7fe536873310c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1a82d expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.345634:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4bc00/0x1cd7fe5368732ffb lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a731 expref: 21 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619898.345672:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fea00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503588993920:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619898.345741:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c000 going to sleep for 11 seconds 00000100:00100000:3.0:1788619898.349106:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fea00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503588993920:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619898.349133:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 410771 00010000:00010000:3.0:1788619898.349147:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048000/0x1cd7fe536873310c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a82d expref: 26 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.349162:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619898.349165:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619898.349170:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048000/0x1cd7fe536873310c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a82d expref: 26 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.349181:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5048000, rc 0) 00000100:00100000:3.0:1788619898.349221:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+25:2354:x1875503571222016:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3676us (3809us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619898.349228:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 505 00000100:00100000:3.0:1788619898.362695:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900e02300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503588994176:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619898.362736:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000d7f6d20/0x1cd7fe536873311a lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a842 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.362784:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d7f6d20/0x1cd7fe536873311a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a842 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.362797:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4b200, rc 0) 00000100:00100000:3.0:1788619898.362856:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571222400:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3830us (4094us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619898.362865:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 309 00000100:00100000:3.0:1788619898.372804:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c7430000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503588994560:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619898.372819:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 410771 00010000:00010000:3.0:1788619898.372832:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe536873312f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a849 expref: 26 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.372846:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619898.372849:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619898.372854:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe536873312f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a849 expref: 26 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.372864:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d800, rc 0) 00000100:00100000:3.0:1788619898.372889:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+25:2354:x1875503571222528:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5031us (5599us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619898.372896:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 507 00000100:00100000:3.0:1788619898.377237:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571222784 00000100:00100000:3.0:1788619898.377247:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571222784 00000100:00100000:3.0:1788619898.377306:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 508 00000100:00100000:3.0:1788619898.377317:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+25:2354:x1875503571222784:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619898.377393:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619898.377417:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048000/0x1cd7fe5368733136 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1a857 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.377458:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4bc00/0x1cd7fe5368732ffb lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a731 expref: 21 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619898.377503:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fea00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503588994688:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619898.377582:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c000 going to sleep for 11 seconds 00000100:00100000:3.0:1788619898.381897:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fea00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503588994688:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619898.381942:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 410771 00010000:00010000:3.0:1788619898.381963:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048000/0x1cd7fe5368733136 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a857 expref: 26 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.381984:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619898.381987:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619898.381992:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048000/0x1cd7fe5368733136 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a857 expref: 26 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.382008:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5048000, rc 0) 00000100:00100000:3.0:1788619898.382120:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+25:2354:x1875503571222784:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4795us (4951us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619898.382128:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 508 00000100:00100000:3.0:1788619898.389527:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571223168 00000100:00100000:3.0:1788619898.389537:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571223168 00000100:00100000:3.0:1788619898.389601:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 312 00000100:00100000:3.0:1788619898.389611:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571223168:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619898.389645:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619898.389664:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000057586f14/0x1cd7fe5368733144 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1a86c expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619898.389720:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503588994944:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619898.389804:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c000 going to sleep for 11 seconds 00000100:00100000:3.0:1788619898.395088:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503588994944:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619898.395143:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000057586f14/0x1cd7fe5368733144 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a86c expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.395172:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000057586f14/0x1cd7fe5368733144 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a86c expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.395186:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5048000, rc 0) 00000100:00100000:3.0:1788619898.395251:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571223168:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5641us (5961us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619898.395260:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 312 00000100:00100000:3.0:1788619898.395632:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c7430000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503588995072:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619898.395649:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 410771 00010000:00010000:3.0:1788619898.395662:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe536873314b lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a865 expref: 26 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.395675:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619898.395678:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619898.395683:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe536873314b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a865 expref: 26 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.395710:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c800, rc 0) 00000100:00100000:3.0:1788619898.395735:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+25:2355:x1875503571223040:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5574us (6505us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619898.395742:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 509 00000100:00100000:3.0:1788619898.400244:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571223296 00000100:00100000:3.0:1788619898.400252:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571223296 00000100:00100000:3.0:1788619898.400285:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 510 00000100:00100000:3.0:1788619898.400293:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+25:2353:x1875503571223296:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619898.400306:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619898.400322:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049e00/0x1cd7fe5368733159 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1a873 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619898.409189:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571223552 00000100:00100000:3.0:1788619898.409200:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571223552 00000100:00100000:3.0:1788619898.409272:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 511 00000100:00100000:3.0:1788619898.409284:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+25:14908:x1875503571223552:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619898.409306:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619898.409364:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048000/0x1cd7fe5368733160 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1a881 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.409411:0:9417:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4bc00/0x1cd7fe5368732ffb lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a731 expref: 21 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.409422:0:9417:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048000/0x1cd7fe5368733160 lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a881 expref: 26 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.409478:0:9417:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4bc00/0x1cd7fe5368732ffb lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a731 expref: 21 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.409492:0:9417:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4bc00/0x1cd7fe5368732ffb lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a731 expref: 21 pid: 5845 timeout: 491 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619898.409506:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fea00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503588995456:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619898.409579:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c000 going to sleep for 11 seconds 00000100:00100000:3.0:1788619898.412900:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fea00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503588995456:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619898.412966:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048000/0x1cd7fe5368733160 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a881 expref: 26 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.412987:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619898.412990:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619898.412995:0:9417:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048000/0x1cd7fe5368733160 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a881 expref: 26 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.413006:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5048000, rc 0) 00000100:00100000:3.0:1788619898.413070:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+26:14908:x1875503571223552:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3788us (3992us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619898.413080:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 511 00000100:00100000:3.0:1788619898.418136:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571223680 00000100:00100000:3.0:1788619898.418147:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571223680 00010000:00010000:3.0:1788619898.418194:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4bc00/0x1cd7fe5368732ffb lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a731 expref: 22 pid: 5845 timeout: 491 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619898.418260:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 168 00000100:00100000:3.0:1788619898.418270:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df964a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+22:2354:x1875503571223680:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619898.418286:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619898.418313:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4bc00/0x1cd7fe5368732ffb lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a731 expref: 22 pid: 5845 timeout: 491 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.418365:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4bc00/0x1cd7fe5368732ffb lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a731 expref: 22 pid: 5845 timeout: 491 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.418402:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4bc00/0x1cd7fe5368732ffb lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a731 expref: 22 pid: 5845 timeout: 491 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.418420:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619898.418426:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048000/0x1cd7fe5368733160 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a881 expref: 25 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.418436:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048000/0x1cd7fe5368733160 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a881 expref: 25 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.418444:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048000/0x1cd7fe5368733160 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a881 expref: 25 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.418482:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048000/0x1cd7fe5368733160 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a881 expref: 25 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619898.418507:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fea00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503588995584:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619898.418592:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c000 going to sleep for 11 seconds 00000100:00100000:3.0:1788619898.422072:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fea00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503588995584:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619898.422102:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619898.422182:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df964a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+21:2354:x1875503571223680:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3911us (4198us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619898.422195:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 168 00000100:00100000:3.0:1788619898.431286:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571223936 00000100:00100000:3.0:1788619898.431297:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571223936 00002000:00100000:3.0:1788619898.431381:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df966680 x1875503571223936/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:749/0 lens 488/0 e 0 to 0 dl 1788619909 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619898.431401:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df966680 with x1875503571223936 00000100:00100000:3.0:1788619898.431441:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 199 00000100:00100000:3.0:1788619898.431451:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df966680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+26:2354:x1875503571223936:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619898.431716:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df966680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+26:2354:x1875503571223936:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 265us (525us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619898.431724:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 199 00002000:00100000:3.0:1788619898.431730:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df966680 x1875503571223936/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:749/0 lens 488/440 e 0 to 0 dl 1788619909 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619898.431738:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df966680 with x1875503571223936 00000100:00100000:3.0:1788619898.436296:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571224064 00000100:00100000:3.0:1788619898.436305:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571224064 00000100:00100000:3.0:1788619898.436408:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 315 00000100:00100000:3.0:1788619898.436421:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df965880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571224064:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619898.436455:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619898.436474:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c3e0ac8/0x1cd7fe536873316e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1a88f expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619898.436532:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503588995840:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619898.436604:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c000 going to sleep for 11 seconds 00000100:00100000:3.0:1788619898.438794:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503588995840:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619898.438834:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005c3e0ac8/0x1cd7fe536873316e lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a88f expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.438860:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c3e0ac8/0x1cd7fe536873316e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a88f expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.438874:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5048400, rc 0) 00000100:00100000:3.0:1788619898.438951:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df965880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571224064:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2532us (2767us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619898.438960:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 315 00000100:00100000:3.0:1788619898.445728:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571224320 00000100:00100000:3.0:1788619898.445738:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571224320 00000100:00100000:3.0:1788619898.445797:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 316 00000100:00100000:3.0:1788619898.445808:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571224320:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619898.445826:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619898.445843:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bf930735/0x1cd7fe5368733175 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1a896 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619898.446044:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503588995968:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619898.446229:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c000 going to sleep for 11 seconds 00000100:00100000:3.0:1788619898.448750:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503588995968:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619898.448786:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bf930735/0x1cd7fe5368733175 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a896 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.448811:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bf930735/0x1cd7fe5368733175 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a896 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.448822:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5048a00, rc 0) 00000100:00100000:3.0:1788619898.448908:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571224320:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3102us (3273us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619898.448917:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 316 00000100:00100000:3.0:1788619898.470341:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503588996992 00000100:00100000:3.0:1788619898.470346:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503588996992 00000100:00100000:3.0:1788619898.470415:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 117 00000100:00100000:3.0:1788619898.470422:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503588996992:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788619898.470468:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503588996992:12345-0@lo:400:kworker.0 Request processed in 47us (3397us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619898.470474:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 117 00000100:00100000:3.0:1788619898.470754:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900e03100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503588996224:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619898.470785:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e3c1224b/0x1cd7fe5368733183 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a8a4 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.470825:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e3c1224b/0x1cd7fe5368733183 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a8a4 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.470837:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4a600, rc 0) 00000100:00100000:3.0:1788619898.470886:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571224832:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5277us (5584us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619898.470893:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 318 00000100:00080000:3.0:1788619898.471684:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1788619898.474902:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571224960 00000100:00100000:3.0:1788619898.474906:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571224960 00000100:00100000:3.0:1788619898.475029:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 319 00000100:00100000:3.0:1788619898.475038:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571224960:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619898.475053:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619898.475074:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bf930735/0x1cd7fe536873318a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1a8ab expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619898.475149:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503588997120:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619898.475233:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c000 going to sleep for 11 seconds 00000100:00100000:3.0:1788619898.481360:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619898.489696:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c0700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503588997248:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619898.489731:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51a00/0x1cd7fe5368733198 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a8b9 expref: 26 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.489749:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619898.489765:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619898.489771:0:5845:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51a00/0x1cd7fe5368733198 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a8b9 expref: 26 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.489782:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b51a00, rc 0) 00000100:00100000:3.0:1788619898.489842:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+26:14908:x1875503571225216:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2474us (2720us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619898.489852:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 513 00000100:00100000:3.0:1788619898.514204:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571225856 00000100:00100000:3.0:1788619898.514215:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571225856 00000100:00100000:3.0:1788619898.514271:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 516 00000100:00100000:3.0:1788619898.514281:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+21:14908:x1875503571225856:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619898.514302:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619898.514322:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048400/0x1cd7fe53687331ad lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1a8ce expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.514368:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51a00/0x1cd7fe5368733198 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a8b9 expref: 26 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.514389:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048400/0x1cd7fe53687331ad lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a8ce expref: 22 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.514427:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51a00/0x1cd7fe5368733198 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a8b9 expref: 26 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.514442:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51a00/0x1cd7fe5368733198 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a8b9 expref: 26 pid: 5845 timeout: 491 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619898.514481:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fe300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503588997760:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619898.514578:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c600 going to sleep for 11 seconds 00000100:00100000:3.0:1788619898.517431:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fe300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503588997760:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619898.517470:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048400/0x1cd7fe53687331ad lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a8ce expref: 22 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.517488:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619898.517492:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619898.517495:0:5845:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048400/0x1cd7fe53687331ad lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a8ce expref: 22 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.517503:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5048400, rc 0) 00000100:00100000:3.0:1788619898.517550:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+22:14908:x1875503571225856:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3272us (3444us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619898.517558:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 516 00000100:00100000:3.0:1788619898.540237:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571226368 00000100:00100000:3.0:1788619898.540242:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571226368 00000100:00100000:3.0:1788619898.540269:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 320 00000100:00100000:3.0:1788619898.540274:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571226368:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619898.540280:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619898.540288:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e7b1808/0x1cd7fe53687331bb lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1a8dc expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619898.540314:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fe300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503588998144:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619898.540521:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c600 going to sleep for 11 seconds 00000100:00100000:3.0:1788619898.542986:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fe300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503588998144:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619898.543036:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005e7b1808/0x1cd7fe53687331bb lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a8dc expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.543059:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e7b1808/0x1cd7fe53687331bb lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a8dc expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.543073:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5049000, rc 0) 00000100:00100000:3.0:1788619898.543320:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571226368:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3045us (3531us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619898.543364:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 320 00000100:00100000:3.0:1788619898.545976:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571226624 00000100:00100000:3.0:1788619898.545985:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571226624 00000100:00100000:3.0:1788619898.546049:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 321 00000100:00100000:3.0:1788619898.546060:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571226624:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619898.546077:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619898.546095:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000039c2d152/0x1cd7fe53687331c2 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1a8ea expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619898.546267:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fe300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503588998272:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619898.546816:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c600 going to sleep for 11 seconds 00000100:00100000:3.0:1788619898.550734:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fe300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503588998272:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619898.550758:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000039c2d152/0x1cd7fe53687331c2 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a8ea expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.550791:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000039c2d152/0x1cd7fe53687331c2 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a8ea expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.550800:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5048600, rc 0) 00000100:00100000:3.0:1788619898.550865:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571226624:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4808us (4939us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619898.550870:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 321 00000100:00100000:3.0:1788619898.553791:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571226752 00000100:00100000:3.0:1788619898.553801:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571226752 00000100:00100000:3.0:1788619898.553863:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 519 00000100:00100000:3.0:1788619898.553875:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+26:2354:x1875503571226752:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619898.553897:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619898.553935:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049000/0x1cd7fe53687331d0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1a8f1 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.553974:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048400/0x1cd7fe53687331ad lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a8ce expref: 21 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619898.554016:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fe300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503588998528:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619898.554117:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c600 going to sleep for 11 seconds 00000100:00100000:3.0:1788619898.559568:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fe300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503588998528:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619898.559598:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 768465 00010000:00010000:3.0:1788619898.559620:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049000/0x1cd7fe53687331d0 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a8f1 expref: 27 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.559659:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619898.559662:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619898.559669:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049000/0x1cd7fe53687331d0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a8f1 expref: 27 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.559685:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5049000, rc 0) 00000100:00100000:3.0:1788619898.559745:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+26:2354:x1875503571226752:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5872us (6025us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619898.559753:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 519 00000100:00100000:3.0:1788619898.567150:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571227136 00000100:00100000:3.0:1788619898.567161:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571227136 00002000:00100000:3.0:1788619898.567219:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c50c1180 x1875503571227136/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:749/0 lens 488/0 e 0 to 0 dl 1788619909 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619898.567231:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c50c1180 with x1875503571227136 00010000:00010000:3.0:1788619898.567240:0:5858:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048400/0x1cd7fe53687331ad lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a8ce expref: 22 pid: 5845 timeout: 491 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.567255:0:5858:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048400/0x1cd7fe53687331ad lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a8ce expref: 22 pid: 5845 timeout: 491 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788619898.567263:0:5858:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e77c50c1180 00000100:00100000:3.0:1788619898.567297:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 13 00000100:00100000:3.0:1788619898.567307:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+22:2354:x1875503571227136:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:3.0:1788619898.572054:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 768465 -> 768465 00000004:00010000:3.0:1788619898.572062:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619898 -> 1788619898 00000004:00010000:3.0:1788619898.572075:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619898 -> 1788619898 00000004:00010000:3.0:1788619898.572077:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619898 -> 1788619898 00000004:00010000:3.0:1788619898.572080:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 774 -> 774 00000100:00100000:3.0:1788619898.572638:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+24:2354:x1875503571227136:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 5323us (5584us total) trans 4294967512 rc 0/0 00000100:00100000:3.0:1788619898.572653:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 13 00002000:00100000:3.0:1788619898.572671:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c50c1180 x1875503571227136/t4294967512(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:749/0 lens 488/448 e 0 to 0 dl 1788619909 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619898.572681:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c50c1180 with x1875503571227136 00010000:00010000:3.0:1788619898.572694:0:5858:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048400/0x1cd7fe53687331ad lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a8ce expref: 24 pid: 5845 timeout: 491 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.572711:0:5858:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048400/0x1cd7fe53687331ad lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a8ce expref: 24 pid: 5845 timeout: 491 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788619898.572718:0:5858:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e77c50c1180 00000100:00100000:3.0:1788619898.577427:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571227264 00000100:00100000:3.0:1788619898.577439:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571227264 00000100:00100000:3.0:1788619898.588179:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619898.592196:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571227520 00000100:00100000:3.0:1788619898.592207:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571227520 00002000:00100000:3.0:1788619898.592257:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5c0ad80 x1875503571227520/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:749/0 lens 488/0 e 0 to 0 dl 1788619909 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619898.592267:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5c0ad80 with x1875503571227520 00000100:00100000:3.0:1788619898.592302:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 206 00000100:00100000:3.0:1788619898.592312:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c0ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+27:2355:x1875503571227520:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619898.592843:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c0ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+27:2355:x1875503571227520:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 531us (758us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619898.592853:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 206 00002000:00100000:3.0:1788619898.592861:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5c0ad80 x1875503571227520/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:749/0 lens 488/4536 e 0 to 0 dl 1788619909 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619898.592870:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5c0ad80 with x1875503571227520 00000100:00100000:3.0:1788619898.614269:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571228160 00000100:00100000:3.0:1788619898.614293:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571228160 00000100:00100000:3.0:1788619898.614416:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 522 00000100:00100000:3.0:1788619898.614428:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c0bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+24:14908:x1875503571228160:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619898.614452:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619898.614456:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1788619898.614511:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687331e5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a906 expref: 24 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.614532:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619898.614537:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788619898.614552:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049000/0x1cd7fe53687331ec lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1a90d expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.614576:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe53687331de lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a8ff expref: 26 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.614598:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049000/0x1cd7fe53687331ec lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a90d expref: 24 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.614651:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe53687331de lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a8ff expref: 26 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.614664:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe53687331de lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a8ff expref: 26 pid: 11397 timeout: 491 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619898.614678:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fd180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503588999040:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619898.614771:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c540 going to sleep for 11 seconds 00000100:00100000:3.0:1788619898.619947:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fd180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503588999040:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619898.619992:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049000/0x1cd7fe53687331ec lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a90d expref: 24 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.620012:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619898.620016:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619898.620020:0:5845:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049000/0x1cd7fe53687331ec lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a90d expref: 24 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.620030:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5049000, rc 0) 00000100:00100000:3.0:1788619898.620111:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c0bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+24:14908:x1875503571228160:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5676us (6979us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619898.620120:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 522 00000100:00100000:3.0:1788619898.623897:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571228288 00000100:00100000:3.0:1788619898.624134:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571228288 00010000:00010000:3.0:1788619898.624177:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe53687331de lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a8ff expref: 27 pid: 11397 timeout: 491 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619898.624303:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 172 00000100:00100000:3.0:1788619898.624314:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c0aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+27:2354:x1875503571228288:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619898.624364:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619898.624404:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe53687331de lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a8ff expref: 27 pid: 11397 timeout: 491 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.624419:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe53687331de lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a8ff expref: 27 pid: 11397 timeout: 491 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.624444:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe53687331de lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a8ff expref: 27 pid: 11397 timeout: 491 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.624462:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619898.624469:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049000/0x1cd7fe53687331ec lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a90d expref: 23 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.624481:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049000/0x1cd7fe53687331ec lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a90d expref: 23 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.624491:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049000/0x1cd7fe53687331ec lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a90d expref: 23 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.624534:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049000/0x1cd7fe53687331ec lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a90d expref: 23 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619898.624562:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fd180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503588999168:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619898.624656:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c540 going to sleep for 11 seconds 00000100:00100000:3.0:1788619898.631718:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fd180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503588999168:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619898.631742:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619898.631794:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c0aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+26:2354:x1875503571228288:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 7482us (7968us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619898.631804:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 172 00000100:00100000:3.0:1788619898.642785:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571228800 00000100:00100000:3.0:1788619898.642801:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571228800 00000100:00100000:3.0:1788619898.652513:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571228928 00000100:00100000:3.0:1788619898.652524:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571228928 00000100:00100000:3.0:1788619898.652584:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 524 00000100:00100000:3.0:1788619898.652594:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c08700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+26:2352:x1875503571228928:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619898.652613:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619898.652633:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048600/0x1cd7fe5368733201 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1a922 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.652670:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049000/0x1cd7fe53687331ec lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a90d expref: 23 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619898.652713:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fd180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503588999552:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619898.652806:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c540 going to sleep for 11 seconds 00000100:00100000:3.0:1788619898.658389:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fd180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503588999552:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619898.658414:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 931584 00010000:00010000:3.0:1788619898.658448:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048600/0x1cd7fe5368733201 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a922 expref: 27 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.658468:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619898.658471:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619898.658475:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048600/0x1cd7fe5368733201 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a922 expref: 27 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.658489:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5048600, rc 0) 00000100:00100000:3.0:1788619898.658548:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c08700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+26:2352:x1875503571228928:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5954us (6163us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619898.658555:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 524 00000100:00100000:3.0:1788619898.662221:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571229312 00000100:00100000:3.0:1788619898.662231:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571229312 00000100:00100000:3.0:1788619898.662358:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 525 00000100:00100000:3.0:1788619898.662371:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+26:14908:x1875503571229312:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:3.0:1788619898.662443:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619898.662473:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048800/0x1cd7fe5368733025 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a74d expref: 26 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.662494:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619898.662500:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788619898.662725:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048e00/0x1cd7fe536873320f lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.662740:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048c00/0x1cd7fe5368733087 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a7a8 expref: 23 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.662768:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048e00/0x1cd7fe536873320f lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.662806:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048c00/0x1cd7fe5368733087 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a7a8 expref: 23 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.662821:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048c00/0x1cd7fe5368733087 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a7a8 expref: 23 pid: 5846 timeout: 491 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619898.662835:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fd180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503588999808:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619898.662941:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c540 going to sleep for 11 seconds 00000100:00100000:3.0:1788619898.665619:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fd180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503588999808:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619898.665645:0:5845:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048e00/0x1cd7fe536873320f lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.667654:0:5845:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048e00/0x1cd7fe536873320f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.667668:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048e00/0x1cd7fe536873320f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.670944:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048e00/0x1cd7fe536873320f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.670963:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048e00/0x1cd7fe536873320f lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.670971:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048e00/0x1cd7fe536873320f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.670993:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c5048e00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:3.0:1788619898.670998:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048e00/0x1cd7fe536873320f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.671008:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048e00/0x1cd7fe536873320f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.671025:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619898.671029:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788619898.671035:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048e00/0x1cd7fe536873320f lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.671042:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048e00/0x1cd7fe536873320f lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619898.675846:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571229696 00000100:00100000:3.0:1788619898.675858:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571229696 00002000:00100000:3.0:1788619898.675925:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78cdcc1c00 x1875503571229696/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:749/0 lens 488/0 e 0 to 0 dl 1788619909 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619898.675945:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78cdcc1c00 with x1875503571229696 00010000:00010000:3.0:1788619898.675953:0:5858:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049000/0x1cd7fe53687331ec lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a90d expref: 23 pid: 5845 timeout: 491 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.675968:0:5858:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049000/0x1cd7fe53687331ec lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a90d expref: 23 pid: 5845 timeout: 491 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788619898.675977:0:5858:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e78cdcc1c00 00000100:00100000:3.0:1788619898.676011:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 14 00000100:00100000:3.0:1788619898.676021:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+23:2355:x1875503571229696:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:3.0:1788619898.681315:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 768465 -> 931584 00000004:00010000:3.0:1788619898.681364:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619898 -> 1788619898 00000004:00010000:3.0:1788619898.681368:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619898 -> 1788619898 00000004:00010000:3.0:1788619898.681379:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619898 -> 1788619898 00000004:00010000:3.0:1788619898.681382:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 774 -> 774 00000100:00100000:3.0:1788619898.681460:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+25:2355:x1875503571229696:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 5435us (5705us total) trans 4294967515 rc 0/0 00000100:00100000:3.0:1788619898.681472:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 14 00002000:00100000:3.0:1788619898.681485:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78cdcc1c00 x1875503571229696/t4294967515(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:749/0 lens 488/448 e 0 to 0 dl 1788619909 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619898.681495:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78cdcc1c00 with x1875503571229696 00010000:00010000:3.0:1788619898.681505:0:5858:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049000/0x1cd7fe53687331ec lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a90d expref: 25 pid: 5845 timeout: 491 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.681521:0:5858:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049000/0x1cd7fe53687331ec lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a90d expref: 25 pid: 5845 timeout: 491 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788619898.681530:0:5858:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e78cdcc1c00 00000100:00100000:3.0:1788619898.690396:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571229952 00000100:00100000:3.0:1788619898.690406:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571229952 00000100:00100000:3.0:1788619898.690463:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 325 00000100:00100000:3.0:1788619898.690474:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:14908:x1875503571229952:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619898.690494:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619898.690513:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000574a4e4c/0x1cd7fe536873321d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1a937 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.690568:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000b0fcf257/0x1cd7fe5368733017 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a746 expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.690578:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=1s v=(7 12) 00010000:00010000:3.0:1788619898.690582:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(8 12) 00010000:00010000:3.0:1788619898.690588:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000574a4e4c/0x1cd7fe536873321d lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a937 expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.690623:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000b0fcf257/0x1cd7fe5368733017 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a746 expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.690637:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000b0fcf257/0x1cd7fe5368733017 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a746 expref: 6 pid: 6697 timeout: 491 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619898.690650:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fd180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589000192:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619898.690733:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c540 going to sleep for 11 seconds 00000100:00100000:3.0:1788619898.696156:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fd180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589000192:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619898.696197:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000574a4e4c/0x1cd7fe536873321d lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a937 expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.696225:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5048e00, rc 0) 00000100:00100000:3.0:1788619898.696304:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:14908:x1875503571229952:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5830us (6469us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619898.696372:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 325 00000100:00100000:3.0:1788619898.698877:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571230080 00000100:00100000:3.0:1788619898.698886:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571230080 00002000:00100000:3.0:1788619898.698946:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1208320->1217578) req@ffff8e78cdcc1500 x1875503571230080/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:749/0 lens 9752/0 e 0 to 0 dl 1788619909 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619898.698958:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78cdcc1500 with x1875503571230080 ext(1208320->1217578) 00010000:00010000:3.0:1788619898.698966:0:6702:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000b0fcf257/0x1cd7fe5368733017 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a746 expref: 7 pid: 6697 timeout: 491 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.698983:0:6702:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000b0fcf257/0x1cd7fe5368733017 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a746 expref: 7 pid: 6697 timeout: 491 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788619898.698991:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff8e78cdcc1500 00000100:00100000:3.0:1788619898.699022:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 3 00000100:00100000:3.0:1788619898.699032:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571230080:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:3.0:1788619898.699536:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571230080:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 504us (741us total) trans 4294967344 rc 0/0 00000100:00100000:3.0:1788619898.699544:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 3 00002000:00100000:3.0:1788619898.699552:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1208320->1217578) req@ffff8e78cdcc1500 x1875503571230080/t4294967344(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:749/0 lens 9752/448 e 0 to 0 dl 1788619909 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619898.699560:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78cdcc1500 with x1875503571230080 ext(1208320->1217578) 00010000:00010000:3.0:1788619898.699566:0:6702:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000b0fcf257/0x1cd7fe5368733017 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a746 expref: 9 pid: 6697 timeout: 491 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.699590:0:6702:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000b0fcf257/0x1cd7fe5368733017 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a746 expref: 9 pid: 6697 timeout: 491 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788619898.699597:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff8e78cdcc1500 00010000:00010000:3.0:1788619898.701626:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788619898.703450:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571230208 00000100:00100000:3.0:1788619898.703459:0:5836:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571230208 00000100:00100000:3.0:1788619898.703512:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 283 00000100:00100000:3.0:1788619898.703523:0:5836:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5ce2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:597450a2-e90c-408c-b401-a0ce48eed204+13:2355:x1875503571230208:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:3.0:1788619898.703583:0:5836:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5ce2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:597450a2-e90c-408c-b401-a0ce48eed204+13:2355:x1875503571230208:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 62us (373us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619898.703591:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 283 00000100:00100000:3.0:1788619898.709100:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571230592 00000100:00100000:3.0:1788619898.709115:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571230592 00002000:00010000:3.0:1788619898.709137:0:5857:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:3.0:1788619898.709141:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78cdcc2300 with x1875503571230592 00000100:00100000:3.0:1788619898.709167:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 212 00000100:00100000:3.0:1788619898.709175:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+28:2354:x1875503571230592:12345-192.168.201.5@tcp:10:fsx.0 00000004:00010000:3.0:1788619898.709417:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 931584 -> 347731 00000004:00010000:3.0:1788619898.709420:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619898 -> 1788619898 00000004:00010000:3.0:1788619898.709423:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619898 -> 1788619898 00000004:00010000:3.0:1788619898.709425:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619898 -> 1788619898 00000004:00010000:3.0:1788619898.709428:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 774 -> 774 00000100:00100000:3.0:1788619898.709473:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+29:2354:x1875503571230592:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 299us (1220us total) trans 4294967516 rc 0/0 00000100:00100000:3.0:1788619898.709497:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 212 00002000:00010000:3.0:1788619898.709500:0:5857:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:3.0:1788619898.709502:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78cdcc2300 with x1875503571230592 00000100:00100000:3.0:1788619898.712467:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571230848 00000100:00100000:3.0:1788619898.712478:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571230848 00000100:00100000:3.0:1788619898.712533:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 527 00000100:00100000:3.0:1788619898.712545:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+29:14908:x1875503571230848:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619898.712571:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619898.712591:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048800/0x1cd7fe5368733224 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1a93e expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.712675:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048600/0x1cd7fe536873322b lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.712686:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048600/0x1cd7fe536873322b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.712696:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048600/0x1cd7fe536873322b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788619898.712722:0:9417:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 347731 00000004:00200000:3.0:1788619898.712726:0:9417:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788619898.712744:0:9417:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048600/0x1cd7fe536873322b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.712769:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048800/0x1cd7fe5368733224 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a93e expref: 31 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.712784:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048600/0x1cd7fe536873322b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a93e expref: 30 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.712803:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619898.712806:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788619898.712810:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5048600, rc 0) 00000100:00100000:3.0:1788619898.712869:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+30:14908:x1875503571230848:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 325us (661us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619898.712876:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 527 00000100:00100000:3.0:1788619898.728283:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fefd4380 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589000576:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619898.728308:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 347731 00010000:00010000:3.0:1788619898.728349:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368733247 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a94c expref: 26 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.728434:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619898.728438:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619898.728446:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368733247 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a94c expref: 26 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.728460:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683cc00, rc 0) 00000100:00100000:3.0:1788619898.728499:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c0aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+25:2352:x1875503571231104:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 9300us (10011us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619898.728506:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 529 00000100:00100000:3.0:1788619898.732528:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571231488 00000100:00100000:3.0:1788619898.732533:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571231488 00000100:00100000:3.0:1788619898.750422:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571231744 00000100:00100000:3.0:1788619898.750433:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571231744 00010000:00010000:3.0:1788619898.750475:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4b400/0x1cd7fe5368733216 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a930 expref: 30 pid: 11397 timeout: 491 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619898.750555:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 176 00000100:00100000:3.0:1788619898.750565:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+30:2354:x1875503571231744:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619898.750811:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619898.750839:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4b400/0x1cd7fe5368733216 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a930 expref: 30 pid: 11397 timeout: 491 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.750854:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4b400/0x1cd7fe5368733216 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a930 expref: 30 pid: 11397 timeout: 491 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.750877:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619898.750883:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51000/0x1cd7fe536873325c lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a968 expref: 26 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.750895:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51000/0x1cd7fe536873325c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a968 expref: 26 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.750904:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51000/0x1cd7fe536873325c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a968 expref: 26 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.751004:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51000/0x1cd7fe536873325c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a968 expref: 26 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619898.751019:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fd180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589001088:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619898.751468:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619898.757828:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fd180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589001088:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619898.757864:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619898.757935:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+29:2354:x1875503571231744:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 7363us (7846us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619898.757946:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 176 00000100:00100000:3.0:1788619898.764539:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571232000 00000100:00100000:3.0:1788619898.764548:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571232000 00000100:00100000:3.0:1788619898.764596:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 330 00000100:00100000:3.0:1788619898.764606:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+8:2353:x1875503571232000:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619898.764624:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619898.764640:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000095315e19/0x1cd7fe536873326a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1a976 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619898.764705:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fd180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589001344:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619898.765114:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619898.767729:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fd180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589001344:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619898.767769:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000095315e19/0x1cd7fe536873326a lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a976 expref: 9 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.767789:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000095315e19/0x1cd7fe536873326a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a976 expref: 9 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.767798:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5048800, rc 0) 00000100:00100000:3.0:1788619898.767845:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+8:2353:x1875503571232000:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3240us (3367us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619898.767854:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 330 00000100:00100000:3.0:1788619898.771706:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571232256 00000100:00100000:3.0:1788619898.771716:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571232256 00000100:00100000:3.0:1788619898.771775:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 532 00000100:00100000:3.0:1788619898.771785:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df964a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+26:14908:x1875503571232256:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:3.0:1788619898.771839:0:11397:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619898.771866:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50e00/0x1cd7fe5368733239 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a945 expref: 26 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.771884:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619898.771889:0:11397:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788619898.772315:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049600/0x1cd7fe5368733271 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.772443:0:11397:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048600/0x1cd7fe536873322b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a93e expref: 28 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.772451:0:11397:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049600/0x1cd7fe5368733271 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619898.781446:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571232512 00000100:00100000:3.0:1788619898.781456:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571232512 00010000:00010000:3.0:1788619898.816440:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe536873328d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.816460:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe536873328d lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.816468:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe536873328d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.816474:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c683c600 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:3.0:1788619898.816499:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe536873328d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.816508:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe536873328d lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.816526:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619898.816530:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788619898.816535:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe536873328d lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.816541:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe536873328d lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619898.818534:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fefd5880 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589001984:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619898.818577:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368733294 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a98b expref: 31 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.818595:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619898.818597:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619898.818601:0:11397:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368733294 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a98b expref: 31 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.818611:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683ce00, rc 0) 00000100:00100000:3.0:1788619898.818656:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5771500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+31:14908:x1875503571233408:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6569us (6748us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619898.818664:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 535 00000100:00100000:3.0:1788619898.826988:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571233664 00000100:00100000:3.0:1788619898.827000:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571233664 00010000:00010000:3.0:1788619898.827145:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51000/0x1cd7fe536873325c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a968 expref: 29 pid: 5845 timeout: 491 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619898.827310:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 180 00000100:00100000:3.0:1788619898.827322:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c80e8c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+29:2354:x1875503571233664:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619898.827369:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619898.827398:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51000/0x1cd7fe536873325c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a968 expref: 29 pid: 5845 timeout: 491 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.827412:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51000/0x1cd7fe536873325c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a968 expref: 29 pid: 5845 timeout: 491 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.827435:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51000/0x1cd7fe536873325c lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a968 expref: 29 pid: 5845 timeout: 491 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.827451:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619898.827457:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368733294 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a98b expref: 30 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.827469:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368733294 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a98b expref: 30 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.827478:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368733294 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a98b expref: 30 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.827518:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368733294 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a98b expref: 30 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619898.827545:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fd180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589002112:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619898.827702:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619898.836772:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fd180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589002112:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619898.836802:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619898.836860:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c80e8c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+28:2354:x1875503571233664:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 9541us (10217us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619898.836870:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 180 00000100:00100000:3.0:1788619898.877847:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619898.884056:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619898.912572:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571234688 00000100:00100000:3.0:1788619898.912583:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571234688 00000100:00100000:3.0:1788619898.912652:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 538 00000100:00100000:3.0:1788619898.912663:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+30:14908:x1875503571234688:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619898.912687:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619898.912707:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048800/0x1cd7fe53687332b7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1a9a7 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.912798:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049800/0x1cd7fe53687332be lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.912817:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049800/0x1cd7fe53687332be lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.912830:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049800/0x1cd7fe53687332be lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788619898.912882:0:11397:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1058733 00000004:00200000:3.0:1788619898.912887:0:11397:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788619898.912906:0:11397:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049800/0x1cd7fe53687332be lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.912934:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048800/0x1cd7fe53687332b7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a9a7 expref: 32 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.912952:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049800/0x1cd7fe53687332be lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a9a7 expref: 31 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.912961:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619898.912964:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788619898.912967:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5049800, rc 0) 00000100:00100000:3.0:1788619898.913031:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+31:14908:x1875503571234688:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 370us (722us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619898.913038:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 538 00000100:00100000:3.0:1788619898.921560:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571234816 00000100:00100000:3.0:1788619898.921575:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571234816 00000100:00100000:3.0:1788619898.921648:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 333 00000100:00100000:3.0:1788619898.921658:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571234816:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619898.921676:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619898.921695:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000095315e19/0x1cd7fe53687332c5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1a9ae expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619898.921758:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fc380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589002752:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619898.921878:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48ce40 going to sleep for 11 seconds 00000100:00100000:3.0:1788619898.925923:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fc380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589002752:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619898.925964:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000095315e19/0x1cd7fe53687332c5 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a9ae expref: 10 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.925991:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000095315e19/0x1cd7fe53687332c5 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a9ae expref: 10 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.926005:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5048800, rc 0) 00000100:00100000:3.0:1788619898.926069:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571234816:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4411us (6458us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619898.926078:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 333 00000100:00100000:3.0:1788619898.938576:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571234944 00000100:00100000:3.0:1788619898.938586:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571234944 00000100:00100000:3.0:1788619898.938650:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 334 00000100:00100000:3.0:1788619898.938661:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571234944:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619898.938681:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619898.938698:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000095315e19/0x1cd7fe53687332cc lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1a9b5 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619898.938756:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fc380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589002880:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619898.938853:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48ce40 going to sleep for 11 seconds 00000100:00100000:3.0:1788619898.951982:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fc380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589002880:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619898.952024:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000095315e19/0x1cd7fe53687332cc lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a9b5 expref: 10 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.952075:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000095315e19/0x1cd7fe53687332cc lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a9b5 expref: 10 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.952090:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5048800, rc 0) 00000100:00100000:3.0:1788619898.952172:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571234944:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 13513us (13706us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619898.952182:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 334 00010000:00010000:0.0:1788619898.963678:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4b200/0x1cd7fe53687332d3 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a9bc expref: 32 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619898.963724:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619898.963728:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619898.963734:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4b200/0x1cd7fe53687332d3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a9bc expref: 32 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619898.963751:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4b200, rc 0) 00000100:00100000:0.0:1788619898.963831:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+31:2354:x1875503571235072:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4872us (5063us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619898.963841:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 539 00000100:00100000:3.0:1788619898.968171:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619898.981220:0:3317:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78cdcc0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3317:1875503589003264:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1788619898.981299:0:3317:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619898.981474:0:6700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589003264 00000100:00100000:3.0:1788619898.981492:0:6700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589003264 00000100:00100000:3.0:1788619898.981569:0:6700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 80 00000100:00100000:3.0:1788619898.981583:0:6700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5772a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3317:x1875503589003264:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1788619898.981631:0:6700:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3856662528 avail 3854565376 00000100:00100000:3.0:1788619898.985347:0:6700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5772a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3317:x1875503589003264:12345-0@lo:13:osp-pre-1-0.0 Request processed in 3738us (4018us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619898.985368:0:6700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 80 00000100:00100000:0.0:1788619898.985777:0:3317:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78cdcc0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3317:1875503589003264:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1788619898.988064:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571235328 00000100:00100000:3.0:1788619898.988084:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571235328 00000100:00100000:3.0:1788619898.988179:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 541 00000100:00100000:3.0:1788619898.988192:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+31:2354:x1875503571235328:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619898.988635:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619898.988664:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048800/0x1cd7fe53687332e1 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1a9ca expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619898.988812:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe53687332b0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a9a0 expref: 30 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619898.988872:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f1f80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589003392:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619898.989003:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c240 going to sleep for 11 seconds 00000100:00100000:3.0:1788619899.035430:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571235712 00000100:00100000:3.0:1788619899.035449:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571235712 00000100:00100000:3.0:1788619899.035542:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 543 00000100:00100000:3.0:1788619899.035555:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7581c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+31:14908:x1875503571235712:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:3.0:1788619899.035633:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619899.035672:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4ae00/0x1cd7fe53687332a9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a999 expref: 31 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619899.035827:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619899.035835:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788619899.036703:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049400/0x1cd7fe53687332ef lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619899.036728:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049800/0x1cd7fe53687332be lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a9a7 expref: 30 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619899.036739:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049400/0x1cd7fe53687332ef lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619899.036787:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049800/0x1cd7fe53687332be lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a9a7 expref: 30 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619899.036833:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049800/0x1cd7fe53687332be lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a9a7 expref: 30 pid: 11397 timeout: 491 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619899.036851:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f1f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589003648:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619899.037381:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48ccc0 going to sleep for 11 seconds 00000020:00000080:3.0:1788619899.040867:0:484:0:(genops.c:844:class_export_destroy()) destroying export ffff8e78cdce2000/92dda994-8428-47f5-a731-0b06f1aba2e4 for lustre-OST0001 00000020:00000080:3.0:1788619899.040943:0:484:0:(genops.c:1183:obd_zombie_import_free()) destroying import ffff8e78cdce1000 for lustre-OST0001 00010000:00010000:3.0:1788619899.046784:0:5845:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049400/0x1cd7fe53687332ef lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619899.046793:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049400/0x1cd7fe53687332ef lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788619899.046959:0:5845:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967523 00000004:00010000:3.0:1788619899.047025:0:5845:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 890767 -> 890767 00000004:00010000:3.0:1788619899.047029:0:5845:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619898 -> 1788619898 00000004:00010000:3.0:1788619899.047032:0:5845:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619898 -> 1788619898 00000004:00010000:3.0:1788619899.047035:0:5845:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619898 -> 1788619898 00000004:00010000:3.0:1788619899.047037:0:5845:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 774 -> 774 00000004:00010000:3.0:1788619899.047046:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000009faf380 reply state 00000000129f692c transno 4294967524 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049400/0x1cd7fe53687332ef lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619899.047057:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788619899.047158:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7581c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+33:14908:x1875503571235712:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 11606us (12555us total) trans 4294967524 rc 0/0 00000100:00100000:3.0:1788619899.047165:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 543 00000020:00080000:3.0:1788619899.047178:0:5845:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1788619899.051551:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619899.061825:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571236096 00000100:00100000:3.0:1788619899.061847:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571236096 00010000:00010000:3.0:1788619899.061925:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe53687332b0 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a9a0 expref: 30 pid: 5845 timeout: 491 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619899.061980:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 184 00000100:00100000:3.0:1788619899.061992:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7583480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+30:2354:x1875503571236096:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619899.062011:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619899.062048:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe53687332b0 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a9a0 expref: 30 pid: 5845 timeout: 491 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619899.062064:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe53687332b0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a9a0 expref: 30 pid: 5845 timeout: 491 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619899.062091:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe53687332b0 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a9a0 expref: 30 pid: 5845 timeout: 491 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619899.062124:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619899.062132:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50800/0x1cd7fe53687332f6 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a9d8 expref: 32 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619899.062147:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50800/0x1cd7fe53687332f6 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a9d8 expref: 32 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619899.062156:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50800/0x1cd7fe53687332f6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a9d8 expref: 32 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619899.062484:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50800/0x1cd7fe53687332f6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a9d8 expref: 32 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619899.062500:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900e02300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589003904:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619899.062606:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c300 going to sleep for 11 seconds 00000100:00100000:3.0:1788619899.069530:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619899.082663:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619899.087487:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619899.087547:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619899.103471:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619899.103873:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571236864 00000100:00100000:3.0:1788619899.103878:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571236864 00000100:00100000:3.0:1788619899.103950:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 336 00000100:00100000:3.0:1788619899.103958:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7582300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571236864:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619899.103966:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619899.103980:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000099f427a5/0x1cd7fe5368733320 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1a9fb expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619899.104052:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df19e680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589004288:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619899.104236:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c000 going to sleep for 11 seconds 00000100:00100000:0.0:1788619899.119044:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571236992 00000100:00100000:0.0:1788619899.119059:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571236992 00000100:00100000:0.0:1788619899.119165:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 214 00000100:00100000:0.0:1788619899.119177:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7583100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+33:2355:x1875503571236992:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:0.0:1788619899.119683:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 890767 -> 890767 00000004:00010000:0.0:1788619899.119751:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619898 -> 1788619898 00000004:00010000:0.0:1788619899.119754:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619898 -> 1788619898 00000004:00010000:0.0:1788619899.119757:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619898 -> 1788619898 00000004:00010000:0.0:1788619899.119759:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 774 -> 774 00000100:00100000:0.0:1788619899.119881:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7583100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+34:2355:x1875503571236992:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 705us (1148us total) trans 4294967525 rc 0/0 00000100:00100000:0.0:1788619899.119890:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 214 00000100:00100000:0.0:1788619899.124764:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619899.125135:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619899.139494:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619899.139597:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619899.143189:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 338 00000100:00100000:0.0:1788619899.143205:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5770700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571237504:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619899.143231:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619899.143543:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e36287d4/0x1cd7fe536873333c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1aa17 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619899.143636:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5771880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589004928:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619899.143912:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78ce2fe900 going to sleep for 11 seconds 00000100:00100000:0.0:1788619899.167947:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619899.168067:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619899.168231:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571237760 00000100:00100000:3.0:1788619899.168246:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571237760 00000100:00100000:3.0:1788619899.168378:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 339 00000100:00100000:3.0:1788619899.168392:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5772d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571237760:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619899.168419:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619899.168463:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008101b10a/0x1cd7fe5368733343 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1aa25 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619899.168552:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df19e680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589005056:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619899.168790:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c240 going to sleep for 11 seconds 00000100:00100000:3.0:1788619899.174305:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df19e680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589005056:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619899.174393:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008101b10a/0x1cd7fe5368733343 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aa25 expref: 10 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619899.174417:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008101b10a/0x1cd7fe5368733343 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aa25 expref: 10 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619899.174430:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5049a00, rc 0) 00000100:00100000:3.0:1788619899.174465:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5772d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571237760:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6076us (6397us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619899.174474:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 339 00000100:00100000:0.0:1788619899.177790:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619899.177836:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619899.197744:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619899.197822:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619899.211084:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619899.218465:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619899.218604:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571238528 00000100:00100000:3.0:1788619899.218618:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571238528 00002000:00100000:3.0:1788619899.218688:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1298432->1339391) req@ffff8e78c5771880 x1875503571238528/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:750/0 lens 488/0 e 0 to 0 dl 1788619910 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619899.218703:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5771880 with x1875503571238528 ext(1298432->1339391) 00010000:00010000:3.0:1788619899.218713:0:6702:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000a72c5200/0x1cd7fe536873329b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a992 expref: 9 pid: 6696 timeout: 491 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619899.218764:0:6702:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000a72c5200/0x1cd7fe536873329b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a992 expref: 9 pid: 6696 timeout: 491 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788619899.218773:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff8e78c5771880 00000100:00100000:3.0:1788619899.218818:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 4 00000100:00100000:3.0:1788619899.218831:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5771880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571238528:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:3.0:1788619899.225145:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619899.226493:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571238656 00000100:00100000:0.0:1788619899.226512:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571238656 00010000:00010000:0.0:1788619899.226581:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000a72c5200/0x1cd7fe536873329b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a992 expref: 11 pid: 6696 timeout: 491 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619899.226636:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 185 00000100:00100000:0.0:1788619899.226648:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed62a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+11:2353:x1875503571238656:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619899.226803:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619899.226833:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000a72c5200/0x1cd7fe536873329b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a992 expref: 11 pid: 6696 timeout: 491 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619899.226871:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000a72c5200/0x1cd7fe536873329b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a992 expref: 11 pid: 6696 timeout: 491 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619899.227007:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a72c5200/0x1cd7fe536873329b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a992 expref: 11 pid: 6696 timeout: 491 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619899.227032:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(11 12) 00010000:00010000:0.0:1788619899.227037:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(11 12) 00010000:00010000:0.0:1788619899.227049:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000485aa315/0x1cd7fe536873336d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1101824->1105919) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aa48 expref: 9 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619899.227062:0:13330:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000485aa315/0x1cd7fe536873336d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1101824->1105919) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aa48 expref: 9 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619899.227131:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000485aa315/0x1cd7fe536873336d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1101824->1105919) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aa48 expref: 9 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619899.227151:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5770380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589005952:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619899.227629:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78ce2fe3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619899.239389:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5770380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589005952:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619899.239425:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619899.239504:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed62a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+10:2353:x1875503571238656:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 12857us (14374us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619899.239519:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 185 00000100:00100000:0.0:1788619899.290704:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571239424 00000100:00100000:0.0:1788619899.290718:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571239424 00002000:00100000:0.0:1788619899.290789:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1122304->1126399) req@ffff8e78c7583800 x1875503571239424/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:750/0 lens 488/0 e 0 to 0 dl 1788619910 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619899.290804:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7583800 with x1875503571239424 ext(1122304->1126399) 00002000:00010000:0.0:1788619899.290811:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c7583800 00000100:00100000:0.0:1788619899.290850:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 173 00000100:00100000:0.0:1788619899.290862:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7583800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571239424:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619899.291226:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7583800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571239424:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 365us (719us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619899.291234:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 173 00002000:00100000:0.0:1788619899.291242:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1122304->1126399) req@ffff8e78c7583800 x1875503571239424/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:750/0 lens 488/4536 e 0 to 0 dl 1788619910 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619899.291251:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7583800 with x1875503571239424 ext(1122304->1126399) 00002000:00010000:0.0:1788619899.291255:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c7583800 00000100:00100000:3.0:1788619899.303385:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571239680 00000100:00100000:3.0:1788619899.303399:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571239680 00002000:00100000:3.0:1788619899.303473:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1130496->1134591) req@ffff8e78c7580a80 x1875503571239680/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:750/0 lens 488/0 e 0 to 0 dl 1788619910 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619899.303487:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7580a80 with x1875503571239680 ext(1130496->1134591) 00002000:00010000:3.0:1788619899.303495:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c7580a80 00000100:00100000:3.0:1788619899.303535:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 175 00000100:00100000:3.0:1788619899.303547:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7580a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571239680:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619899.303926:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7580a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571239680:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 380us (723us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619899.303935:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 175 00002000:00100000:3.0:1788619899.303944:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1130496->1134591) req@ffff8e78c7580a80 x1875503571239680/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:750/0 lens 488/4536 e 0 to 0 dl 1788619910 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619899.303952:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7580a80 with x1875503571239680 ext(1130496->1134591) 00002000:00010000:3.0:1788619899.303957:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c7580a80 00000100:00100000:0.0:1788619899.308539:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571239808 00000100:00100000:0.0:1788619899.308554:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571239808 00002000:00100000:0.0:1788619899.308627:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1134592->1138687) req@ffff8e78c7582d80 x1875503571239808/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:750/0 lens 488/0 e 0 to 0 dl 1788619910 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619899.308641:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7582d80 with x1875503571239808 ext(1134592->1138687) 00002000:00010000:0.0:1788619899.308660:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c7582d80 00000100:00100000:0.0:1788619899.308704:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 176 00000100:00100000:0.0:1788619899.308716:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7582d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571239808:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619899.309064:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7582d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571239808:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 349us (643us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619899.309102:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 176 00002000:00100000:0.0:1788619899.309110:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1134592->1138687) req@ffff8e78c7582d80 x1875503571239808/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:750/0 lens 488/4536 e 0 to 0 dl 1788619910 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619899.309120:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7582d80 with x1875503571239808 ext(1134592->1138687) 00002000:00010000:0.0:1788619899.309124:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c7582d80 00000100:00100000:0.0:1788619899.320725:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571240064 00000100:00100000:0.0:1788619899.320744:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571240064 00002000:00100000:0.0:1788619899.320830:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1142784->1146879) req@ffff8e78c7583b80 x1875503571240064/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:750/0 lens 488/0 e 0 to 0 dl 1788619910 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619899.320846:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7583b80 with x1875503571240064 ext(1142784->1146879) 00002000:00010000:0.0:1788619899.320854:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c7583b80 00000100:00100000:0.0:1788619899.320884:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 178 00000100:00100000:0.0:1788619899.320896:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7583b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571240064:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619899.321294:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7583b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571240064:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 399us (1794us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619899.321304:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 178 00002000:00100000:0.0:1788619899.321311:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1142784->1146879) req@ffff8e78c7583b80 x1875503571240064/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:750/0 lens 488/4536 e 0 to 0 dl 1788619910 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619899.321321:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7583b80 with x1875503571240064 ext(1142784->1146879) 00002000:00010000:0.0:1788619899.321344:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c7583b80 00000100:00100000:0.0:1788619899.326856:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571240192 00000100:00100000:0.0:1788619899.326870:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571240192 00002000:00100000:0.0:1788619899.326947:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1146880->1150975) req@ffff8e78cdcc3100 x1875503571240192/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:750/0 lens 488/0 e 0 to 0 dl 1788619910 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619899.326987:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78cdcc3100 with x1875503571240192 ext(1146880->1150975) 00002000:00010000:0.0:1788619899.326994:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78cdcc3100 00000100:00100000:0.0:1788619899.327029:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 179 00000100:00100000:0.0:1788619899.327039:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571240192:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619899.327426:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571240192:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 386us (718us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619899.327435:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 179 00002000:00100000:0.0:1788619899.327443:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1146880->1150975) req@ffff8e78cdcc3100 x1875503571240192/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:750/0 lens 488/4536 e 0 to 0 dl 1788619910 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619899.327452:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78cdcc3100 with x1875503571240192 ext(1146880->1150975) 00002000:00010000:0.0:1788619899.327456:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78cdcc3100 00000100:00100000:3.0:1788619899.332274:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619899.332597:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571240320 00000100:00100000:0.0:1788619899.332612:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571240320 00002000:00100000:0.0:1788619899.332695:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1150976->1155071) req@ffff8e78fed63480 x1875503571240320/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:750/0 lens 488/0 e 0 to 0 dl 1788619910 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619899.332708:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fed63480 with x1875503571240320 ext(1150976->1155071) 00002000:00010000:0.0:1788619899.332716:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fed63480 00000100:00100000:0.0:1788619899.332746:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 180 00000100:00100000:0.0:1788619899.332756:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed63480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571240320:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619899.333114:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed63480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571240320:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 357us (854us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619899.333123:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 180 00002000:00100000:0.0:1788619899.333152:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1150976->1155071) req@ffff8e78fed63480 x1875503571240320/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:750/0 lens 488/4536 e 0 to 0 dl 1788619910 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619899.333449:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fed63480 with x1875503571240320 ext(1150976->1155071) 00002000:00010000:0.0:1788619899.333454:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fed63480 00000100:00100000:3.0:1788619899.339161:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619899.349538:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619899.361462:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619899.361667:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571240704 00000100:00100000:0.0:1788619899.361679:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571240704 00002000:00100000:0.0:1788619899.361748:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1101824->1163263) req@ffff8e78fed60380 x1875503571240704/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:750/0 lens 488/0 e 0 to 0 dl 1788619910 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619899.361760:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fed60380 with x1875503571240704 ext(1101824->1163263) 00002000:00010000:0.0:1788619899.361767:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fed60380 00000100:00100000:0.0:1788619899.361811:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 183 00000100:00100000:0.0:1788619899.361822:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed60380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571240704:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:3.0:1788619899.366162:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed60380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2353:x1875503571240704:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 4324us (4698us total) trans 4294967350 rc 0/0 00000100:00100000:3.0:1788619899.366182:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 183 00002000:00100000:3.0:1788619899.366198:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1101824->1163263) req@ffff8e78fed60380 x1875503571240704/t4294967350(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:750/0 lens 488/448 e 0 to 0 dl 1788619910 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619899.366210:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fed60380 with x1875503571240704 ext(1101824->1163263) 00002000:00010000:3.0:1788619899.366218:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fed60380 00000100:00100000:3.0:1788619899.369699:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619899.369813:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571240832 00000100:00100000:0.0:1788619899.369825:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571240832 00000100:00100000:0.0:1788619899.369898:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 343 00000100:00100000:0.0:1788619899.369910:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed60a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+12:2352:x1875503571240832:12345-192.168.201.5@tcp:16:fsx.0 00000100:00100000:0.0:1788619899.420734:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed60a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571240832:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 50814us (51038us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619899.420751:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 343 00000100:00100000:3.0:1788619899.425771:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571240960 00000100:00100000:3.0:1788619899.425793:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571240960 00000100:00100000:3.0:1788619899.425933:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 553 00000100:00100000:3.0:1788619899.425955:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+34:14908:x1875503571240960:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619899.426010:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619899.426047:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048200/0x1cd7fe5368733374 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1aa4f expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619899.426189:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048e00/0x1cd7fe536873337b lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619899.426223:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048e00/0x1cd7fe536873337b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619899.426317:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048e00/0x1cd7fe536873337b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619899.445040:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619899.452035:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619899.458755:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571241472 00000100:00100000:3.0:1788619899.458768:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571241472 00000100:00100000:0.0:1788619899.458891:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 555 00000100:00100000:0.0:1788619899.458907:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+30:2355:x1875503571241472:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619899.458943:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619899.458971:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4bc00/0x1cd7fe5368733397 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1aa6b expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619899.459041:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50800/0x1cd7fe53687332f6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a9d8 expref: 34 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619899.459112:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5771c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589006464:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619899.459896:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78ce2fe180 going to sleep for 11 seconds 00000100:00100000:3.0:1788619899.462215:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5771c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589006464:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619899.462265:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 890767 00010000:00010000:3.0:1788619899.462289:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4bc00/0x1cd7fe5368733397 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aa6b expref: 31 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619899.462316:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619899.462319:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619899.462367:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4bc00/0x1cd7fe5368733397 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aa6b expref: 31 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619899.462386:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4bc00, rc 0) 00000100:00100000:3.0:1788619899.462470:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+30:2355:x1875503571241472:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3566us (3844us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619899.462480:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 555 00000100:00100000:0.0:1788619899.483642:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619899.504049:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571241856 00000100:00100000:3.0:1788619899.504063:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571241856 00000100:00100000:3.0:1788619899.504154:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 558 00000100:00100000:3.0:1788619899.504167:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dc9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+30:2352:x1875503571241856:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619899.504195:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619899.504224:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049400/0x1cd7fe53687333ac lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1aa80 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619899.504290:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50800/0x1cd7fe53687332f6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a9d8 expref: 34 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619899.504800:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed62300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589006848:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619899.504923:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa600 going to sleep for 11 seconds 00000100:00100000:3.0:1788619899.512294:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed62300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589006848:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619899.512381:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 890767 00010000:00010000:3.0:1788619899.512408:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049400/0x1cd7fe53687333ac lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aa80 expref: 31 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619899.512434:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619899.512438:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619899.512444:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049400/0x1cd7fe53687333ac lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aa80 expref: 31 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619899.512461:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5049400, rc 0) 00000100:00100000:3.0:1788619899.512536:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dc9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+30:2352:x1875503571241856:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8370us (8615us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619899.512546:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 558 00000100:00100000:0.0:1788619899.523793:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc480 going to sleep for 11 seconds 00000100:00100000:0.0:1788619899.530404:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fca80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589006976:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619899.530443:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 890767 00010000:00010000:0.0:1788619899.530468:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505de00/0x1cd7fe53687333b3 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aa87 expref: 31 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619899.530495:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619899.530498:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619899.530504:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505de00/0x1cd7fe53687333b3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aa87 expref: 31 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619899.530520:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505de00, rc 0) 00000100:00100000:0.0:1788619899.530597:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dcad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+30:2353:x1875503571241984:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7488us (7768us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619899.530606:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 559 00000100:00100000:3.0:1788619899.537528:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619899.537803:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571242112 00000100:00100000:0.0:1788619899.537816:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571242112 00000100:00100000:0.0:1788619899.537899:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 346 00000100:00100000:0.0:1788619899.537911:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed62300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571242112:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619899.537957:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619899.537980:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d7f6d20/0x1cd7fe53687333ba lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1aa8e expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619899.538069:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589007104:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619899.538192:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78ce2fe540 going to sleep for 11 seconds 00000100:00100000:0.0:1788619899.545984:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589007104:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619899.546041:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000d7f6d20/0x1cd7fe53687333ba lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aa8e expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619899.546077:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d7f6d20/0x1cd7fe53687333ba lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aa8e expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619899.546105:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4b200, rc 0) 00000100:00100000:0.0:1788619899.546188:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed62300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571242112:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8279us (8673us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619899.546199:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 346 00000100:00100000:0.0:1788619899.552852:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571242240 00000100:00100000:0.0:1788619899.552865:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571242240 00000100:00100000:0.0:1788619899.552943:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 347 00000100:00100000:0.0:1788619899.552956:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dc8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571242240:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619899.552980:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619899.553004:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d7f6d20/0x1cd7fe53687333c1 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1aa95 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619899.553119:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589007232:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619899.553230:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78ce2fe540 going to sleep for 11 seconds 00000100:00100000:0.0:1788619899.590921:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619899.591042:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571242496 00000100:00100000:3.0:1788619899.591055:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571242496 00000100:00100000:3.0:1788619899.591147:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 561 00000100:00100000:3.0:1788619899.591160:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+30:2354:x1875503571242496:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619899.591187:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619899.591213:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049400/0x1cd7fe53687333cf lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1aaa3 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619899.591489:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50800/0x1cd7fe53687332f6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a9d8 expref: 34 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619899.591545:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed62680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589007488:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619899.591854:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa600 going to sleep for 11 seconds 00000100:00100000:3.0:1788619899.598893:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed62680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589007488:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619899.598940:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 890767 00010000:00010000:3.0:1788619899.598967:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049400/0x1cd7fe53687333cf lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aaa3 expref: 31 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619899.598996:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619899.599000:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619899.599007:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049400/0x1cd7fe53687333cf lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aaa3 expref: 31 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619899.599028:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5049400, rc 0) 00000100:00100000:3.0:1788619899.599179:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+30:2354:x1875503571242496:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8021us (8274us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619899.599189:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 561 00000100:00100000:0.0:1788619899.609859:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619899.618197:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619899.628525:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571242880 00000100:00100000:3.0:1788619899.628539:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571242880 00000100:00100000:0.0:1788619899.642616:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619899.642687:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619899.643827:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571243136 00000100:00100000:3.0:1788619899.643836:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571243136 00000100:00100000:3.0:1788619899.643895:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 348 00000100:00100000:3.0:1788619899.643905:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571243136:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619899.643918:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619899.643944:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008807c503/0x1cd7fe53687333e4 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1aab8 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619899.643999:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed62680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589008000:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619899.644060:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa600 going to sleep for 11 seconds 00000100:00100000:3.0:1788619899.660657:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed62680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589008000:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619899.660713:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008807c503/0x1cd7fe53687333e4 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aab8 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619899.660744:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008807c503/0x1cd7fe53687333e4 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aab8 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619899.660761:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5049400, rc 0) 00000100:00100000:3.0:1788619899.660837:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571243136:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 16934us (18148us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619899.660847:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 348 00000100:00100000:3.0:1788619899.670540:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571243264 00000100:00100000:3.0:1788619899.670557:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571243264 00000100:00100000:3.0:1788619899.670666:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 564 00000100:00100000:3.0:1788619899.670681:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010fd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+34:2354:x1875503571243264:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619899.670714:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619899.670744:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049c00/0x1cd7fe53687333eb lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1aabf expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619899.670801:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51e00/0x1cd7fe53687333d6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aaaa expref: 31 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619899.670888:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed62680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589008128:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619899.671870:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa600 going to sleep for 11 seconds 00000100:00100000:0.0:1788619899.683275:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed62680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589008128:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619899.683319:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 890767 00010000:00010000:0.0:1788619899.683376:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049c00/0x1cd7fe53687333eb lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aabf expref: 35 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619899.683409:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619899.683413:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619899.683420:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049c00/0x1cd7fe53687333eb lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aabf expref: 35 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619899.683439:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5049c00, rc 0) 00000100:00100000:0.0:1788619899.683527:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010fd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+34:2354:x1875503571243264:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 12848us (13247us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619899.683538:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 564 00000100:00100000:0.0:1788619899.703454:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619899.703519:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619899.713045:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619899.737571:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571244160 00000100:00100000:3.0:1788619899.737590:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571244160 00002000:00100000:3.0:1788619899.737671:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1634304->1638399) req@ffff8e7900e01880 x1875503571244160/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:750/0 lens 488/0 e 0 to 0 dl 1788619910 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619899.737686:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e7900e01880 with x1875503571244160 ext(1634304->1638399) 00002000:00010000:3.0:1788619899.737694:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e7900e01880 00000100:00100000:3.0:1788619899.737739:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 185 00000100:00100000:3.0:1788619899.737751:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e01880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571244160:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619899.738304:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e01880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571244160:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 552us (1468us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619899.738314:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 185 00002000:00100000:3.0:1788619899.738323:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1634304->1638399) req@ffff8e7900e01880 x1875503571244160/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:750/0 lens 488/4536 e 0 to 0 dl 1788619910 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619899.738424:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e7900e01880 with x1875503571244160 ext(1634304->1638399) 00002000:00010000:3.0:1788619899.738429:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e7900e01880 00000100:00100000:0.0:1788619899.741712:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619899.747205:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571244416 00000100:00100000:3.0:1788619899.747223:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571244416 00002000:00100000:3.0:1788619899.747319:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1642496->1646591) req@ffff8e7900e03800 x1875503571244416/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:750/0 lens 488/0 e 0 to 0 dl 1788619910 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619899.747390:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e7900e03800 with x1875503571244416 ext(1642496->1646591) 00002000:00010000:3.0:1788619899.747397:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e7900e03800 00000100:00100000:3.0:1788619899.747466:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 187 00000100:00100000:3.0:1788619899.747479:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e03800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571244416:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619899.757679:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619899.768835:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619899.787760:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619899.792761:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619899.799723:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619899.806069:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619899.816496:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571245440 00000100:00100000:0.0:1788619899.816512:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571245440 00002000:00100000:0.0:1788619899.816588:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1675264->1679359) req@ffff8e7900e03480 x1875503571245440/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:750/0 lens 488/0 e 0 to 0 dl 1788619910 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619899.816603:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e7900e03480 with x1875503571245440 ext(1675264->1679359) 00002000:00010000:0.0:1788619899.816611:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e7900e03480 00000100:00100000:0.0:1788619899.816654:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 195 00000100:00100000:0.0:1788619899.816666:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e03480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571245440:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619899.817020:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e03480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571245440:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 355us (862us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619899.817028:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 195 00002000:00100000:0.0:1788619899.817036:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1675264->1679359) req@ffff8e7900e03480 x1875503571245440/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:750/0 lens 488/4536 e 0 to 0 dl 1788619910 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619899.817045:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e7900e03480 with x1875503571245440 ext(1675264->1679359) 00002000:00010000:0.0:1788619899.817049:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e7900e03480 00000100:00100000:0.0:1788619899.822209:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619899.834829:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619899.840203:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619899.846029:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619899.862168:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619899.862269:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571246208 00000100:00100000:3.0:1788619899.862281:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571246208 00000100:00100000:3.0:1788619899.862378:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 567 00000100:00100000:3.0:1788619899.862387:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5694a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+34:2353:x1875503571246208:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619899.862414:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619899.862448:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049600/0x1cd7fe5368733415 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1aae9 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619899.862504:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51e00/0x1cd7fe53687333d6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aaaa expref: 31 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619899.862553:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed62d80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589009024:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619899.862663:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa180 going to sleep for 11 seconds 00000100:00100000:3.0:1788619899.864933:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed62d80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589009024:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619899.864971:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 890767 00010000:00010000:3.0:1788619899.864993:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049600/0x1cd7fe5368733415 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aae9 expref: 35 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619899.865015:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619899.865017:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619899.865022:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049600/0x1cd7fe5368733415 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aae9 expref: 35 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619899.865056:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5049600, rc 0) 00000100:00100000:3.0:1788619899.865141:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5694a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+34:2353:x1875503571246208:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2755us (2986us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619899.865151:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 567 00000100:00100000:0.0:1788619899.875073:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5dc8380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589009152:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619899.875124:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 890767 00010000:00010000:0.0:1788619899.875169:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51400/0x1cd7fe536873341c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aaf0 expref: 35 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619899.875196:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619899.875200:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619899.875206:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51400/0x1cd7fe536873341c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aaf0 expref: 35 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619899.875224:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b51400, rc 0) 00000100:00100000:0.0:1788619899.875311:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+34:2355:x1875503571246336:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3880us (4454us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619899.875322:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 568 00000100:00100000:0.0:1788619899.880104:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571246464 00000100:00100000:0.0:1788619899.880117:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571246464 00000100:00100000:0.0:1788619899.880193:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 569 00000100:00100000:0.0:1788619899.880205:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+34:2353:x1875503571246464:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619899.880232:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619899.880259:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4aa00/0x1cd7fe5368733423 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1aaf7 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619899.880306:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51e00/0x1cd7fe53687333d6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aaaa expref: 31 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619899.880510:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5696d80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589009280:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619899.880619:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78ce2fed80 going to sleep for 11 seconds 00000100:00100000:0.0:1788619899.889759:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5696d80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589009280:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619899.889803:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 890767 00010000:00010000:0.0:1788619899.889861:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4aa00/0x1cd7fe5368733423 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aaf7 expref: 35 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619899.889891:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619899.889894:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619899.889901:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4aa00/0x1cd7fe5368733423 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aaf7 expref: 35 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619899.889930:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4aa00, rc 0) 00000100:00100000:0.0:1788619899.890012:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+34:2353:x1875503571246464:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 9807us (10146us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619899.890022:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 569 00000100:00100000:3.0:1788619899.894078:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571246720 00000100:00100000:3.0:1788619899.894104:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571246720 00000100:00100000:3.0:1788619899.894180:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 570 00000100:00100000:3.0:1788619899.894192:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dc9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+34:14908:x1875503571246720:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:3.0:1788619899.894267:0:9417:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619899.894303:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048e00/0x1cd7fe536873337b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aa4f expref: 34 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619899.894382:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619899.894388:0:9417:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788619899.894678:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049600/0x1cd7fe536873342a lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619899.894695:0:9417:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368733304 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a9df expref: 31 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619899.894705:0:9417:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049600/0x1cd7fe536873342a lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619899.894746:0:9417:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368733304 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a9df expref: 31 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619899.894762:0:9417:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368733304 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a9df expref: 31 pid: 5846 timeout: 492 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619899.894801:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed62d80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589009408:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619899.894901:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa180 going to sleep for 11 seconds 00010000:00010000:3.0:1788619899.902208:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368733304 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1a9df expref: 31 pid: 5846 timeout: 492 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619899.902231:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed62d80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589009408:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619899.902270:0:9417:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049600/0x1cd7fe536873342a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619899.902281:0:9417:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049600/0x1cd7fe536873342a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619899.902287:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049600/0x1cd7fe536873342a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788619899.902504:0:9417:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967527 00000004:00010000:3.0:1788619899.902575:0:9417:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 890767 -> 890767 00000004:00010000:3.0:1788619899.902579:0:9417:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619899 -> 1788619899 00000004:00010000:3.0:1788619899.902582:0:9417:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619899 -> 1788619899 00000004:00010000:3.0:1788619899.902584:0:9417:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619899 -> 1788619899 00000004:00010000:3.0:1788619899.902587:0:9417:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 774 -> 774 00000004:00010000:3.0:1788619899.902596:0:9417:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000057bcc9d1 reply state 0000000092d47278 transno 4294967528 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049600/0x1cd7fe536873342a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619899.902606:0:9417:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788619899.902725:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dc9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+36:14908:x1875503571246720:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 8535us (8771us total) trans 4294967528 rc 0/0 00000100:00100000:3.0:1788619899.902751:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 570 00000020:00080000:3.0:1788619899.902756:0:9417:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788619899.906551:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049600/0x1cd7fe536873342a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619899.906575:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049600/0x1cd7fe536873342a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619899.906584:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049600/0x1cd7fe536873342a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619899.906591:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c5049600 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788619899.906598:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049600/0x1cd7fe536873342a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619899.906607:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049600/0x1cd7fe536873342a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619899.906629:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619899.906632:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788619899.906661:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049600/0x1cd7fe536873342a lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619899.906668:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049600/0x1cd7fe536873342a lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619899.907857:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619899.915285:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619899.915462:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571247104 00000100:00100000:0.0:1788619899.915475:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571247104 00010000:00010000:0.0:1788619899.915531:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51e00/0x1cd7fe53687333d6 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aaaa expref: 31 pid: 5844 timeout: 492 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619899.915586:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 189 00000100:00100000:0.0:1788619899.915598:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed62d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+31:2354:x1875503571247104:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619899.915617:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619899.915655:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51e00/0x1cd7fe53687333d6 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aaaa expref: 31 pid: 5844 timeout: 492 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619899.915671:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51e00/0x1cd7fe53687333d6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aaaa expref: 31 pid: 5844 timeout: 492 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619899.915700:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51e00/0x1cd7fe53687333d6 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aaaa expref: 31 pid: 5844 timeout: 492 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619899.915722:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619899.915729:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50000/0x1cd7fe5368733431 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aafe expref: 35 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619899.915742:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50000/0x1cd7fe5368733431 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aafe expref: 35 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619899.915776:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50000/0x1cd7fe5368733431 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aafe expref: 35 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619899.915828:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50000/0x1cd7fe5368733431 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aafe expref: 35 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619899.915848:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5695f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589009664:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619899.915979:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78ce2fed80 going to sleep for 11 seconds 00000100:00100000:3.0:1788619899.922211:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571247232 00000100:00100000:3.0:1788619899.922222:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571247232 00000100:00100000:3.0:1788619899.922287:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 352 00000100:00100000:3.0:1788619899.922298:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:14908:x1875503571247232:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619899.922321:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619899.922388:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000574a4e4c/0x1cd7fe5368733438 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ab05 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619899.922427:0:6696:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000007f93e9f3/0x1cd7fe5368733407 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1630208->1634303) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aadb expref: 7 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619899.922437:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(1 11) 00010000:00010000:3.0:1788619899.922443:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(1 11) 00010000:00010000:3.0:1788619899.922449:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000574a4e4c/0x1cd7fe5368733438 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab05 expref: 7 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619899.922492:0:6696:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000007f93e9f3/0x1cd7fe5368733407 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1630208->1634303) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aadb expref: 7 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619899.922507:0:6696:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000007f93e9f3/0x1cd7fe5368733407 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1630208->1634303) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aadb expref: 7 pid: 7654 timeout: 492 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619899.922524:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed62a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589009792:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619899.922630:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa180 going to sleep for 11 seconds 00000100:00100000:3.0:1788619899.926540:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619899.935925:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619899.939311:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619899.948679:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571247744 00000100:00100000:0.0:1788619899.948694:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571247744 00000100:00100000:0.0:1788619899.948772:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 572 00000100:00100000:0.0:1788619899.948785:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dcb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+30:14908:x1875503571247744:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619899.948819:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619899.948849:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4bc00/0x1cd7fe536873343f lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ab0c expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619899.949103:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4b000/0x1cd7fe5368733446 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619899.949125:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4b000/0x1cd7fe5368733446 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619899.949140:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4b000/0x1cd7fe5368733446 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619899.949199:0:5846:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 209827 00000004:00200000:0.0:1788619899.949204:0:5846:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619899.949229:0:5846:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4b000/0x1cd7fe5368733446 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619899.949264:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4bc00/0x1cd7fe536873343f lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab0c expref: 32 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619899.949285:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4b000/0x1cd7fe5368733446 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab0c expref: 31 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619899.949297:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619899.949301:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619899.949305:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4b000, rc 0) 00000100:00100000:0.0:1788619899.949450:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dcb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+31:14908:x1875503571247744:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 666us (984us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619899.949458:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 572 00000100:00100000:0.0:1788619899.953397:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619899.953551:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571247872 00000100:00100000:3.0:1788619899.953566:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571247872 00000100:00100000:3.0:1788619899.953643:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 573 00000100:00100000:3.0:1788619899.953655:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5695180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+31:2355:x1875503571247872:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619899.953684:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619899.953713:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049400/0x1cd7fe536873344d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ab13 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619899.953761:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50000/0x1cd7fe5368733431 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aafe expref: 36 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619899.953820:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed62a00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589010048:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619899.953935:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaacc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619899.958869:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed62a00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589010048:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619899.958920:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 209827 00010000:00010000:3.0:1788619899.958949:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049400/0x1cd7fe536873344d lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab13 expref: 32 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619899.958978:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619899.958982:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619899.958988:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049400/0x1cd7fe536873344d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab13 expref: 32 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619899.959007:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5049400, rc 0) 00000100:00100000:3.0:1788619899.959096:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5695180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+31:2355:x1875503571247872:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5441us (5713us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619899.959129:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 573 00000100:00100000:0.0:1788619899.965381:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571248128 00000100:00100000:0.0:1788619899.965396:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571248128 00000100:00100000:0.0:1788619899.965475:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 574 00000100:00100000:0.0:1788619899.965488:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dc8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+31:2354:x1875503571248128:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619899.965515:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619899.965543:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4bc00/0x1cd7fe536873345b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ab21 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619899.965593:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50000/0x1cd7fe5368733431 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aafe expref: 36 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619899.965668:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5696a00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589010304:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619899.965793:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78ce2fe3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619899.968917:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5696a00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589010304:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619899.968956:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 209827 00010000:00010000:0.0:1788619899.968982:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4bc00/0x1cd7fe536873345b lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab21 expref: 32 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619899.969006:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619899.969009:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619899.969015:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4bc00/0x1cd7fe536873345b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab21 expref: 32 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619899.969032:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4bc00, rc 0) 00000100:00100000:0.0:1788619899.969122:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dc8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+31:2354:x1875503571248128:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3636us (4040us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619899.969133:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 574 00000100:00100000:3.0:1788619899.986968:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619899.993784:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571248640 00000100:00100000:0.0:1788619899.993802:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571248640 00002000:00100000:0.0:1788619899.993880:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5dc9c00 x1875503571248640/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:750/0 lens 488/0 e 0 to 0 dl 1788619910 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619899.993892:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5dc9c00 with x1875503571248640 00000100:00100000:0.0:1788619899.993957:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 219 00000100:00100000:0.0:1788619899.993969:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dc9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+37:2352:x1875503571248640:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619899.994449:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dc9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+37:2352:x1875503571248640:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 480us (811us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619899.994467:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 219 00002000:00100000:0.0:1788619899.994475:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5dc9c00 x1875503571248640/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:750/0 lens 488/4536 e 0 to 0 dl 1788619910 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619899.994484:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5dc9c00 with x1875503571248640 00000100:00100000:3.0:1788619899.998510:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619900.004103:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619900.004979:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79008ad880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+37:2353:x1875503571248896:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 402us (905us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619900.005013:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 221 00002000:00100000:2.0:1788619900.005021:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79008ad880 x1875503571248896/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:751/0 lens 488/4536 e 0 to 0 dl 1788619911 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619900.005030:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79008ad880 with x1875503571248896 00000100:00080000:3.0:1788619900.005591:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788619900.008596:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619900.008832:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571249024 00000100:00100000:0.0:1788619900.008849:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571249024 00002000:00100000:0.0:1788619900.008926:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79008af800 x1875503571249024/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:751/0 lens 488/0 e 0 to 0 dl 1788619911 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619900.008938:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79008af800 with x1875503571249024 00000100:00100000:0.0:1788619900.008985:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 222 00000100:00100000:0.0:1788619900.008995:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79008af800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+37:2353:x1875503571249024:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619900.009411:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79008af800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+37:2353:x1875503571249024:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 415us (831us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619900.009419:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 222 00002000:00100000:0.0:1788619900.009427:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79008af800 x1875503571249024/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:751/0 lens 488/4536 e 0 to 0 dl 1788619911 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619900.009434:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79008af800 with x1875503571249024 00000100:00100000:3.0:1788619900.010120:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78cdcc2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589010560:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1788619900.010187:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:3.0:1788619900.011103:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1788619900.014398:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619900.014552:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571249152 00000100:00100000:0.0:1788619900.014567:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571249152 00002000:00100000:0.0:1788619900.014640:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900d5f480 x1875503571249152/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:751/0 lens 488/0 e 0 to 0 dl 1788619911 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619900.014653:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900d5f480 with x1875503571249152 00000100:00100000:0.0:1788619900.014703:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 223 00000100:00100000:0.0:1788619900.014714:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+37:2352:x1875503571249152:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619900.015135:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+37:2352:x1875503571249152:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 422us (753us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619900.015145:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 223 00002000:00100000:0.0:1788619900.015153:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900d5f480 x1875503571249152/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:751/0 lens 488/4536 e 0 to 0 dl 1788619911 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619900.015161:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900d5f480 with x1875503571249152 00000100:00100000:3.0:1788619900.018986:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619900.019400:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571249280 00000100:00100000:2.0:1788619900.019416:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571249280 00002000:00100000:2.0:1788619900.019506:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79008ae300 x1875503571249280/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:751/0 lens 488/0 e 0 to 0 dl 1788619911 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619900.019522:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79008ae300 with x1875503571249280 00000100:00100000:2.0:1788619900.019577:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 224 00000100:00100000:2.0:1788619900.019590:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79008ae300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+37:2353:x1875503571249280:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619900.020054:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79008ae300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+37:2353:x1875503571249280:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 465us (1083us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619900.020064:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 224 00002000:00100000:2.0:1788619900.020072:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79008ae300 x1875503571249280/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:751/0 lens 488/4536 e 0 to 0 dl 1788619911 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619900.020093:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79008ae300 with x1875503571249280 00000100:00100000:3.0:1788619900.024458:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619900.024680:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571249408 00000100:00100000:2.0:1788619900.024697:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571249408 00002000:00100000:2.0:1788619900.024778:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79008ad180 x1875503571249408/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:751/0 lens 488/0 e 0 to 0 dl 1788619911 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619900.024792:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79008ad180 with x1875503571249408 00000100:00100000:2.0:1788619900.024845:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 225 00000100:00100000:2.0:1788619900.024875:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79008ad180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+37:2352:x1875503571249408:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619900.025203:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79008ad180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+37:2352:x1875503571249408:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 328us (757us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619900.025212:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 225 00002000:00100000:2.0:1788619900.025222:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79008ad180 x1875503571249408/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:751/0 lens 488/4536 e 0 to 0 dl 1788619911 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619900.025231:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79008ad180 with x1875503571249408 00000100:00100000:3.0:1788619900.029255:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619900.029494:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571249536 00000100:00100000:0.0:1788619900.029508:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571249536 00002000:00100000:0.0:1788619900.029578:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79008af100 x1875503571249536/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:751/0 lens 488/0 e 0 to 0 dl 1788619911 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619900.029590:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79008af100 with x1875503571249536 00000100:00100000:0.0:1788619900.029636:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 226 00000100:00100000:0.0:1788619900.029647:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79008af100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+37:2353:x1875503571249536:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619900.029952:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79008af100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+37:2353:x1875503571249536:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 304us (711us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619900.029961:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 226 00002000:00100000:0.0:1788619900.029969:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79008af100 x1875503571249536/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:751/0 lens 488/4536 e 0 to 0 dl 1788619911 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619900.029978:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79008af100 with x1875503571249536 00000100:00100000:3.0:1788619900.034635:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619900.034814:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571249664 00000100:00100000:0.0:1788619900.034829:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571249664 00000100:00100000:0.0:1788619900.034923:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 575 00000100:00100000:0.0:1788619900.034936:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79008afb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+37:14908:x1875503571249664:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619900.034963:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619900.035009:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4bc00/0x1cd7fe5368733469 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ab2f expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.035132:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4a600/0x1cd7fe5368733470 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.035152:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4a600/0x1cd7fe5368733470 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.035165:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4a600/0x1cd7fe5368733470 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619900.035197:0:5846:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 209827 00000004:00200000:0.0:1788619900.035202:0:5846:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619900.035224:0:5846:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4a600/0x1cd7fe5368733470 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.035256:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4bc00/0x1cd7fe5368733469 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab2f expref: 39 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.035277:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4a600/0x1cd7fe5368733470 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab2f expref: 38 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.035287:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619900.035289:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619900.035294:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4a600, rc 0) 00000100:00100000:0.0:1788619900.035675:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79008afb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+38:14908:x1875503571249664:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 740us (1056us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619900.035683:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 575 00000100:00100000:3.0:1788619900.040270:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619900.040389:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571249792 00000100:00100000:0.0:1788619900.040421:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571249792 00000100:00100000:0.0:1788619900.040516:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 576 00000100:00100000:0.0:1788619900.040529:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79008ac380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+31:14908:x1875503571249792:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619900.040563:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619900.040592:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4bc00/0x1cd7fe5368733477 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ab36 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.040633:0:9417:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50000/0x1cd7fe5368733431 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aafe expref: 37 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.040645:0:9417:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4bc00/0x1cd7fe5368733477 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab36 expref: 32 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.040688:0:9417:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50000/0x1cd7fe5368733431 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aafe expref: 37 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.040703:0:9417:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50000/0x1cd7fe5368733431 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aafe expref: 37 pid: 5844 timeout: 492 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619900.040718:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5696a00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589010688:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619900.040834:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78ce2fe900 going to sleep for 11 seconds 00000100:00080000:3.0:1788619900.041011:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1788619900.044742:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5696a00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589010688:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619900.044798:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4bc00/0x1cd7fe5368733477 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab36 expref: 32 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.044823:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619900.044827:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619900.044854:0:9417:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4bc00/0x1cd7fe5368733477 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab36 expref: 32 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.044866:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4bc00, rc 0) 00000100:00100000:0.0:1788619900.044963:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79008ac380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+32:14908:x1875503571249792:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4436us (4709us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619900.044972:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 576 00000100:00100000:2.0:1788619900.047091:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619900.054123:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619900.054239:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571250048 00000100:00100000:2.0:1788619900.054253:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571250048 00000100:00100000:2.0:1788619900.054396:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 227 00000100:00100000:2.0:1788619900.054413:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79008ad500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+32:2355:x1875503571250048:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:2.0:1788619900.054753:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 209827 -> 209827 00000004:00010000:2.0:1788619900.054758:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619899 -> 1788619899 00000004:00010000:2.0:1788619900.054761:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619899 -> 1788619899 00000004:00010000:2.0:1788619900.054764:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619899 -> 1788619899 00000004:00010000:2.0:1788619900.054767:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 774 -> 774 00000100:00100000:2.0:1788619900.054854:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79008ad500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+33:2355:x1875503571250048:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 447us (749us total) trans 4294967530 rc 0/0 00000100:00100000:2.0:1788619900.054863:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 227 00000100:00100000:3.0:1788619900.060131:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619900.060264:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571250176 00000100:00100000:0.0:1788619900.060277:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571250176 00000100:00100000:0.0:1788619900.060418:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 577 00000100:00100000:0.0:1788619900.060432:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79008ac700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+37:2353:x1875503571250176:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619900.060461:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619900.060492:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4b200/0x1cd7fe536873347e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ab3d expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.060547:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4bc00/0x1cd7fe5368733477 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab36 expref: 32 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619900.060605:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5696a00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589010944:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619900.060710:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78ce2fe900 going to sleep for 11 seconds 00000100:00100000:0.0:1788619900.063485:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5696a00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589010944:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619900.063530:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 209827 00010000:00010000:0.0:1788619900.063555:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4b200/0x1cd7fe536873347e lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab3d expref: 38 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.063580:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619900.063584:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619900.063612:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4b200/0x1cd7fe536873347e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab3d expref: 38 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.063631:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4b200, rc 0) 00000100:00100000:0.0:1788619900.063712:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79008ac700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+37:2353:x1875503571250176:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3282us (3599us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619900.063721:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 577 00000100:00100000:3.0:1788619900.067182:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619900.067278:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571250304 00000100:00100000:0.0:1788619900.067291:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571250304 00000100:00100000:0.0:1788619900.067403:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 578 00000100:00100000:0.0:1788619900.067417:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c31500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+37:2355:x1875503571250304:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619900.067448:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619900.067479:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4b600/0x1cd7fe5368733485 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ab44 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.067533:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4bc00/0x1cd7fe5368733477 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab36 expref: 32 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619900.067587:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5696a00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589011072:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619900.067703:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78ce2fe900 going to sleep for 11 seconds 00000100:00100000:0.0:1788619900.071381:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5696a00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589011072:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619900.071422:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 209827 00010000:00010000:0.0:1788619900.071448:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4b600/0x1cd7fe5368733485 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab44 expref: 38 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.071475:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619900.071478:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619900.071505:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4b600/0x1cd7fe5368733485 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab44 expref: 38 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.071525:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4b600, rc 0) 00000100:00100000:0.0:1788619900.071605:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c31500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+37:2355:x1875503571250304:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4191us (4440us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619900.071614:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 578 00000100:00100000:3.0:1788619900.077865:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619900.078128:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571250432 00000100:00100000:0.0:1788619900.078144:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571250432 00000100:00100000:0.0:1788619900.078241:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 355 00000100:00100000:0.0:1788619900.078254:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c33800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571250432:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619900.078277:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619900.078301:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007b9ecde9/0x1cd7fe536873348c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ab4b expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619900.078561:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5696a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589011200:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619900.078687:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78ce2fe900 going to sleep for 11 seconds 00000100:00080000:3.0:1788619900.079318:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1788619900.083171:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5696a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589011200:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619900.083226:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007b9ecde9/0x1cd7fe536873348c lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab4b expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.083255:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007b9ecde9/0x1cd7fe536873348c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab4b expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.083270:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4b600, rc 0) 00000100:00100000:0.0:1788619900.083499:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c33800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571250432:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5246us (5648us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619900.083531:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 355 00000100:00100000:2.0:1788619900.087603:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619900.087676:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571250560 00000100:00100000:0.0:1788619900.087691:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571250560 00002000:00100000:0.0:1788619900.087765:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4ba2d80 x1875503571250560/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:751/0 lens 488/0 e 0 to 0 dl 1788619911 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619900.087778:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4ba2d80 with x1875503571250560 00000100:00100000:0.0:1788619900.087830:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 228 00000100:00100000:0.0:1788619900.087843:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+33:2354:x1875503571250560:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619900.089074:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+33:2354:x1875503571250560:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 1224us (1476us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619900.089097:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 228 00002000:00100000:0.0:1788619900.089108:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4ba2d80 x1875503571250560/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:751/0 lens 488/4536 e 0 to 0 dl 1788619911 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619900.089119:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4ba2d80 with x1875503571250560 00000100:00100000:2.0:1788619900.093469:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619900.093640:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571250688 00000100:00100000:0.0:1788619900.093653:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571250688 00002000:00100000:0.0:1788619900.093726:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4ba0a80 x1875503571250688/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:751/0 lens 488/0 e 0 to 0 dl 1788619911 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619900.093739:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4ba0a80 with x1875503571250688 00000100:00100000:0.0:1788619900.093782:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 229 00000100:00100000:0.0:1788619900.093795:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+33:2353:x1875503571250688:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619900.094368:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+33:2353:x1875503571250688:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 531us (871us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619900.094378:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 229 00002000:00100000:0.0:1788619900.094406:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4ba0a80 x1875503571250688/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:751/0 lens 488/4536 e 0 to 0 dl 1788619911 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619900.094416:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4ba0a80 with x1875503571250688 00000100:00100000:0.0:1788619900.098848:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571250816 00000100:00100000:0.0:1788619900.098862:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571250816 00002000:00100000:0.0:1788619900.098992:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78cdcc2680 x1875503571250816/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:751/0 lens 488/0 e 0 to 0 dl 1788619911 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619900.099005:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78cdcc2680 with x1875503571250816 00000100:00100000:0.0:1788619900.099105:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 230 00000100:00100000:0.0:1788619900.099117:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+33:2354:x1875503571250816:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619900.099442:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+33:2354:x1875503571250816:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 326us (720us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619900.099451:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 230 00002000:00100000:0.0:1788619900.099459:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78cdcc2680 x1875503571250816/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:751/0 lens 488/4536 e 0 to 0 dl 1788619911 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619900.099467:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78cdcc2680 with x1875503571250816 00000100:00100000:3.0:1788619900.102648:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619900.103617:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571250944 00000100:00100000:2.0:1788619900.103633:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571250944 00002000:00100000:2.0:1788619900.103709:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5c30a80 x1875503571250944/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:751/0 lens 488/0 e 0 to 0 dl 1788619911 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619900.103742:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5c30a80 with x1875503571250944 00000100:00100000:2.0:1788619900.103776:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 231 00000100:00100000:2.0:1788619900.103787:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c30a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+33:2353:x1875503571250944:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619900.104139:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c30a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+33:2353:x1875503571250944:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 352us (1505us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619900.104148:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 231 00002000:00100000:2.0:1788619900.104155:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5c30a80 x1875503571250944/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:751/0 lens 488/4536 e 0 to 0 dl 1788619911 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619900.104164:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5c30a80 with x1875503571250944 00000100:00100000:3.0:1788619900.108720:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619900.108890:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571251072 00000100:00100000:2.0:1788619900.108903:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571251072 00002000:00100000:2.0:1788619900.108993:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5c31180 x1875503571251072/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:751/0 lens 488/0 e 0 to 0 dl 1788619911 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619900.109006:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5c31180 with x1875503571251072 00000100:00100000:2.0:1788619900.109055:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 232 00000100:00100000:2.0:1788619900.109069:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c31180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+33:2354:x1875503571251072:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619900.109522:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c31180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+33:2354:x1875503571251072:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 453us (822us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619900.109531:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 232 00002000:00100000:2.0:1788619900.109539:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5c31180 x1875503571251072/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:751/0 lens 488/4536 e 0 to 0 dl 1788619911 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619900.109548:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5c31180 with x1875503571251072 00000100:00100000:0.0:1788619900.114502:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619900.120690:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619900.120866:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571251328 00000100:00100000:2.0:1788619900.120897:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571251328 00002000:00100000:2.0:1788619900.120975:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5c31f80 x1875503571251328/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:751/0 lens 488/0 e 0 to 0 dl 1788619911 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619900.120988:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5c31f80 with x1875503571251328 00000100:00100000:2.0:1788619900.121039:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 234 00000100:00100000:2.0:1788619900.121051:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c31f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+33:2354:x1875503571251328:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619900.121584:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c31f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+33:2354:x1875503571251328:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 531us (907us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619900.121594:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 234 00002000:00100000:2.0:1788619900.121603:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5c31f80 x1875503571251328/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:751/0 lens 488/4536 e 0 to 0 dl 1788619911 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619900.121612:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5c31f80 with x1875503571251328 00000100:00100000:2.0:1788619900.125963:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619900.126094:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571251456 00000100:00100000:3.0:1788619900.126110:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571251456 00002000:00100000:3.0:1788619900.126183:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4ba3100 x1875503571251456/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:751/0 lens 488/0 e 0 to 0 dl 1788619911 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619900.126224:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4ba3100 with x1875503571251456 00000100:00100000:3.0:1788619900.126273:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 235 00000100:00100000:3.0:1788619900.126285:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+33:2353:x1875503571251456:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619900.126983:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+33:2353:x1875503571251456:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 695us (1033us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619900.126993:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 235 00002000:00100000:0.0:1788619900.127001:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4ba3100 x1875503571251456/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:751/0 lens 488/4536 e 0 to 0 dl 1788619911 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619900.127007:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4ba3100 with x1875503571251456 00000100:00100000:0.0:1788619900.133480:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619900.133574:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571251584 00000100:00100000:3.0:1788619900.133587:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571251584 00000100:00100000:3.0:1788619900.133663:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 356 00000100:00100000:3.0:1788619900.133675:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5697100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571251584:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619900.133700:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619900.133725:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000095315e19/0x1cd7fe5368733493 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ab52 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619900.133808:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c31880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589011328:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619900.133939:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaacc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619900.137393:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c31880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589011328:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619900.137453:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000095315e19/0x1cd7fe5368733493 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab52 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619900.137484:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000095315e19/0x1cd7fe5368733493 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab52 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619900.137500:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5048800, rc 0) 00000100:00100000:2.0:1788619900.137568:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5697100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571251584:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3894us (4105us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619900.137578:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 356 00000100:00100000:0.0:1788619900.141966:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619900.142097:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571251712 00000100:00100000:2.0:1788619900.142112:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571251712 00000100:00100000:2.0:1788619900.142188:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 357 00000100:00100000:2.0:1788619900.142200:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5695880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571251712:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619900.142226:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619900.142249:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008b900302/0x1cd7fe536873349a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ab59 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619900.142386:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4ba3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589011456:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619900.142487:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc000 going to sleep for 11 seconds 00000100:00100000:2.0:1788619900.146138:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4ba3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589011456:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619900.146195:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008b900302/0x1cd7fe536873349a lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab59 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619900.146228:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008b900302/0x1cd7fe536873349a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab59 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619900.146243:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505c000, rc 0) 00000100:00100000:2.0:1788619900.146319:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5695880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571251712:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4120us (4394us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619900.146384:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 357 00000100:00100000:3.0:1788619900.150430:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619900.150693:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571251840 00000100:00100000:2.0:1788619900.150706:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571251840 00000100:00100000:2.0:1788619900.150785:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 358 00000100:00100000:2.0:1788619900.150797:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c33b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571251840:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619900.150843:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619900.150865:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008b900302/0x1cd7fe53687334a1 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ab60 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619900.150955:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4ba3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589011584:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619900.151054:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc000 going to sleep for 11 seconds 00000100:00100000:2.0:1788619900.156651:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4ba3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589011584:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619900.156717:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008b900302/0x1cd7fe53687334a1 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab60 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619900.156753:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008b900302/0x1cd7fe53687334a1 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab60 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619900.156770:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505c000, rc 0) 00000100:00100000:2.0:1788619900.156915:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c33b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571251840:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6120us (6500us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619900.156926:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 358 00000100:00100000:0.0:1788619900.162611:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619900.167888:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619900.168088:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571252096 00000100:00100000:0.0:1788619900.168102:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571252096 00002000:00100000:0.0:1788619900.168171:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4ba3b80 x1875503571252096/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:751/0 lens 488/0 e 0 to 0 dl 1788619911 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619900.168183:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4ba3b80 with x1875503571252096 00000100:00100000:0.0:1788619900.168226:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 237 00000100:00100000:0.0:1788619900.168236:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+33:2354:x1875503571252096:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619900.168568:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+33:2354:x1875503571252096:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 332us (696us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619900.168577:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 237 00002000:00100000:0.0:1788619900.168584:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4ba3b80 x1875503571252096/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:751/0 lens 488/4536 e 0 to 0 dl 1788619911 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619900.168592:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4ba3b80 with x1875503571252096 00000100:00100000:2.0:1788619900.176896:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619900.177065:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571252224 00000100:00100000:0.0:1788619900.177088:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571252224 00002000:00100000:0.0:1788619900.177160:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4ba3480 x1875503571252224/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:751/0 lens 488/0 e 0 to 0 dl 1788619911 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619900.177175:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4ba3480 with x1875503571252224 00000100:00100000:0.0:1788619900.177223:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 238 00000100:00100000:0.0:1788619900.177234:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+33:2353:x1875503571252224:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619900.177617:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+33:2353:x1875503571252224:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 383us (738us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619900.177627:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 238 00002000:00100000:0.0:1788619900.177635:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4ba3480 x1875503571252224/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:751/0 lens 488/4536 e 0 to 0 dl 1788619911 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619900.177651:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4ba3480 with x1875503571252224 00000100:00100000:3.0:1788619900.181437:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619900.187632:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571252480 00000100:00100000:2.0:1788619900.187647:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571252480 00002000:00100000:2.0:1788619900.187727:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c7581500 x1875503571252480/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:751/0 lens 488/0 e 0 to 0 dl 1788619911 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619900.187742:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c7581500 with x1875503571252480 00000100:00100000:2.0:1788619900.187788:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 240 00000100:00100000:2.0:1788619900.187822:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7581500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+33:2354:x1875503571252480:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619900.188182:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7581500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+33:2354:x1875503571252480:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 360us (738us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619900.188192:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 240 00002000:00100000:2.0:1788619900.188199:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c7581500 x1875503571252480/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:751/0 lens 488/4536 e 0 to 0 dl 1788619911 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619900.188208:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c7581500 with x1875503571252480 00000100:00100000:2.0:1788619900.192778:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619900.193063:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571252608 00000100:00100000:0.0:1788619900.193088:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571252608 00002000:00100000:0.0:1788619900.193164:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df965f80 x1875503571252608/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:751/0 lens 488/0 e 0 to 0 dl 1788619911 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619900.193178:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df965f80 with x1875503571252608 00000100:00100000:0.0:1788619900.193226:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 241 00000100:00100000:0.0:1788619900.193237:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df965f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+33:2353:x1875503571252608:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619900.193567:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df965f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+33:2353:x1875503571252608:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 330us (805us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619900.193574:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 241 00002000:00100000:0.0:1788619900.193582:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df965f80 x1875503571252608/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:751/0 lens 488/4536 e 0 to 0 dl 1788619911 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619900.193590:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df965f80 with x1875503571252608 00000100:00100000:3.0:1788619900.198011:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619900.202826:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571252864 00000100:00100000:2.0:1788619900.202841:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571252864 00002000:00100000:2.0:1788619900.202926:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c7583480 x1875503571252864/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:751/0 lens 488/0 e 0 to 0 dl 1788619911 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619900.202940:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c7583480 with x1875503571252864 00000100:00100000:2.0:1788619900.202986:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 243 00000100:00100000:2.0:1788619900.203002:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7583480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+33:2354:x1875503571252864:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619900.203609:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7583480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+33:2354:x1875503571252864:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 609us (911us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619900.203619:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 243 00002000:00100000:2.0:1788619900.203627:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c7583480 x1875503571252864/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:751/0 lens 488/4536 e 0 to 0 dl 1788619911 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619900.203635:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c7583480 with x1875503571252864 00000100:00100000:2.0:1788619900.208725:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619900.208854:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571252992 00000100:00100000:0.0:1788619900.208869:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571252992 00000100:00100000:0.0:1788619900.209044:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 579 00000100:00100000:0.0:1788619900.209058:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df965880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+37:2352:x1875503571252992:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619900.209095:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619900.209127:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4ba00/0x1cd7fe53687334a8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ab67 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.209176:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4bc00/0x1cd7fe5368733477 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab36 expref: 32 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619900.209252:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5695500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589011712:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619900.209393:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78ce2fe180 going to sleep for 11 seconds 00000004:00010000:0.0:1788619900.211635:0:9417:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619899 -> 1788619900 00000100:00100000:0.0:1788619900.211683:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5695500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589011712:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619900.211714:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 209827 00010000:00010000:0.0:1788619900.211740:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4ba00/0x1cd7fe53687334a8 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab67 expref: 38 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.211767:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619900.211770:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619900.211776:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4ba00/0x1cd7fe53687334a8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab67 expref: 38 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.211794:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4ba00, rc 0) 00000100:00100000:0.0:1788619900.211918:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df965880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+37:2352:x1875503571252992:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2862us (3205us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619900.211927:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 579 00000100:00100000:0.0:1788619900.215662:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571253120 00000100:00100000:0.0:1788619900.215674:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571253120 00000100:00100000:0.0:1788619900.215779:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 580 00000100:00100000:0.0:1788619900.215793:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe060e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+37:2355:x1875503571253120:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619900.215821:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619900.215849:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4a400/0x1cd7fe53687334af lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ab6e expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.216039:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4bc00/0x1cd7fe5368733477 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab36 expref: 32 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619900.216103:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5695500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589011840:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619900.216270:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78ce2fe180 going to sleep for 11 seconds 00000100:00100000:0.0:1788619900.219414:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5695500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589011840:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619900.219454:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 209827 00010000:00010000:0.0:1788619900.219481:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4a400/0x1cd7fe53687334af lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab6e expref: 38 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.219508:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619900.219512:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619900.219517:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4a400/0x1cd7fe53687334af lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab6e expref: 38 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.219534:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4a400, rc 0) 00000100:00100000:0.0:1788619900.219659:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe060e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+37:2355:x1875503571253120:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3868us (4118us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619900.219669:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 580 00000100:00100000:2.0:1788619900.223242:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619900.223526:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571253248 00000100:00100000:0.0:1788619900.223541:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571253248 00000100:00100000:0.0:1788619900.223619:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 359 00000100:00100000:0.0:1788619900.223632:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571253248:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619900.223655:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619900.223678:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e52c06f9/0x1cd7fe53687334b6 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ab75 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619900.223779:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5695500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589011968:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619900.224111:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78ce2fe180 going to sleep for 11 seconds 00002000:00010000:0.0:1788619900.227586:0:7654:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619899 -> 1788619900 00000100:00100000:0.0:1788619900.227633:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5695500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589011968:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619900.227668:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e52c06f9/0x1cd7fe53687334b6 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab75 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.227699:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e52c06f9/0x1cd7fe53687334b6 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab75 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.227715:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4ba00, rc 0) 00000100:00100000:0.0:1788619900.227788:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571253248:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4158us (4564us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619900.227799:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 359 00000100:00100000:2.0:1788619900.236529:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619900.237010:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571253376 00000100:00100000:3.0:1788619900.237025:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571253376 00000100:00100000:3.0:1788619900.237098:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 360 00000100:00100000:3.0:1788619900.237111:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571253376:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619900.237134:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619900.237158:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008807c503/0x1cd7fe53687334bd lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ab7c expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619900.237234:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c30380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589012096:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619900.238148:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaab40 going to sleep for 11 seconds 00000100:00100000:3.0:1788619900.249109:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c30380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589012096:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619900.249160:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008807c503/0x1cd7fe53687334bd lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab7c expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.249190:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008807c503/0x1cd7fe53687334bd lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab7c expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.249203:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5049400, rc 0) 00000100:00100000:3.0:1788619900.249276:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571253376:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 12168us (12761us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619900.249286:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 360 00000100:00100000:3.0:1788619900.253742:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619900.253965:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571253504 00000100:00100000:0.0:1788619900.253979:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571253504 00000100:00100000:0.0:1788619900.254048:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 581 00000100:00100000:0.0:1788619900.254060:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c30380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+37:2353:x1875503571253504:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619900.254098:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619900.254126:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4ba00/0x1cd7fe53687334c4 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ab83 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.254174:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4bc00/0x1cd7fe5368733477 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab36 expref: 32 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619900.254250:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5697480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589012224:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619900.254629:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78ce2fe180 going to sleep for 11 seconds 00000004:00010000:3.0:1788619900.270889:0:5846:0:(mdt_lvb.c:177:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size: 209827 -> 1031643 00000004:00010000:3.0:1788619900.270912:0:5846:0:(mdt_lvb.c:183:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime: 1788619899 -> 1788619900 00000004:00010000:3.0:1788619900.270916:0:5846:0:(mdt_lvb.c:195:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime: 1788619899 -> 1788619900 00000100:00100000:3.0:1788619900.270963:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5697480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589012224:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619900.270997:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1031643 00010000:00010000:3.0:1788619900.271027:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4ba00/0x1cd7fe53687334c4 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab83 expref: 38 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.271056:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619900.271059:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619900.271067:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4ba00/0x1cd7fe53687334c4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab83 expref: 38 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.271109:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4ba00, rc 0) 00000100:00100000:3.0:1788619900.271195:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c30380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+37:2353:x1875503571253504:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 17136us (17468us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619900.271205:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 581 00000100:00100000:3.0:1788619900.275560:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619900.276985:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571253632 00000100:00100000:0.0:1788619900.277002:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571253632 00000100:00100000:0.0:1788619900.277098:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 582 00000100:00100000:0.0:1788619900.277111:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c32d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+37:2353:x1875503571253632:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619900.277140:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619900.277167:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4a400/0x1cd7fe53687334cb lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ab8a expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.277216:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4bc00/0x1cd7fe5368733477 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab36 expref: 32 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619900.277270:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5695c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589012352:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619900.277408:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78ce2fe540 going to sleep for 11 seconds 00000100:00100000:0.0:1788619900.287297:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5695c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589012352:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619900.287353:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1031643 00010000:00010000:0.0:1788619900.287381:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4a400/0x1cd7fe53687334cb lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab8a expref: 38 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.287409:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619900.287413:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619900.287420:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4a400/0x1cd7fe53687334cb lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab8a expref: 38 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.287455:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4a400, rc 0) 00000100:00100000:0.0:1788619900.287552:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c32d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+37:2353:x1875503571253632:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 10443us (12009us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619900.287564:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 582 00000100:00100000:3.0:1788619900.301520:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619900.301853:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571253888 00000100:00100000:2.0:1788619900.301883:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571253888 00000100:00100000:2.0:1788619900.302009:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 583 00000100:00100000:2.0:1788619900.302022:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c32a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+37:14908:x1875503571253888:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:2.0:1788619900.302109:0:5844:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619900.302146:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4a600/0x1cd7fe5368733470 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab2f expref: 37 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619900.302167:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619900.302174:0:5844:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788619900.304261:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d000/0x1cd7fe53687334d2 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619900.304301:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4b000/0x1cd7fe5368733446 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab0c expref: 32 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619900.304314:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d000/0x1cd7fe53687334d2 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619900.304394:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4b000/0x1cd7fe5368733446 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab0c expref: 32 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619900.304541:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4b000/0x1cd7fe5368733446 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab0c expref: 32 pid: 5846 timeout: 492 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619900.304561:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5ff4e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589012480:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619900.305317:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc000 going to sleep for 11 seconds 00000100:00100000:2.0:1788619900.311226:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5ff4e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589012480:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619900.311269:0:5844:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d000/0x1cd7fe53687334d2 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619900.314120:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571254016 00000100:00100000:2.0:1788619900.314134:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571254016 00010000:00010000:2.0:1788619900.314193:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4b000/0x1cd7fe5368733446 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab0c expref: 33 pid: 5846 timeout: 492 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619900.314287:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 192 00000100:00100000:2.0:1788619900.314301:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe063800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+33:2355:x1875503571254016:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619900.314321:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619900.314362:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4b000/0x1cd7fe5368733446 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab0c expref: 33 pid: 5846 timeout: 492 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619900.314392:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4b000/0x1cd7fe5368733446 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab0c expref: 33 pid: 5846 timeout: 492 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619900.314422:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4b000/0x1cd7fe5368733446 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab0c expref: 33 pid: 5846 timeout: 492 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619900.314438:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619900.314444:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d000/0x1cd7fe53687334d2 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619900.314456:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d000/0x1cd7fe53687334d2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619900.314465:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d000/0x1cd7fe53687334d2 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619900.314498:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788619900.314542:0:5844:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d000/0x1cd7fe53687334d2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.314554:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d000/0x1cd7fe53687334d2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619900.314599:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe063800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+32:2355:x1875503571254016:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 301us (614us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619900.314608:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 192 00000001:00080000:3.0:1788619900.315505:0:5844:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967531 00000004:00010000:3.0:1788619900.315572:0:5844:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1031643 -> 209827 00000004:00010000:3.0:1788619900.315577:0:5844:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619900 -> 1788619900 00000004:00010000:3.0:1788619900.315580:0:5844:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619900 -> 1788619899 00000004:00010000:3.0:1788619900.315583:0:5844:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619900 -> 1788619900 00000004:00010000:3.0:1788619900.315586:0:5844:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 774 -> 774 00000004:00010000:3.0:1788619900.315606:0:5844:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000b627e19d reply state 00000000af201a4a transno 4294967532 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d000/0x1cd7fe53687334d2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619900.315617:0:5844:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788619900.316015:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c32a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+39:14908:x1875503571253888:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 13993us (14511us total) trans 4294967532 rc 0/0 00000100:00100000:3.0:1788619900.316025:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 583 00000020:00080000:3.0:1788619900.316030:0:5844:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:3.0:1788619900.319153:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d000/0x1cd7fe53687334d2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.320547:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d000/0x1cd7fe53687334d2 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.320559:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d000/0x1cd7fe53687334d2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.320566:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c505d000 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:3.0:1788619900.320572:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d000/0x1cd7fe53687334d2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.320583:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d000/0x1cd7fe53687334d2 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.324965:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619900.324982:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788619900.324998:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d000/0x1cd7fe53687334d2 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.325012:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d000/0x1cd7fe53687334d2 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619900.329061:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619900.329536:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571254272 00000100:00100000:0.0:1788619900.329550:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571254272 00002000:00100000:0.0:1788619900.329887:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5c31880 x1875503571254272/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:751/0 lens 4040/0 e 0 to 0 dl 1788619911 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619900.330071:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5c31880 with x1875503571254272 00010000:00010000:0.0:1788619900.330092:0:5857:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4bc00/0x1cd7fe5368733477 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab36 expref: 32 pid: 9417 timeout: 492 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.330111:0:5857:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4bc00/0x1cd7fe5368733477 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab36 expref: 32 pid: 9417 timeout: 492 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788619900.330119:0:5857:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e78c5c31880 00000100:00100000:0.0:1788619900.330210:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 16 00000100:00100000:0.0:1788619900.330223:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c31880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+32:2354:x1875503571254272:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:0.0:1788619900.330832:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 209827 -> 1031643 00000004:00010000:0.0:1788619900.330837:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619900 -> 1788619900 00000004:00010000:0.0:1788619900.330840:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619899 -> 1788619899 00000004:00010000:0.0:1788619900.330843:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619900 -> 1788619900 00000004:00010000:0.0:1788619900.330846:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 774 -> 774 00000100:00100000:0.0:1788619900.330920:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c31880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+34:2354:x1875503571254272:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 700us (1874us total) trans 4294967533 rc 0/0 00000100:00100000:0.0:1788619900.330934:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 16 00002000:00100000:0.0:1788619900.330943:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5c31880 x1875503571254272/t4294967533(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:751/0 lens 4040/448 e 0 to 0 dl 1788619911 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619900.330951:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5c31880 with x1875503571254272 00010000:00010000:0.0:1788619900.330958:0:5857:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4bc00/0x1cd7fe5368733477 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab36 expref: 34 pid: 9417 timeout: 492 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.330970:0:5857:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4bc00/0x1cd7fe5368733477 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab36 expref: 34 pid: 9417 timeout: 492 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788619900.330977:0:5857:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e78c5c31880 00000100:00100000:2.0:1788619900.336452:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619900.337134:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571254400 00000100:00100000:0.0:1788619900.337149:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571254400 00010000:00010000:0.0:1788619900.337283:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4bc00/0x1cd7fe5368733477 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab36 expref: 34 pid: 9417 timeout: 492 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619900.338819:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 193 00000100:00100000:0.0:1788619900.338847:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff4e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+34:2353:x1875503571254400:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619900.338876:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619900.338931:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 1s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4bc00/0x1cd7fe5368733477 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab36 expref: 34 pid: 9417 timeout: 492 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.339037:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4bc00/0x1cd7fe5368733477 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab36 expref: 34 pid: 9417 timeout: 492 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.339083:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4bc00/0x1cd7fe5368733477 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab36 expref: 34 pid: 9417 timeout: 492 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.339127:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619900.339136:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50600/0x1cd7fe53687334d9 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab91 expref: 38 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.339150:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50600/0x1cd7fe53687334d9 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab91 expref: 38 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.339162:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50600/0x1cd7fe53687334d9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab91 expref: 38 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.339228:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50600/0x1cd7fe53687334d9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab91 expref: 38 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619900.339247:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5695c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589012736:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619900.339439:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78ce2fe780 going to sleep for 11 seconds 00000100:00100000:3.0:1788619900.350518:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619900.351719:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5695c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589012736:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619900.351749:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00080000:3.0:1788619900.351819:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1788619900.352564:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff4e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+33:2353:x1875503571254400:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 13730us (16126us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619900.352574:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 193 00000100:00100000:3.0:1788619900.358065:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571254656 00000100:00100000:3.0:1788619900.358087:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571254656 00002000:00100000:3.0:1788619900.358170:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1814528->1818623) req@ffff8e78fe062300 x1875503571254656/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:751/0 lens 488/0 e 0 to 0 dl 1788619911 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619900.358183:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fe062300 with x1875503571254656 ext(1814528->1818623) 00002000:00010000:3.0:1788619900.358191:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fe062300 00000100:00100000:3.0:1788619900.358259:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 202 00000100:00100000:3.0:1788619900.358273:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe062300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571254656:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619900.358667:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe062300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571254656:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 396us (1092us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619900.358676:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 202 00002000:00100000:3.0:1788619900.358684:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1814528->1818623) req@ffff8e78fe062300 x1875503571254656/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:751/0 lens 488/4536 e 0 to 0 dl 1788619911 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619900.358693:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fe062300 with x1875503571254656 ext(1814528->1818623) 00002000:00010000:3.0:1788619900.358698:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fe062300 00000100:00100000:3.0:1788619900.369146:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571254784 00000100:00100000:3.0:1788619900.369160:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571254784 00002000:00100000:3.0:1788619900.369245:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1818624->1822719) req@ffff8e78fe061f80 x1875503571254784/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:751/0 lens 488/0 e 0 to 0 dl 1788619911 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619900.369258:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fe061f80 with x1875503571254784 ext(1818624->1822719) 00002000:00010000:3.0:1788619900.369266:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fe061f80 00000100:00100000:3.0:1788619900.369300:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 203 00000100:00100000:3.0:1788619900.369311:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe061f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571254784:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619900.369688:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe061f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571254784:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 377us (719us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619900.369698:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 203 00002000:00100000:3.0:1788619900.369706:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1818624->1822719) req@ffff8e78fe061f80 x1875503571254784/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:751/0 lens 488/4536 e 0 to 0 dl 1788619911 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619900.369714:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fe061f80 with x1875503571254784 ext(1818624->1822719) 00002000:00010000:3.0:1788619900.369718:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fe061f80 00000100:00100000:3.0:1788619900.378512:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619900.378564:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571254912 00000100:00100000:0.0:1788619900.378579:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571254912 00002000:00100000:0.0:1788619900.379536:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1822720->1826815) req@ffff8e78fd93b100 x1875503571254912/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:751/0 lens 488/0 e 0 to 0 dl 1788619911 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619900.379552:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fd93b100 with x1875503571254912 ext(1822720->1826815) 00002000:00010000:0.0:1788619900.379572:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fd93b100 00000100:00100000:0.0:1788619900.379625:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 204 00000100:00100000:3.0:1788619900.394783:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619900.394829:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571255040 00000100:00100000:3.0:1788619900.394842:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571255040 00002000:00100000:3.0:1788619900.394905:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1826816->1830911) req@ffff8e78fd939880 x1875503571255040/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:751/0 lens 488/0 e 0 to 0 dl 1788619911 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619900.394918:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fd939880 with x1875503571255040 ext(1826816->1830911) 00002000:00010000:3.0:1788619900.394925:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fd939880 00000100:00100000:3.0:1788619900.394958:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 205 00000100:00100000:3.0:1788619900.394969:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd939880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2352:x1875503571255040:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619900.397602:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd939880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2352:x1875503571255040:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 2620us (2823us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619900.397618:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 205 00002000:00100000:3.0:1788619900.397632:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1826816->1830911) req@ffff8e78fd939880 x1875503571255040/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:751/0 lens 488/4536 e 0 to 0 dl 1788619911 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619900.397644:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fd939880 with x1875503571255040 ext(1826816->1830911) 00002000:00010000:3.0:1788619900.397653:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fd939880 00000100:00100000:3.0:1788619900.441660:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619900.464193:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619900.464419:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571255424 00000100:00100000:0.0:1788619900.464437:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571255424 00000100:00100000:0.0:1788619900.464509:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 361 00000100:00100000:0.0:1788619900.464522:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd938e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+10:2353:x1875503571255424:12345-192.168.201.5@tcp:16:fsx.0 00000100:00100000:3.0:1788619900.505703:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619900.505778:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571255552 00000100:00100000:3.0:1788619900.505791:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571255552 00000100:00100000:3.0:1788619900.505869:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 585 00000100:00100000:3.0:1788619900.505887:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd939500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+39:14908:x1875503571255552:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619900.506110:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619900.506157:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049400/0x1cd7fe53687334e0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ab98 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.506277:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049800/0x1cd7fe53687334e7 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.506297:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049800/0x1cd7fe53687334e7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.506313:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049800/0x1cd7fe53687334e7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788619900.506379:0:9417:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1834168 00000004:00200000:3.0:1788619900.506385:0:9417:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788619900.506412:0:9417:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049800/0x1cd7fe53687334e7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.506445:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049400/0x1cd7fe53687334e0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab98 expref: 41 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.506465:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049800/0x1cd7fe53687334e7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab98 expref: 40 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.506475:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619900.506479:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788619900.506484:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5049800, rc 0) 00000100:00100000:3.0:1788619900.506568:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd939500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+40:14908:x1875503571255552:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 688us (881us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619900.506576:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 585 00000100:00100000:3.0:1788619900.523502:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe060380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589012864:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619900.523575:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51800/0x1cd7fe53687334ee lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab9f expref: 34 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.523664:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619900.523668:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619900.523674:0:5844:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51800/0x1cd7fe53687334ee lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab9f expref: 34 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.523689:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b51800, rc 0) 00000100:00100000:3.0:1788619900.523843:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe063480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+34:14908:x1875503571255680:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8154us (8307us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619900.523854:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 586 00000100:00100000:3.0:1788619900.527384:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571255808 00000100:00100000:3.0:1788619900.527402:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571255808 00010000:00010000:3.0:1788619900.527459:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50600/0x1cd7fe53687334d9 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab91 expref: 40 pid: 11397 timeout: 493 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619900.527511:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 194 00000100:00100000:3.0:1788619900.527524:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c80ea450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+40:2355:x1875503571255808:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619900.527544:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619900.527579:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50600/0x1cd7fe53687334d9 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab91 expref: 40 pid: 11397 timeout: 493 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.527595:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50600/0x1cd7fe53687334d9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab91 expref: 40 pid: 11397 timeout: 493 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.527622:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50600/0x1cd7fe53687334d9 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab91 expref: 40 pid: 11397 timeout: 493 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.527644:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619900.527651:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51800/0x1cd7fe53687334ee lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab9f expref: 33 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.527662:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51800/0x1cd7fe53687334ee lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab9f expref: 33 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.527673:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51800/0x1cd7fe53687334ee lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab9f expref: 33 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.527726:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51800/0x1cd7fe53687334ee lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab9f expref: 33 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619900.527743:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fd93b480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589012992:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619900.527927:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619900.533660:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fd93b480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589012992:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619900.533695:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619900.533786:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c80ea450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+39:2355:x1875503571255808:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 6262us (6522us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619900.533801:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 194 00000100:00100000:2.0:1788619900.533953:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571255936 00000100:00100000:2.0:1788619900.533964:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571255936 00000100:00100000:2.0:1788619900.534125:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 244 00000100:00100000:2.0:1788619900.534139:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe061c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+34:2354:x1875503571255936:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:2.0:1788619900.534609:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1031643 -> 1031643 00000004:00010000:2.0:1788619900.534615:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619900 -> 1788619900 00000004:00010000:2.0:1788619900.534618:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619899 -> 1788619899 00000004:00010000:2.0:1788619900.534620:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619900 -> 1788619900 00000004:00010000:2.0:1788619900.534623:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 774 -> 774 00000100:00100000:2.0:1788619900.534755:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe061c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+35:2354:x1875503571255936:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 617us (1201us total) trans 4294967534 rc 0/0 00000100:00100000:2.0:1788619900.534764:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 244 00000100:00100000:3.0:1788619900.542699:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571256064 00000100:00100000:3.0:1788619900.542712:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571256064 00000100:00100000:3.0:1788619900.542795:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 587 00000100:00100000:3.0:1788619900.542807:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe061880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+39:2355:x1875503571256064:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619900.542836:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619900.542872:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049400/0x1cd7fe53687334f5 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1aba6 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.542940:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51800/0x1cd7fe53687334ee lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab9f expref: 34 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619900.543002:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fd93b480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589013120:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619900.543130:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c0c0 going to sleep for 11 seconds 00000004:00010000:3.0:1788619900.547821:0:11397:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619899 -> 1788619900 00000100:00100000:3.0:1788619900.547873:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fd93b480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589013120:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619900.548043:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1031643 00010000:00010000:3.0:1788619900.548082:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049400/0x1cd7fe53687334f5 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aba6 expref: 40 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.548110:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619900.548113:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619900.548120:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049400/0x1cd7fe53687334f5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aba6 expref: 40 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.548139:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5049400, rc 0) 00000100:00100000:3.0:1788619900.548444:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe061880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+39:2355:x1875503571256064:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5637us (5853us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619900.548456:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 587 00000100:00100000:2.0:1788619900.554496:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571256192 00000100:00100000:2.0:1788619900.554510:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571256192 00000100:00100000:2.0:1788619900.554591:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 588 00000100:00100000:2.0:1788619900.554603:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe060700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+39:2354:x1875503571256192:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619900.554633:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619900.554660:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d200/0x1cd7fe53687334fc lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1abad expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619900.554707:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51800/0x1cd7fe53687334ee lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab9f expref: 34 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619900.554772:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5ff4e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589013248:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619900.554886:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccf00 going to sleep for 11 seconds 00000100:00100000:3.0:1788619900.560239:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5ff4e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589013248:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619900.560282:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1031643 00010000:00010000:3.0:1788619900.560307:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d200/0x1cd7fe53687334fc lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1abad expref: 40 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.560452:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619900.560456:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619900.560463:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d200/0x1cd7fe53687334fc lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1abad expref: 40 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.560479:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505d200, rc 0) 00000100:00100000:3.0:1788619900.560557:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe060700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+39:2354:x1875503571256192:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5955us (7341us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619900.560568:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 588 00000100:00100000:3.0:1788619900.566706:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619900.570299:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571256320 00000100:00100000:2.0:1788619900.570318:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571256320 00000100:00100000:2.0:1788619900.570413:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 589 00000100:00100000:2.0:1788619900.570425:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd93b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+39:2355:x1875503571256320:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619900.570455:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619900.570486:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c200/0x1cd7fe5368733503 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1abb4 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619900.570542:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51800/0x1cd7fe53687334ee lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab9f expref: 34 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619900.570598:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5ff4e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589013376:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619900.570714:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619900.588475:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571256448 00000100:00100000:3.0:1788619900.588500:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571256448 00000100:00100000:3.0:1788619900.588596:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 590 00000100:00100000:3.0:1788619900.588610:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe062680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+39:14908:x1875503571256448:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619900.588639:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619900.588667:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049400/0x1cd7fe536873350a lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1abbb expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.588706:0:11397:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51800/0x1cd7fe53687334ee lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab9f expref: 34 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.588717:0:11397:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049400/0x1cd7fe536873350a lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1abbb expref: 40 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.588762:0:11397:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51800/0x1cd7fe53687334ee lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab9f expref: 34 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.589129:0:11397:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51800/0x1cd7fe53687334ee lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab9f expref: 34 pid: 5844 timeout: 493 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619900.589147:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fd93aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589013504:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619900.589275:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619900.597672:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619900.598317:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fd93aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589013504:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619900.598482:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049400/0x1cd7fe536873350a lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1abbb expref: 40 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.598508:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619900.598512:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619900.598516:0:11397:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049400/0x1cd7fe536873350a lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1abbb expref: 40 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.598529:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5049400, rc 0) 00000100:00100000:3.0:1788619900.598623:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe062680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+40:14908:x1875503571256448:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 10015us (13765us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619900.598634:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 590 00000100:00100000:3.0:1788619900.598691:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571256576 00000100:00100000:3.0:1788619900.598700:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571256576 00010000:00010000:3.0:1788619900.598742:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51800/0x1cd7fe53687334ee lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab9f expref: 35 pid: 5844 timeout: 493 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619900.598797:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 195 00000100:00100000:3.0:1788619900.598819:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd939f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+35:2354:x1875503571256576:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619900.598837:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619900.598870:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51800/0x1cd7fe53687334ee lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab9f expref: 35 pid: 5844 timeout: 493 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.598884:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51800/0x1cd7fe53687334ee lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab9f expref: 35 pid: 5844 timeout: 493 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.599057:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51800/0x1cd7fe53687334ee lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab9f expref: 35 pid: 5844 timeout: 493 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.599082:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619900.599089:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049400/0x1cd7fe536873350a lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1abbb expref: 39 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.599099:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049400/0x1cd7fe536873350a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1abbb expref: 39 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.599110:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049400/0x1cd7fe536873350a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1abbb expref: 39 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.599150:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049400/0x1cd7fe536873350a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1abbb expref: 39 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619900.599167:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fd93aa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589013632:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619900.599239:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619900.603202:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fd93aa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589013632:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619900.603249:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619900.603322:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd939f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+34:2354:x1875503571256576:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 4505us (5668us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619900.603369:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 195 00000100:00100000:2.0:1788619900.609381:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619900.612293:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571256704 00000100:00100000:0.0:1788619900.612316:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571256704 00002000:00100000:0.0:1788619900.612420:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5ff4e00 x1875503571256704/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:751/0 lens 488/0 e 0 to 0 dl 1788619911 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619900.612434:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5ff4e00 with x1875503571256704 00000100:00100000:0.0:1788619900.612482:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 245 00000100:00100000:0.0:1788619900.612493:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+40:2354:x1875503571256704:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619900.612804:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+40:2354:x1875503571256704:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 312us (3439us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619900.612814:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 245 00002000:00100000:0.0:1788619900.612822:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5ff4e00 x1875503571256704/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:751/0 lens 488/4536 e 0 to 0 dl 1788619911 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619900.612831:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5ff4e00 with x1875503571256704 00000100:00100000:0.0:1788619900.616023:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619900.616114:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571256832 00000100:00100000:3.0:1788619900.616128:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571256832 00002000:00100000:3.0:1788619900.616198:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5695c00 x1875503571256832/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:751/0 lens 4584/0 e 0 to 0 dl 1788619911 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619900.616212:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5695c00 with x1875503571256832 00000100:00100000:3.0:1788619900.616419:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 246 00000100:00100000:3.0:1788619900.616431:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5695c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+40:2352:x1875503571256832:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:3.0:1788619900.617449:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1031643 -> 1031643 00000004:00010000:3.0:1788619900.617460:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619900 -> 1788619900 00000004:00010000:3.0:1788619900.617463:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619900 -> 1788619899 00000004:00010000:3.0:1788619900.617466:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619900 -> 1788619900 00000004:00010000:3.0:1788619900.617468:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 774 -> 774 00000100:00100000:3.0:1788619900.617570:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5695c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+42:2352:x1875503571256832:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 1136us (1558us total) trans 4294967535 rc 0/0 00000100:00100000:3.0:1788619900.617583:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 246 00002000:00100000:3.0:1788619900.617594:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5695c00 x1875503571256832/t4294967535(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:751/0 lens 4584/448 e 0 to 0 dl 1788619911 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619900.617603:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5695c00 with x1875503571256832 00000100:00100000:0.0:1788619900.626369:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619900.626406:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571256960 00000100:00100000:3.0:1788619900.626420:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571256960 00000100:00100000:3.0:1788619900.626498:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 591 00000100:00100000:3.0:1788619900.626522:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5694e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+42:2352:x1875503571256960:12345-192.168.201.5@tcp:16:fsx.0 00000100:00100000:3.0:1788619900.703306:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5694e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+42:2352:x1875503571256960:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 76770us (76964us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619900.703358:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 591 00010000:00010000:0.0:1788619900.708587:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788619900.715920:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619900.721431:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619900.727449:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619900.734232:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619900.741089:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619900.741213:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571257600 00000100:00100000:2.0:1788619900.741226:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571257600 00002000:00100000:2.0:1788619900.741295:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1253376->1257471) req@ffff8e78c5697480 x1875503571257600/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:751/0 lens 488/0 e 0 to 0 dl 1788619911 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619900.741309:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5697480 with x1875503571257600 ext(1253376->1257471) 00002000:00010000:2.0:1788619900.741316:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5697480 00000100:00100000:2.0:1788619900.741397:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 212 00000100:00100000:2.0:1788619900.741410:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5697480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571257600:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619900.741923:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5697480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571257600:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 513us (857us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619900.741932:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 212 00002000:00100000:2.0:1788619900.741941:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1253376->1257471) req@ffff8e78c5697480 x1875503571257600/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:751/0 lens 488/4536 e 0 to 0 dl 1788619911 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619900.741951:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5697480 with x1875503571257600 ext(1253376->1257471) 00002000:00010000:2.0:1788619900.741956:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5697480 00000100:00100000:0.0:1788619900.747610:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619900.747733:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571257728 00000100:00100000:3.0:1788619900.747748:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571257728 00002000:00100000:3.0:1788619900.747821:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1257472->1261567) req@ffff8e78c5695500 x1875503571257728/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:751/0 lens 488/0 e 0 to 0 dl 1788619911 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619900.747835:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5695500 with x1875503571257728 ext(1257472->1261567) 00002000:00010000:3.0:1788619900.747843:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5695500 00000100:00100000:3.0:1788619900.747886:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 213 00000100:00100000:3.0:1788619900.748103:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5695500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571257728:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619900.748701:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5695500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571257728:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 601us (1104us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619900.748712:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 213 00002000:00100000:2.0:1788619900.748722:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1257472->1261567) req@ffff8e78c5695500 x1875503571257728/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:751/0 lens 488/4536 e 0 to 0 dl 1788619911 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619900.748733:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5695500 with x1875503571257728 ext(1257472->1261567) 00002000:00010000:2.0:1788619900.748737:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5695500 00000100:00100000:2.0:1788619900.753792:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619900.754763:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 214 00000100:00100000:0.0:1788619900.754780:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7430700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571257856:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619900.755218:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7430700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571257856:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 441us (1439us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619900.755227:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 214 00002000:00100000:0.0:1788619900.755236:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1261568->1265663) req@ffff8e78c7430700 x1875503571257856/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:751/0 lens 488/4536 e 0 to 0 dl 1788619911 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619900.755246:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7430700 with x1875503571257856 ext(1261568->1265663) 00002000:00010000:0.0:1788619900.755250:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c7430700 00000100:00100000:3.0:1788619900.759300:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619900.760227:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571257984 00000100:00100000:2.0:1788619900.760243:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571257984 00002000:00100000:2.0:1788619900.760321:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1265664->1269759) req@ffff8e78fd93b800 x1875503571257984/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:751/0 lens 488/0 e 0 to 0 dl 1788619911 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619900.760435:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fd93b800 with x1875503571257984 ext(1265664->1269759) 00002000:00010000:2.0:1788619900.760443:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fd93b800 00000100:00100000:2.0:1788619900.760477:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 215 00000100:00100000:2.0:1788619900.760489:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd93b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571257984:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619900.760853:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd93b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571257984:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 363us (1567us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619900.760882:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 215 00002000:00100000:2.0:1788619900.760969:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1265664->1269759) req@ffff8e78fd93b800 x1875503571257984/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:751/0 lens 488/4536 e 0 to 0 dl 1788619911 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619900.760978:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fd93b800 with x1875503571257984 ext(1265664->1269759) 00002000:00010000:2.0:1788619900.760982:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fd93b800 00000100:00100000:3.0:1788619900.766530:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619900.767061:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571258112 00000100:00100000:2.0:1788619900.767088:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571258112 00002000:00100000:2.0:1788619900.767166:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1269760->1273855) req@ffff8e78fd93bb80 x1875503571258112/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:751/0 lens 488/0 e 0 to 0 dl 1788619911 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619900.767180:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fd93bb80 with x1875503571258112 ext(1269760->1273855) 00002000:00010000:2.0:1788619900.767188:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fd93bb80 00000100:00100000:2.0:1788619900.767234:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 216 00000100:00100000:2.0:1788619900.767246:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd93bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571258112:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619900.767644:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd93bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571258112:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 398us (1130us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619900.767654:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 216 00002000:00100000:2.0:1788619900.767662:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1269760->1273855) req@ffff8e78fd93bb80 x1875503571258112/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:751/0 lens 488/4536 e 0 to 0 dl 1788619911 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619900.767671:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fd93bb80 with x1875503571258112 ext(1269760->1273855) 00002000:00010000:2.0:1788619900.767676:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fd93bb80 00000100:00100000:3.0:1788619900.771967:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619900.772396:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571258240 00000100:00100000:2.0:1788619900.772413:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571258240 00002000:00100000:2.0:1788619900.772492:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1273856->1277951) req@ffff8e78fd938380 x1875503571258240/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:751/0 lens 488/0 e 0 to 0 dl 1788619911 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619900.772530:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fd938380 with x1875503571258240 ext(1273856->1277951) 00002000:00010000:2.0:1788619900.772538:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fd938380 00000100:00100000:2.0:1788619900.772586:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 217 00000100:00100000:2.0:1788619900.772598:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd938380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571258240:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619900.773004:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd938380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571258240:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 406us (1056us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619900.773013:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 217 00002000:00100000:2.0:1788619900.774267:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1273856->1277951) req@ffff8e78fd938380 x1875503571258240/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:751/0 lens 488/4536 e 0 to 0 dl 1788619911 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619900.774284:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fd938380 with x1875503571258240 ext(1273856->1277951) 00002000:00010000:2.0:1788619900.774290:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fd938380 00000100:00100000:3.0:1788619900.778567:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619900.779489:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571258368 00000100:00100000:0.0:1788619900.779506:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571258368 00002000:00100000:0.0:1788619900.779585:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1277952->1282047) req@ffff8e78fd938a80 x1875503571258368/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:751/0 lens 488/0 e 0 to 0 dl 1788619911 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619900.779599:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fd938a80 with x1875503571258368 ext(1277952->1282047) 00002000:00010000:0.0:1788619900.779607:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fd938a80 00000100:00100000:0.0:1788619900.779642:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 218 00000100:00100000:0.0:1788619900.779654:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd938a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571258368:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619900.780389:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd938a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571258368:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 730us (1832us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619900.780401:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 218 00002000:00100000:0.0:1788619900.780410:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1277952->1282047) req@ffff8e78fd938a80 x1875503571258368/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:751/0 lens 488/4536 e 0 to 0 dl 1788619911 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619900.780422:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fd938a80 with x1875503571258368 ext(1277952->1282047) 00002000:00010000:0.0:1788619900.780427:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fd938a80 00000100:00100000:3.0:1788619900.785977:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619900.786245:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571258496 00000100:00100000:2.0:1788619900.786258:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571258496 00002000:00100000:2.0:1788619900.786493:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1282048->1286143) req@ffff8e78fd938700 x1875503571258496/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:751/0 lens 488/0 e 0 to 0 dl 1788619911 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619900.786507:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fd938700 with x1875503571258496 ext(1282048->1286143) 00002000:00010000:2.0:1788619900.786515:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fd938700 00000100:00100000:2.0:1788619900.786558:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 219 00000100:00100000:2.0:1788619900.786571:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd938700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571258496:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619900.787407:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd938700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571258496:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 835us (1442us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619900.787418:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 219 00002000:00100000:2.0:1788619900.787448:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1282048->1286143) req@ffff8e78fd938700 x1875503571258496/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:751/0 lens 488/4536 e 0 to 0 dl 1788619911 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619900.787460:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fd938700 with x1875503571258496 ext(1282048->1286143) 00002000:00010000:2.0:1788619900.787465:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fd938700 00000100:00100000:2.0:1788619900.793885:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619900.793981:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571258624 00000100:00100000:3.0:1788619900.793994:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571258624 00002000:00100000:3.0:1788619900.794086:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1236992->1286143) req@ffff8e78c7430a80 x1875503571258624/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:751/0 lens 488/0 e 0 to 0 dl 1788619911 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619900.794100:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7430a80 with x1875503571258624 ext(1236992->1286143) 00002000:00010000:3.0:1788619900.794108:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c7430a80 00000100:00100000:3.0:1788619900.794173:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 220 00000100:00100000:3.0:1788619900.794183:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7430a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571258624:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:3.0:1788619900.811942:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7430a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571258624:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 17742us (18053us total) trans 4294967354 rc 0/0 00000100:00100000:3.0:1788619900.811962:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 220 00002000:00100000:3.0:1788619900.811980:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1236992->1286143) req@ffff8e78c7430a80 x1875503571258624/t4294967354(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:751/0 lens 488/448 e 0 to 0 dl 1788619911 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619900.811992:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7430a80 with x1875503571258624 ext(1236992->1286143) 00002000:00010000:3.0:1788619900.812001:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c7430a80 00000100:00100000:2.0:1788619900.819653:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619900.819839:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571258752 00000100:00100000:0.0:1788619900.819853:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571258752 00000100:00100000:0.0:1788619900.819951:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 362 00000100:00100000:0.0:1788619900.819964:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7430000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571258752:12345-192.168.201.5@tcp:16:fsx.0 00000100:00100000:0.0:1788619900.870434:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7430000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571258752:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 50328us (50653us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619900.870450:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 362 00000100:00100000:0.0:1788619900.877714:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619900.885527:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571259008 00000100:00100000:3.0:1788619900.885546:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571259008 00000100:00100000:3.0:1788619900.885688:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 593 00000100:00100000:3.0:1788619900.885703:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe062a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571259008:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619900.885731:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619900.885774:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049a00/0x1cd7fe536873351f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1abc9 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.885825:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049400/0x1cd7fe536873350a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1abbb expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619900.885883:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed62d80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589013760:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619900.886038:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c600 going to sleep for 11 seconds 00000100:00100000:2.0:1788619900.890951:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619900.891265:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571259136 00000100:00100000:0.0:1788619900.891279:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571259136 00000100:00100000:0.0:1788619900.891643:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 363 00000100:00100000:0.0:1788619900.891658:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571259136:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619900.891686:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619900.891714:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e36287d4/0x1cd7fe5368733526 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1abd0 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619900.891812:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c75f5f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589013888:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619900.891938:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78e6f98780 going to sleep for 11 seconds 00000004:00010000:0.0:1788619900.893875:0:11397:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619899 -> 1788619900 00000100:00100000:0.0:1788619900.893921:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed62d80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589013760:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619900.893975:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1031643 00010000:00010000:0.0:1788619900.894004:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049a00/0x1cd7fe536873351f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1abc9 expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.894037:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619900.894041:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619900.894323:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049a00/0x1cd7fe536873351f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1abc9 expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.894515:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5049a00, rc 0) 00000100:00100000:0.0:1788619900.894654:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe062a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571259008:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8954us (10525us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619900.894663:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 593 00002000:00010000:0.0:1788619900.897706:0:7654:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619898 -> 1788619900 00000100:00100000:0.0:1788619900.897825:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c75f5f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589013888:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619900.897876:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e36287d4/0x1cd7fe5368733526 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1abd0 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.897908:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e36287d4/0x1cd7fe5368733526 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1abd0 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.897925:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4a800, rc 0) 00000100:00100000:0.0:1788619900.898259:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571259136:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6602us (7322us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619900.898271:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 363 00000100:00100000:2.0:1788619900.910570:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571259392 00000100:00100000:2.0:1788619900.910587:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571259392 00000100:00100000:2.0:1788619900.910664:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 364 00000100:00100000:2.0:1788619900.910676:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe061180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571259392:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619900.910702:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619900.910728:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000de00eb24/0x1cd7fe536873352d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1abde expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619900.910830:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589014016:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619900.910949:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccc00 going to sleep for 11 seconds 00000100:00100000:3.0:1788619900.911475:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571259264 00000100:00100000:3.0:1788619900.911482:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571259264 00000100:00100000:3.0:1788619900.911535:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 594 00000100:00100000:3.0:1788619900.911544:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe061500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571259264:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619900.911579:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619900.911599:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048600/0x1cd7fe5368733534 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1abd7 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.911638:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049400/0x1cd7fe536873350a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1abbb expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619900.911671:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed62680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589014144:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619900.911816:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619900.916393:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589014016:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619900.916450:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000de00eb24/0x1cd7fe536873352d lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1abde expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619900.916482:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000de00eb24/0x1cd7fe536873352d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1abde expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619900.916498:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505ca00, rc 0) 00000100:00100000:2.0:1788619900.916745:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe061180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571259392:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6061us (6285us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619900.916756:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 364 00000100:00100000:3.0:1788619900.925783:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed62680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589014144:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619900.925825:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1031643 00010000:00010000:3.0:1788619900.925850:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048600/0x1cd7fe5368733534 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1abd7 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.925875:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619900.925877:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619900.925883:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048600/0x1cd7fe5368733534 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1abd7 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.925902:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5048600, rc 0) 00000100:00100000:3.0:1788619900.925986:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe061500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571259264:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 14442us (15687us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619900.925997:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 594 00000100:00100000:2.0:1788619900.937220:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571259648 00000100:00100000:2.0:1788619900.937238:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571259648 00000100:00100000:2.0:1788619900.937308:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 595 00000100:00100000:2.0:1788619900.937321:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe063800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571259648:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:2.0:1788619900.937952:0:5844:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619900.937992:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049800/0x1cd7fe53687334e7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab98 expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619900.938019:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619900.938025:0:5844:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788619900.939494:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505ca00/0x1cd7fe536873353b lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619900.939522:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50000/0x1cd7fe5368733518 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1abc2 expref: 8 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619900.939530:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505ca00/0x1cd7fe536873353b lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619900.939573:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50000/0x1cd7fe5368733518 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1abc2 expref: 8 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619900.939586:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50000/0x1cd7fe5368733518 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1abc2 expref: 8 pid: 5844 timeout: 493 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619900.939601:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c0a80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589014272:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619900.940363:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccc00 going to sleep for 11 seconds 00000100:00100000:3.0:1788619900.952950:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571259776 00000100:00100000:3.0:1788619900.952963:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571259776 00010000:00010000:3.0:1788619900.953019:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50000/0x1cd7fe5368733518 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1abc2 expref: 9 pid: 5844 timeout: 493 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619900.953125:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 196 00000100:00100000:3.0:1788619900.953139:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe060380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571259776:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619900.953161:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619900.953171:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50000/0x1cd7fe5368733518 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1abc2 expref: 9 pid: 5844 timeout: 493 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.953187:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50000/0x1cd7fe5368733518 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1abc2 expref: 9 pid: 5844 timeout: 493 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.953216:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619900.953223:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505ca00/0x1cd7fe536873353b lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.953236:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505ca00/0x1cd7fe536873353b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.953245:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505ca00/0x1cd7fe536873353b lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.953269:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619900.953659:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe060380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571259776:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 520us (832us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619900.953669:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 196 00010000:00010000:2.0:1788619900.954236:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50000/0x1cd7fe5368733518 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1abc2 expref: 8 pid: 5844 timeout: 493 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619900.954259:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c0a80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589014272:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619900.954290:0:5844:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505ca00/0x1cd7fe536873353b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619900.954301:0:5844:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505ca00/0x1cd7fe536873353b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619900.954308:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505ca00/0x1cd7fe536873353b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788619900.954528:0:5844:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967536 00000004:00010000:2.0:1788619900.954596:0:5844:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1031643 -> 1031643 00000004:00010000:2.0:1788619900.954600:0:5844:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619900 -> 1788619900 00000004:00010000:2.0:1788619900.954603:0:5844:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619900 -> 1788619899 00000004:00010000:2.0:1788619900.954606:0:5844:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619900 -> 1788619900 00000004:00010000:2.0:1788619900.954609:0:5844:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 774 -> 646 00000004:00010000:2.0:1788619900.954616:0:5844:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000bf3b5735 reply state 0000000047a2fb56 transno 4294967537 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505ca00/0x1cd7fe536873353b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619900.954627:0:5844:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788619900.954747:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe063800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:14908:x1875503571259648:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 17417us (19125us total) trans 4294967537 rc 0/0 00000100:00100000:2.0:1788619900.954755:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 595 00000020:00080000:2.0:1788619900.954760:0:5844:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788619900.958900:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505ca00/0x1cd7fe536873353b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.958923:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505ca00/0x1cd7fe536873353b lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.958932:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505ca00/0x1cd7fe536873353b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.958964:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c505ca00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788619900.958971:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505ca00/0x1cd7fe536873353b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.958982:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505ca00/0x1cd7fe536873353b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.959006:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619900.959009:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788619900.959015:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505ca00/0x1cd7fe536873353b lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.959023:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505ca00/0x1cd7fe536873353b lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619900.961273:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571260032 00000100:00100000:2.0:1788619900.961286:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571260032 00002000:00010000:2.0:1788619900.961381:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:2.0:1788619900.961387:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa39880 with x1875503571260032 ext(0->18446744073709551615) 00002000:00010000:2.0:1788619900.961403:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa39880. 00000100:00100000:2.0:1788619900.961437:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 221 00000100:00100000:2.0:1788619900.961450:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa39880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571260032:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:2.0:1788619900.962471:0:6702:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1834168 -> 0 00002000:00010000:2.0:1788619900.962480:0:6702:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619900 -> 1788619900 00002000:00010000:2.0:1788619900.962483:0:6702:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619900 -> 1788619899 00002000:00010000:2.0:1788619900.962486:0:6702:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619900 -> 1788619900 00002000:00010000:2.0:1788619900.962488:0:6702:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 519 -> 391 00000100:00100000:2.0:1788619900.962574:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa39880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2353:x1875503571260032:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 1122us (1659us total) trans 4294967355 rc 0/0 00000100:00100000:2.0:1788619900.962585:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 221 00002000:00010000:2.0:1788619900.962590:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:2.0:1788619900.962595:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa39880 with x1875503571260032 ext(0->18446744073709551615) 00002000:00010000:2.0:1788619900.962600:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa39880. 00000100:00100000:0.0:1788619900.963320:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571259904 00000100:00100000:0.0:1788619900.963377:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571259904 00002000:00010000:0.0:1788619900.963425:0:5859:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:0.0:1788619900.963432:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78dfa38000 with x1875503571259904 00000100:00100000:0.0:1788619900.963472:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 247 00000100:00100000:0.0:1788619900.963482:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa38000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2353:x1875503571259904:12345-192.168.201.5@tcp:10:fsx.0 00000004:00010000:0.0:1788619900.963901:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1031643 -> 714697 00000004:00010000:0.0:1788619900.963907:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619900 -> 1788619900 00000004:00010000:0.0:1788619900.963909:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619899 -> 1788619900 00000004:00010000:0.0:1788619900.963912:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619900 -> 1788619900 00000004:00010000:0.0:1788619900.963914:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 646 -> 646 00000100:00100000:0.0:1788619900.963990:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa38000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2353:x1875503571259904:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 508us (3253us total) trans 4294967538 rc 0/0 00000100:00100000:0.0:1788619900.964373:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 247 00002000:00010000:0.0:1788619900.964379:0:5859:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:0.0:1788619900.964383:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78dfa38000 with x1875503571259904 00000100:00100000:0.0:1788619900.979553:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571260160 00000100:00100000:0.0:1788619900.979568:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571260160 00000100:00100000:0.0:1788619900.979640:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 596 00000100:00100000:0.0:1788619900.979650:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa3aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+8:14908:x1875503571260160:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619900.979674:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619900.979699:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4a800/0x1cd7fe5368733542 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1abe5 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.979813:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4be00/0x1cd7fe5368733549 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.979832:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4be00/0x1cd7fe5368733549 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.979843:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4be00/0x1cd7fe5368733549 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619900.979871:0:11397:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 714697 00000004:00200000:0.0:1788619900.979876:0:11397:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619900.979897:0:11397:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4be00/0x1cd7fe5368733549 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.979928:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4a800/0x1cd7fe5368733542 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1abe5 expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.979951:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4be00/0x1cd7fe5368733549 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1abe5 expref: 9 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619900.979984:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619900.979987:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619900.979991:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4be00, rc 0) 00000100:00100000:0.0:1788619900.980090:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa3aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571260160:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 439us (830us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619900.980097:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 596 00000100:00100000:3.0:1788619900.984552:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619900.984600:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571260288 00000100:00100000:3.0:1788619900.984613:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571260288 00000100:00100000:3.0:1788619900.984687:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 597 00000100:00100000:3.0:1788619900.984719:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed62680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571260288:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619900.984750:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619900.984780:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048600/0x1cd7fe5368733550 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1abec expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.984829:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049400/0x1cd7fe536873350a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1abbb expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619900.984881:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed62d80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589014400:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619900.985230:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619900.988874:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571260416 00000100:00100000:0.0:1788619900.988889:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571260416 00000100:00100000:0.0:1788619900.988980:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 365 00000100:00100000:0.0:1788619900.988993:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa3b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571260416:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619900.989102:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619900.989128:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e36287d4/0x1cd7fe5368733557 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1abf3 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619900.989217:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c75f5f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589014528:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619900.989323:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78e6f98780 going to sleep for 11 seconds 00000100:00100000:3.0:1788619900.989727:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed62d80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589014400:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619900.989766:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 714697 00010000:00010000:3.0:1788619900.989788:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048600/0x1cd7fe5368733550 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1abec expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.989813:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619900.989817:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619900.989825:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5048600/0x1cd7fe5368733550 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1abec expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619900.989843:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5048600, rc 0) 00000100:00100000:2.0:1788619900.989962:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed62680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571260288:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5245us (5424us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619900.989973:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 597 00000100:00100000:2.0:1788619901.003918:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571260544 00000100:00100000:2.0:1788619901.003931:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571260544 00000100:00100000:0.0:1788619901.004141:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571260672 00000100:00100000:3.0:1788619901.004142:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 598 00000100:00100000:0.0:1788619901.004146:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571260672 00000100:00100000:3.0:1788619901.004158:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa39180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571260544:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619901.004188:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1788619901.004193:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 366 00000100:00100000:0.0:1788619901.004201:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa3b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571260672:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619901.004210:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619901.004216:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049200/0x1cd7fe536873355e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1abfa expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619901.004221:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf7745d2/0x1cd7fe5368733565 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ac01 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619901.004298:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c75f4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589014656:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619901.004439:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78e6f983c0 going to sleep for 11 seconds 00010000:00010000:3.0:1788619901.005156:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049400/0x1cd7fe536873350a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1abbb expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619901.005199:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed62d80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589014784:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619901.005813:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619901.011858:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed62d80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589014784:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619901.011916:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 714697 00010000:00010000:3.0:1788619901.011942:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049200/0x1cd7fe536873355e lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1abfa expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619901.011969:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619901.011972:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619901.011978:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049200/0x1cd7fe536873355e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1abfa expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619901.011996:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5049200, rc 0) 00000100:00100000:3.0:1788619901.012085:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa39180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571260544:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7929us (8227us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619901.012095:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 598 00000100:00100000:0.0:1788619901.014703:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c75f4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589014656:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619901.014759:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000cf7745d2/0x1cd7fe5368733565 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ac01 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619901.014806:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf7745d2/0x1cd7fe5368733565 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ac01 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619901.014824:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4a000, rc 0) 00000100:00100000:0.0:1788619901.014952:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa3b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571260672:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 10750us (11014us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619901.014963:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 366 00000100:00100000:3.0:1788619901.024653:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571260800 00000100:00100000:3.0:1788619901.024667:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571260800 00002000:00100000:3.0:1788619901.024734:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78dfa3ad80 x1875503571260800/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:752/0 lens 488/0 e 0 to 0 dl 1788619912 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619901.024747:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78dfa3ad80 with x1875503571260800 00000100:00100000:3.0:1788619901.024831:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 248 00000100:00100000:2.0:1788619901.024887:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa3ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571260800:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619901.025274:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa3ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571260800:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 411us (1093us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619901.025293:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 248 00002000:00100000:2.0:1788619901.025301:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78dfa3ad80 x1875503571260800/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:752/0 lens 488/4536 e 0 to 0 dl 1788619912 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619901.025309:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78dfa3ad80 with x1875503571260800 00000100:00100000:0.0:1788619901.032588:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619901.032678:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571260928 00000100:00100000:2.0:1788619901.032690:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571260928 00002000:00100000:2.0:1788619901.032760:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c75f4700 x1875503571260928/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:752/0 lens 488/0 e 0 to 0 dl 1788619912 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619901.032770:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c75f4700 with x1875503571260928 00000100:00100000:2.0:1788619901.032811:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 249 00000100:00100000:2.0:1788619901.032823:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2353:x1875503571260928:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619901.033218:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2353:x1875503571260928:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 396us (647us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619901.033228:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 249 00002000:00100000:2.0:1788619901.033236:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c75f4700 x1875503571260928/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:752/0 lens 488/4536 e 0 to 0 dl 1788619912 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619901.033245:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c75f4700 with x1875503571260928 00000100:00100000:2.0:1788619901.040387:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571261056 00000100:00100000:2.0:1788619901.040401:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571261056 00002000:00100000:2.0:1788619901.040475:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78dfa38700 x1875503571261056/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:752/0 lens 488/0 e 0 to 0 dl 1788619912 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619901.040488:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78dfa38700 with x1875503571261056 00000100:00100000:2.0:1788619901.040535:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 250 00000100:00100000:2.0:1788619901.040548:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa38700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571261056:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619901.041917:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa38700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571261056:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 1361us (1639us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619901.041930:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 250 00002000:00100000:2.0:1788619901.041941:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78dfa38700 x1875503571261056/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:752/0 lens 488/4536 e 0 to 0 dl 1788619912 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619901.041952:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78dfa38700 with x1875503571261056 00000100:00100000:2.0:1788619901.050985:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571261184 00000100:00100000:2.0:1788619901.050999:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571261184 00002000:00100000:2.0:1788619901.051141:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78dfa39500 x1875503571261184/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:752/0 lens 488/0 e 0 to 0 dl 1788619912 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619901.051151:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78dfa39500 with x1875503571261184 00000100:00100000:2.0:1788619901.051185:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 251 00000100:00100000:2.0:1788619901.051194:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa39500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2353:x1875503571261184:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619901.051668:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa39500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2353:x1875503571261184:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 471us (1686us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619901.051680:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 251 00002000:00100000:0.0:1788619901.051688:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78dfa39500 x1875503571261184/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:752/0 lens 488/4536 e 0 to 0 dl 1788619912 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619901.051697:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78dfa39500 with x1875503571261184 00000100:00100000:0.0:1788619901.061320:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619901.061951:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571261312 00000100:00100000:2.0:1788619901.061966:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571261312 00002000:00100000:2.0:1788619901.062131:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c75f6300 x1875503571261312/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:752/0 lens 488/0 e 0 to 0 dl 1788619912 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619901.062144:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c75f6300 with x1875503571261312 00000100:00100000:2.0:1788619901.062194:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 252 00000100:00100000:2.0:1788619901.062206:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571261312:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619901.062578:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571261312:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 372us (1285us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619901.062587:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 252 00002000:00100000:2.0:1788619901.062595:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c75f6300 x1875503571261312/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:752/0 lens 488/4536 e 0 to 0 dl 1788619912 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619901.062622:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c75f6300 with x1875503571261312 00000100:00100000:2.0:1788619901.087947:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619901.088154:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571261440 00000100:00100000:0.0:1788619901.088168:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571261440 00002000:00100000:0.0:1788619901.088237:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c50c0a80 x1875503571261440/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:752/0 lens 488/0 e 0 to 0 dl 1788619912 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619901.088251:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c50c0a80 with x1875503571261440 00000100:00100000:0.0:1788619901.088317:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 253 00000100:00100000:0.0:1788619901.088373:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2353:x1875503571261440:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619901.088693:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2353:x1875503571261440:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 364us (763us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619901.088703:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 253 00002000:00100000:0.0:1788619901.088711:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c50c0a80 x1875503571261440/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:752/0 lens 488/4536 e 0 to 0 dl 1788619912 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619901.088720:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c50c0a80 with x1875503571261440 00000100:00100000:2.0:1788619901.107519:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619901.119486:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619901.123458:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571261696 00000100:00100000:2.0:1788619901.123482:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571261696 00002000:00100000:2.0:1788619901.123570:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fed62d80 x1875503571261696/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:752/0 lens 488/0 e 0 to 0 dl 1788619912 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619901.123585:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fed62d80 with x1875503571261696 00000100:00100000:0.0:1788619901.125663:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 255 00000100:00100000:0.0:1788619901.125697:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed62d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2353:x1875503571261696:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619901.126212:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed62d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2353:x1875503571261696:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 531us (6741us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619901.126222:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 255 00002000:00100000:0.0:1788619901.126233:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fed62d80 x1875503571261696/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:752/0 lens 488/4536 e 0 to 0 dl 1788619912 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619901.126245:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fed62d80 with x1875503571261696 00000100:00100000:2.0:1788619901.144748:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571261824 00000100:00100000:2.0:1788619901.144763:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571261824 00002000:00100000:2.0:1788619901.144844:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78dfa38380 x1875503571261824/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:752/0 lens 488/0 e 0 to 0 dl 1788619912 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619901.144860:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78dfa38380 with x1875503571261824 00000100:00100000:2.0:1788619901.144915:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 256 00000100:00100000:2.0:1788619901.144928:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa38380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2352:x1875503571261824:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619901.145294:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa38380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2352:x1875503571261824:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 365us (773us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619901.145303:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 256 00002000:00100000:2.0:1788619901.145312:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78dfa38380 x1875503571261824/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:752/0 lens 488/4536 e 0 to 0 dl 1788619912 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619901.145320:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78dfa38380 with x1875503571261824 00000100:00100000:2.0:1788619901.149581:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619901.149671:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571261952 00000100:00100000:0.0:1788619901.149684:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571261952 00002000:00100000:0.0:1788619901.149754:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5c09500 x1875503571261952/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:752/0 lens 488/0 e 0 to 0 dl 1788619912 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619901.149768:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5c09500 with x1875503571261952 00000100:00100000:0.0:1788619901.149813:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 257 00000100:00100000:0.0:1788619901.149824:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c09500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2353:x1875503571261952:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:0.0:1788619901.163765:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 714697 -> 714697 00000004:00010000:0.0:1788619901.163789:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619900 -> 1788619901 00000004:00010000:0.0:1788619901.163792:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619900 -> 1788619900 00000004:00010000:0.0:1788619901.163793:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619900 -> 1788619901 00000004:00010000:0.0:1788619901.163795:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 646 -> 646 00000100:00100000:0.0:1788619901.163964:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c09500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2353:x1875503571261952:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 14132us (14388us total) trans 4294967539 rc 0/0 00000100:00100000:0.0:1788619901.163978:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 257 00002000:00100000:0.0:1788619901.163996:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5c09500 x1875503571261952/t4294967539(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:752/0 lens 488/448 e 0 to 0 dl 1788619912 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619901.164018:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5c09500 with x1875503571261952 00000100:00100000:2.0:1788619901.169199:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000001:00080000:0.0:1788619901.204521:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967539 is committed 00000100:00100000:0.0:1788619901.204702:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c09880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571262080:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 35183us (35511us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619901.204717:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 599 00000100:00100000:0.0:1788619901.210271:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571262336 00000100:00100000:0.0:1788619901.210285:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571262336 00000100:00100000:0.0:1788619901.210622:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 367 00000100:00100000:0.0:1788619901.210638:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a89f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571262336:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619901.210666:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619901.210694:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eabba910/0x1cd7fe536873356c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ac0f expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619901.210784:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c75f4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589014912:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619901.210924:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78e6f98b40 going to sleep for 11 seconds 00000100:00100000:2.0:1788619901.213418:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571262208 00000100:00100000:2.0:1788619901.213454:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571262208 00000100:00100000:2.0:1788619901.213533:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 600 00000100:00100000:2.0:1788619901.213545:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a8bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571262208:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619901.213570:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619901.213596:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505de00/0x1cd7fe5368733573 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ac08 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619901.213644:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049400/0x1cd7fe536873350a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1abbb expref: 11 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619901.213696:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c0b100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589015040:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619901.213754:0:7654:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime: 1788619900 -> 1788619901 00002000:00010000:3.0:1788619901.213761:0:7654:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime: 1788619900 -> 1788619901 00000100:00100000:3.0:1788619901.213789:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c75f4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589014912:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619901.213830:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cca80 going to sleep for 11 seconds 00010000:00010000:3.0:1788619901.213839:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000eabba910/0x1cd7fe536873356c lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ac0f expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619901.213868:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eabba910/0x1cd7fe536873356c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ac0f expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619901.215315:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4b800, rc 0) 00000100:00100000:3.0:1788619901.215469:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a89f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571262336:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4829us (5294us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619901.215481:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 367 00000100:00100000:0.0:1788619901.222602:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c0b100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589015040:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619901.222672:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 714697 00010000:00010000:0.0:1788619901.222700:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505de00/0x1cd7fe5368733573 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ac08 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619901.222729:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619901.222732:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619901.222739:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505de00/0x1cd7fe5368733573 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ac08 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619901.222757:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505de00, rc 0) 00000100:00100000:0.0:1788619901.222860:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a8bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571262208:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 9316us (12817us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619901.222869:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 600 00000100:00100000:0.0:1788619901.227897:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619901.227956:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619901.228915:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571262464 00000100:00100000:2.0:1788619901.228922:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571262464 00000100:00100000:2.0:1788619901.228973:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 601 00000100:00100000:2.0:1788619901.228982:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571262464:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619901.228996:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619901.229013:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d400/0x1cd7fe5368733581 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ac16 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619901.229051:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049400/0x1cd7fe536873350a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1abbb expref: 11 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619901.229097:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c09f80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589015296:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619901.229164:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc780 going to sleep for 11 seconds 00000100:00100000:2.0:1788619901.236766:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c09f80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589015296:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619901.236787:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 714697 00010000:00010000:2.0:1788619901.236809:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d400/0x1cd7fe5368733581 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ac16 expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619901.236825:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619901.236828:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619901.236832:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d400/0x1cd7fe5368733581 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ac16 expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619901.236846:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505d400, rc 0) 00000100:00100000:2.0:1788619901.236880:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571262464:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7900us (8924us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619901.236887:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 601 00000100:00100000:2.0:1788619901.241237:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571262848 00000100:00100000:2.0:1788619901.241249:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571262848 00000100:00100000:2.0:1788619901.241321:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 369 00000100:00100000:2.0:1788619901.241363:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a8b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571262848:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619901.241387:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619901.241411:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071354985/0x1cd7fe5368733588 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ac2b expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619901.241491:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c09f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589015424:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619901.241596:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc780 going to sleep for 11 seconds 00000100:00100000:0.0:1788619901.241828:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571262720 00000100:00100000:0.0:1788619901.241833:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571262720 00000100:00100000:0.0:1788619901.241880:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 602 00000100:00100000:0.0:1788619901.241888:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a89880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571262720:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619901.241902:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619901.241918:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4a600/0x1cd7fe536873358f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ac24 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619901.241955:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049400/0x1cd7fe536873350a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1abbb expref: 11 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619901.241986:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c75f6680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589015552:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619901.242041:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78e6f98480 going to sleep for 11 seconds 00000100:00100000:2.0:1788619901.251786:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c09f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589015424:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619901.251808:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000071354985/0x1cd7fe5368733588 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ac2b expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619901.251822:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071354985/0x1cd7fe5368733588 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ac2b expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619901.251846:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505c800, rc 0) 00000100:00100000:2.0:1788619901.251864:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a8b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571262848:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 10506us (10776us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619901.251869:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 369 00000100:00100000:0.0:1788619901.263948:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571262976 00000100:00100000:0.0:1788619901.263962:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571262976 00000100:00100000:0.0:1788619901.264042:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 603 00000100:00100000:0.0:1788619901.264066:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a8a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571262976:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619901.264094:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619901.264122:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4ac00/0x1cd7fe5368733596 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ac32 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619901.264172:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049400/0x1cd7fe536873350a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1abbb expref: 11 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619901.264225:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c75f7b80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589015680:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619901.264376:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78e6f98a80 going to sleep for 11 seconds 00000100:00100000:2.0:1788619901.264557:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571263104 00000100:00100000:2.0:1788619901.264563:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571263104 00000100:00100000:2.0:1788619901.264609:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 370 00000100:00100000:2.0:1788619901.264618:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b35500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571263104:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619901.264628:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619901.264642:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071354985/0x1cd7fe536873359d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ac39 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619901.264692:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c09f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589015808:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619901.264951:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc780 going to sleep for 11 seconds 00000100:00100000:2.0:1788619901.274039:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c09f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589015808:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619901.274113:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000071354985/0x1cd7fe536873359d lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ac39 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619901.274158:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071354985/0x1cd7fe536873359d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ac39 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619901.274175:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505c800, rc 0) 00000100:00100000:2.0:1788619901.275466:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b35500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571263104:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 10837us (11632us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619901.275500:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 370 00000100:00100000:3.0:1788619901.280521:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c75f7b80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589015680:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619901.280567:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 714697 00010000:00010000:3.0:1788619901.280593:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4ac00/0x1cd7fe5368733596 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ac32 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619901.280620:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619901.280623:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619901.280630:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4ac00/0x1cd7fe5368733596 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ac32 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619901.280647:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4ac00, rc 0) 00000100:00100000:3.0:1788619901.280747:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a8a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571262976:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 16694us (17010us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619901.280776:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 603 00000100:00100000:3.0:1788619901.290783:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619901.290888:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571263232 00000100:00100000:2.0:1788619901.290903:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571263232 00000100:00100000:2.0:1788619901.290982:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 604 00000100:00100000:2.0:1788619901.290996:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+12:14908:x1875503571263232:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619901.291025:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619901.291052:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c800/0x1cd7fe53687335a4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ac40 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619901.292027:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d400/0x1cd7fe53687335ab lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619901.292054:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d400/0x1cd7fe53687335ab lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619901.292081:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d400/0x1cd7fe53687335ab lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788619901.292419:0:11397:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 714697 00000004:00200000:2.0:1788619901.292427:0:11397:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788619901.292456:0:11397:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d400/0x1cd7fe53687335ab lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619901.292504:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c800/0x1cd7fe53687335a4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ac40 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619901.292528:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d400/0x1cd7fe53687335ab lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ac40 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619901.292565:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619901.292568:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788619901.292573:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505d400, rc 0) 00000100:00100000:2.0:1788619901.292656:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571263232:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 1662us (1887us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619901.292665:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 604 00000100:00100000:3.0:1788619901.303604:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571263488 00000100:00100000:3.0:1788619901.303618:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571263488 00000100:00100000:3.0:1788619901.303700:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 371 00000100:00100000:3.0:1788619901.303713:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b36d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571263488:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619901.303737:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619901.303762:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000039c2d152/0x1cd7fe53687335b2 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ac4e expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619901.303770:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571263360 00000100:00100000:0.0:1788619901.303777:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571263360 00000100:00100000:0.0:1788619901.303825:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 605 00000100:00100000:0.0:1788619901.303835:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b35180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571263360:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619901.303849:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1788619901.303854:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900d5c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589015936:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619901.303881:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4b200/0x1cd7fe53687335b9 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ac47 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619901.303921:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049400/0x1cd7fe536873350a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1abbb expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619901.303955:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c75f5880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589016064:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619901.303968:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c780 going to sleep for 11 seconds 00000100:00100000:0.0:1788619901.303988:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78e6f98900 going to sleep for 11 seconds 00000100:00100000:3.0:1788619901.318602:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900d5c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589015936:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619901.318635:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000039c2d152/0x1cd7fe53687335b2 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ac4e expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619901.318656:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000039c2d152/0x1cd7fe53687335b2 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ac4e expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619901.318668:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5048600, rc 0) 00000100:00100000:3.0:1788619901.318717:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b36d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571263488:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 15007us (15266us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619901.318725:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 371 00000100:00100000:0.0:1788619901.322312:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619901.322523:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571263616 00000100:00100000:3.0:1788619901.322558:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571263616 00000100:00100000:3.0:1788619901.322643:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 606 00000100:00100000:3.0:1788619901.322656:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571263616:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619901.322688:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619901.322860:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049200/0x1cd7fe53687335c0 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ac55 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619901.322969:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049400/0x1cd7fe536873350a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1abbb expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619901.322985:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049200/0x1cd7fe53687335c0 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ac55 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619901.323037:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049400/0x1cd7fe536873350a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1abbb expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619901.323053:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049400/0x1cd7fe536873350a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1abbb expref: 12 pid: 11397 timeout: 493 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619901.323081:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900d5c000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589016192:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619901.323194:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c780 going to sleep for 11 seconds 00000100:00100000:0.0:1788619901.329318:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619901.329556:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900d5c000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589016192:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619901.329604:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049200/0x1cd7fe53687335c0 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ac55 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619901.329631:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619901.329636:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619901.329656:0:5845:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049200/0x1cd7fe53687335c0 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ac55 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619901.329671:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5049200, rc 0) 00000100:00100000:3.0:1788619901.329771:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:14908:x1875503571263616:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7117us (7476us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619901.329782:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 606 00000100:00100000:0.0:1788619901.343975:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619901.351482:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619901.351615:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571264000 00000100:00100000:2.0:1788619901.351631:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571264000 00002000:00100000:2.0:1788619901.351713:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c75f6a00 x1875503571264000/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:752/0 lens 488/0 e 0 to 0 dl 1788619912 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619901.351727:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c75f6a00 with x1875503571264000 00000100:00100000:2.0:1788619901.351791:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 259 00000100:00100000:2.0:1788619901.351822:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571264000:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619901.352323:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571264000:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 520us (856us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619901.352389:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 259 00002000:00100000:2.0:1788619901.352398:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c75f6a00 x1875503571264000/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:752/0 lens 488/4536 e 0 to 0 dl 1788619912 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619901.352408:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c75f6a00 with x1875503571264000 00000100:00100000:0.0:1788619901.368892:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619901.369197:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571264128 00000100:00100000:2.0:1788619901.369210:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571264128 00002000:00100000:2.0:1788619901.369280:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c75f7800 x1875503571264128/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:752/0 lens 488/0 e 0 to 0 dl 1788619912 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619901.369294:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c75f7800 with x1875503571264128 00000100:00100000:2.0:1788619901.369466:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 260 00000100:00100000:2.0:1788619901.369480:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571264128:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619901.371419:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571264128:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 1925us (2536us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619901.371437:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 260 00002000:00100000:2.0:1788619901.371451:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c75f7800 x1875503571264128/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:752/0 lens 488/4536 e 0 to 0 dl 1788619912 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619901.371463:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c75f7800 with x1875503571264128 00000100:00100000:0.0:1788619901.378921:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619901.395199:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619901.395451:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571264384 00000100:00100000:2.0:1788619901.395464:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571264384 00000100:00100000:2.0:1788619901.395538:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 607 00000100:00100000:2.0:1788619901.395549:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+12:2352:x1875503571264384:12345-192.168.201.5@tcp:16:fsx.0 00000001:00080000:0.0:1788619901.484986:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967540 is committed 00000100:00100000:2.0:1788619901.488519:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571264384:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 92952us (93317us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619901.488539:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 607 00000100:00100000:0.0:1788619901.493781:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619901.514390:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619901.538393:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619901.538592:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571264768 00000100:00100000:2.0:1788619901.538608:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571264768 00002000:00100000:2.0:1788619901.538692:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c75f4e00 x1875503571264768/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:752/0 lens 504/0 e 0 to 0 dl 1788619912 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619901.538709:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c75f4e00 with x1875503571264768 00000100:00100000:2.0:1788619901.538762:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 262 00000100:00100000:2.0:1788619901.538774:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571264768:12345-192.168.201.5@tcp:3:fsx.0 00000004:00010000:2.0:1788619901.565071:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 646 -> 774 00000100:00100000:2.0:1788619901.565275:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571264768:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 26494us (26956us total) trans 0 rc 704512/704512 00000100:00100000:2.0:1788619901.565313:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 262 00002000:00100000:2.0:1788619901.565353:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c75f4e00 x1875503571264768/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:752/0 lens 504/440 e 0 to 0 dl 1788619912 ref 1 fl Complete:/600/0 rc 704512/704512 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619901.565365:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c75f4e00 with x1875503571264768 00000100:00100000:0.0:1788619901.573698:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619901.591504:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619901.591606:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571265024 00000100:00100000:3.0:1788619901.591620:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571265024 00000100:00100000:3.0:1788619901.591808:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 609 00000100:00100000:3.0:1788619901.591821:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2355:x1875503571265024:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619901.591849:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619901.591888:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe53687335dc lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ac71 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619901.591950:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049200/0x1cd7fe53687335c0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ac55 expref: 9 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619901.592014:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900d5c000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589016832:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619901.592143:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c240 going to sleep for 11 seconds 00000100:00100000:3.0:1788619901.597690:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900d5c000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589016832:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619901.597732:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 714697 00010000:00010000:3.0:1788619901.597759:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe53687335dc lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ac71 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619901.597786:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619901.597790:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619901.597819:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe53687335dc lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ac71 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619901.597838:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683cc00, rc 0) 00000100:00100000:3.0:1788619901.597976:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2355:x1875503571265024:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6156us (6483us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619901.597986:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 609 00000100:00100000:2.0:1788619901.604139:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571265152 00000100:00100000:2.0:1788619901.604153:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571265152 00000100:00100000:2.0:1788619901.604270:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 610 00000100:00100000:2.0:1788619901.604284:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b34a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2355:x1875503571265152:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619901.604313:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619901.604370:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c800/0x1cd7fe53687335e3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ac78 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619901.604423:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049200/0x1cd7fe53687335c0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ac55 expref: 9 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619901.604479:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c09f80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589016960:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619901.604596:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619901.608757:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c09f80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589016960:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619901.608798:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 714697 00010000:00010000:2.0:1788619901.608825:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c800/0x1cd7fe53687335e3 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ac78 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619901.608854:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619901.608858:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619901.608865:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c800/0x1cd7fe53687335e3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ac78 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619901.608984:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505c800, rc 0) 00000100:00100000:2.0:1788619901.609203:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b34a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2355:x1875503571265152:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4921us (5383us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619901.609214:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 610 00000100:00100000:0.0:1788619901.615386:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571265280 00000100:00100000:0.0:1788619901.615399:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571265280 00000100:00100000:0.0:1788619901.615482:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 611 00000100:00100000:0.0:1788619901.615495:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5025880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2355:x1875503571265280:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619901.615525:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619901.615556:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4b400/0x1cd7fe53687335ea lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ac7f expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619901.615609:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049200/0x1cd7fe53687335c0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ac55 expref: 9 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619901.615679:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c71f1500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589017088:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619901.615804:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7d80 going to sleep for 11 seconds 00000100:00100000:0.0:1788619901.622178:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c71f1500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589017088:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619901.622222:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 714697 00010000:00010000:0.0:1788619901.622260:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4b400/0x1cd7fe53687335ea lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ac7f expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619901.622290:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619901.622293:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619901.622301:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4b400/0x1cd7fe53687335ea lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ac7f expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619901.622320:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4b400, rc 0) 00000100:00100000:2.0:1788619901.630612:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619901.631097:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571265408 00000100:00100000:0.0:1788619901.631112:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571265408 00000100:00100000:0.0:1788619901.631245:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 374 00000100:00100000:0.0:1788619901.631258:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c09f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571265408:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619901.631285:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619901.631311:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d00ed39d/0x1cd7fe53687335f1 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ac86 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619901.631452:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c71f1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589017216:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619901.631569:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7d80 going to sleep for 11 seconds 00002000:00010000:0.0:1788619901.637600:0:6698:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619900 -> 1788619901 00000100:00100000:0.0:1788619901.637651:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c71f1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589017216:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619901.637709:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d00ed39d/0x1cd7fe53687335f1 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ac86 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619901.637742:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d00ed39d/0x1cd7fe53687335f1 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ac86 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619901.637791:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4a400, rc 0) 00000100:00100000:0.0:1788619901.637955:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c09f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571265408:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6698us (7356us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619901.637966:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 374 00000100:00100000:3.0:1788619901.642816:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619901.642926:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571265536 00000100:00100000:0.0:1788619901.642939:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571265536 00000100:00100000:0.0:1788619901.643118:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 375 00000100:00100000:0.0:1788619901.643132:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571265536:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619901.643157:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619901.643181:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000083d4a48f/0x1cd7fe53687335f8 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ac8d expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619901.643262:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c71f1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589017344:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619901.644398:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7d80 going to sleep for 11 seconds 00000100:00100000:0.0:1788619901.647291:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c71f1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589017344:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619901.647562:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000083d4a48f/0x1cd7fe53687335f8 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ac8d expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619901.647599:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000083d4a48f/0x1cd7fe53687335f8 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ac8d expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619901.647617:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4b400, rc 0) 00000100:00100000:0.0:1788619901.647759:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571265536:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4628us (4957us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619901.647794:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 375 00000100:00100000:3.0:1788619901.652831:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619901.659587:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619901.669658:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619901.691393:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571266048 00000100:00100000:3.0:1788619901.691408:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571266048 00000100:00100000:3.0:1788619901.691497:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 612 00000100:00100000:3.0:1788619901.691508:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5027480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571266048:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619901.691536:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619901.691562:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe536873360d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1aca2 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619901.691612:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049200/0x1cd7fe53687335c0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ac55 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619901.691661:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c74ab480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589017728:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619901.691766:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c240 going to sleep for 11 seconds 00000100:00100000:3.0:1788619901.694923:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c74ab480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589017728:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619901.694963:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 714697 00010000:00010000:3.0:1788619901.694992:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe536873360d lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aca2 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619901.695066:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619901.695070:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619901.695078:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe536873360d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aca2 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619901.695097:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683cc00, rc 0) 00000100:00100000:3.0:1788619901.695234:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5027480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571266048:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3726us (3977us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619901.695243:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 612 00000100:00100000:2.0:1788619901.706886:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571266176 00000100:00100000:2.0:1788619901.706900:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571266176 00000100:00100000:2.0:1788619901.706980:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 613 00000100:00100000:2.0:1788619901.706992:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5026300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571266176:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619901.707029:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619901.707068:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c600/0x1cd7fe5368733614 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1aca9 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619901.707119:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049200/0x1cd7fe53687335c0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ac55 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619901.707184:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c0b100 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589017856:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619901.707994:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc3c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619901.715599:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c0b100 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589017856:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619901.715640:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 714697 00010000:00010000:3.0:1788619901.715668:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c600/0x1cd7fe5368733614 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aca9 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619901.715697:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619901.715701:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619901.715707:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c600/0x1cd7fe5368733614 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aca9 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619901.715726:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505c600, rc 0) 00000100:00100000:3.0:1788619901.715804:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5026300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571266176:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8813us (9142us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619901.715814:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 613 00000100:00100000:0.0:1788619901.720976:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571266304 00000100:00100000:0.0:1788619901.720992:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571266304 00000100:00100000:0.0:1788619901.721094:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 378 00000100:00100000:0.0:1788619901.721108:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5024700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571266304:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619901.721133:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619901.721160:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d18e093/0x1cd7fe536873361b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1acb0 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619901.721247:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c71f1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589017984:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619901.721427:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7d80 going to sleep for 11 seconds 00002000:00010000:0.0:1788619901.725126:0:6698:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size: 0 -> 1472871 00000100:00100000:0.0:1788619901.725177:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c71f1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589017984:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619901.725231:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009d18e093/0x1cd7fe536873361b lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1acb0 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619901.725265:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d18e093/0x1cd7fe536873361b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1acb0 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619901.725282:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4bc00, rc 0) 00000100:00100000:0.0:1788619901.725765:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5024700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571266304:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4656us (4962us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619901.725778:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 378 00000100:00100000:2.0:1788619901.744761:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619901.747240:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571266432 00000100:00100000:0.0:1788619901.747306:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571266432 00002000:00100000:0.0:1788619901.747425:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5c0b100 x1875503571266432/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:752/0 lens 8680/0 e 0 to 0 dl 1788619912 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619901.747441:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5c0b100 with x1875503571266432 00000100:00100000:0.0:1788619901.747768:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 264 00000100:00100000:0.0:1788619901.747782:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c0b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571266432:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:0.0:1788619901.748830:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 714697 -> 714697 00000004:00010000:0.0:1788619901.748840:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619901 -> 1788619901 00000004:00010000:0.0:1788619901.748843:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619901 -> 1788619901 00000004:00010000:0.0:1788619901.748846:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619901 -> 1788619901 00000004:00010000:0.0:1788619901.748849:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 774 -> 774 00000100:00100000:0.0:1788619901.749218:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c0b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571266432:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 1434us (4467us total) trans 4294967542 rc 0/0 00000100:00100000:0.0:1788619901.749232:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 264 00002000:00100000:0.0:1788619901.749244:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5c0b100 x1875503571266432/t4294967542(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:752/0 lens 8680/448 e 0 to 0 dl 1788619912 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619901.749254:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5c0b100 with x1875503571266432 00000100:00100000:2.0:1788619901.765139:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571266560 00000100:00100000:2.0:1788619901.765164:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571266560 00000100:00100000:2.0:1788619901.765281:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 614 00000100:00100000:2.0:1788619901.765646:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5024000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571266560:12345-192.168.201.5@tcp:16:fsx.0 00000100:00100000:3.0:1788619901.834448:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5024000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571266560:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 69147us (73757us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619901.834489:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 614 00000100:00100000:0.0:1788619901.838276:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571266688 00000100:00100000:0.0:1788619901.838292:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571266688 00000100:00100000:0.0:1788619901.838429:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 379 00000100:00100000:0.0:1788619901.838444:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5025f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571266688:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619901.839221:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619901.839250:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d18e093/0x1cd7fe5368733622 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1acb7 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619901.839442:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c71f1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589018112:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619901.839752:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfcbc840 going to sleep for 11 seconds 00000100:00100000:0.0:1788619901.843249:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c71f1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589018112:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619901.843307:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009d18e093/0x1cd7fe5368733622 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1acb7 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619901.843384:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d18e093/0x1cd7fe5368733622 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1acb7 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619901.843401:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4bc00, rc 0) 00000100:00100000:0.0:1788619901.843764:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5025f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571266688:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5321us (5963us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619901.843776:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 379 00000100:00100000:2.0:1788619901.847986:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571266816 00000100:00100000:2.0:1788619901.848077:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571266816 00000100:00100000:2.0:1788619901.848216:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 380 00000100:00100000:2.0:1788619901.848231:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5027100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571266816:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619901.848255:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619901.848280:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cac82bb5/0x1cd7fe5368733629 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1acbe expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619901.848485:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df8b3100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589018240:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619901.848609:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619901.856928:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df8b3100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589018240:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619901.856989:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000cac82bb5/0x1cd7fe5368733629 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1acbe expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619901.857107:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cac82bb5/0x1cd7fe5368733629 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1acbe expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619901.857123:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505da00, rc 0) 00000100:00100000:2.0:1788619901.857208:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5027100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571266816:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8980us (9475us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619901.857219:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 380 00000100:00100000:3.0:1788619901.865780:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619901.866272:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571266944 00000100:00100000:0.0:1788619901.866287:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571266944 00000100:00100000:0.0:1788619901.866695:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 615 00000100:00100000:0.0:1788619901.866709:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74ab480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2353:x1875503571266944:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619901.866738:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619901.866764:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4bc00/0x1cd7fe5368733630 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1acc5 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619901.866812:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049200/0x1cd7fe53687335c0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ac55 expref: 9 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619901.866894:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c71f1500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589018368:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619901.867225:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfcbc840 going to sleep for 11 seconds 00000004:00010000:0.0:1788619901.881202:0:5845:0:(mdt_lvb.c:201:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks: 774 -> 902 00000100:00100000:0.0:1788619901.881251:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c71f1500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589018368:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619901.881286:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 714697 00010000:00010000:0.0:1788619901.881314:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4bc00/0x1cd7fe5368733630 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1acc5 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619901.881395:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619901.881399:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619901.881405:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4bc00/0x1cd7fe5368733630 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1acc5 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619901.881424:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4bc00, rc 0) 00000100:00100000:0.0:1788619901.881505:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74ab480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2353:x1875503571266944:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 14799us (15739us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619901.881515:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 615 00000100:00100000:3.0:1788619901.902925:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619901.903125:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571267072 00000100:00100000:2.0:1788619901.903138:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571267072 00000100:00100000:2.0:1788619901.903213:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 616 00000100:00100000:2.0:1788619901.903225:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74a9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2355:x1875503571267072:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619901.903253:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619901.903280:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505da00/0x1cd7fe5368733637 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1accc expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619901.903417:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049200/0x1cd7fe53687335c0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ac55 expref: 9 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619901.903474:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df8b3100 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589018496:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619901.903661:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619901.914710:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df8b3100 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589018496:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619901.914750:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 714697 00010000:00010000:2.0:1788619901.914778:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505da00/0x1cd7fe5368733637 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1accc expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619901.914809:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619901.914813:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619901.914820:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505da00/0x1cd7fe5368733637 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1accc expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619901.914840:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505da00, rc 0) 00000100:00100000:2.0:1788619901.914924:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74a9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2355:x1875503571267072:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 11699us (12014us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619901.914935:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 616 00000100:00100000:2.0:1788619901.925845:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571267200 00000100:00100000:2.0:1788619901.925868:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571267200 00000100:00100000:2.0:1788619901.926008:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 381 00000100:00100000:2.0:1788619901.926023:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe139c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571267200:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619901.926056:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619901.926082:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cac82bb5/0x1cd7fe536873363e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1acd3 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619901.926198:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df8b3100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589018624:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619901.926354:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619901.940751:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df8b3100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589018624:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619901.940809:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000cac82bb5/0x1cd7fe536873363e lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1acd3 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619901.940843:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cac82bb5/0x1cd7fe536873363e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1acd3 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619901.940860:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505da00, rc 0) 00000100:00100000:2.0:1788619901.940949:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe139c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571267200:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 14928us (19381us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619901.940959:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 381 00000100:00100000:2.0:1788619901.949655:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619901.949864:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571267328 00000100:00100000:0.0:1788619901.949878:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571267328 00000100:00100000:0.0:1788619901.949952:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 382 00000100:00100000:0.0:1788619901.949965:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571267328:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619901.949990:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619901.950015:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d18e093/0x1cd7fe5368733645 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1acda expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619901.950152:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c71f1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589018752:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619901.950271:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfcbc840 going to sleep for 11 seconds 00000100:00100000:0.0:1788619901.961437:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c71f1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589018752:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619901.961496:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009d18e093/0x1cd7fe5368733645 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1acda expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619901.961529:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d18e093/0x1cd7fe5368733645 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1acda expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619901.961545:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4bc00, rc 0) 00000100:00100000:0.0:1788619901.961677:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571267328:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 11713us (12035us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619901.961687:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 382 00000100:00100000:2.0:1788619901.994387:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571267456 00000100:00100000:2.0:1788619901.994403:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571267456 00002000:00100000:2.0:1788619901.994494:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5025180 x1875503571267456/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:752/0 lens 488/0 e 0 to 0 dl 1788619912 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619901.994510:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5025180 with x1875503571267456 00000100:00100000:2.0:1788619901.995976:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 265 00000100:00100000:2.0:1788619901.996004:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5025180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571267456:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619901.996450:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5025180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571267456:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 456us (2422us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619901.996484:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 265 00002000:00100000:2.0:1788619901.996495:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5025180 x1875503571267456/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:752/0 lens 488/440 e 0 to 0 dl 1788619912 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619901.996506:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5025180 with x1875503571267456 00000100:00100000:3.0:1788619902.001399:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619902.001655:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571267584 00000100:00100000:0.0:1788619902.001669:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571267584 00000100:00100000:0.0:1788619902.001749:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 383 00000100:00100000:0.0:1788619902.001760:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa15500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571267584:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619902.001780:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619902.001802:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d18e093/0x1cd7fe536873364c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ace1 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1788619902.001938:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1788619902.001959:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c71f1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589018880:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619902.002077:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfcbc840 going to sleep for 11 seconds 00000100:00100000:0.0:1788619902.004711:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c71f1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589018880:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619902.004761:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009d18e093/0x1cd7fe536873364c lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ace1 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619902.004788:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d18e093/0x1cd7fe536873364c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ace1 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619902.004801:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4bc00, rc 0) 00000100:00100000:0.0:1788619902.004864:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa15500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571267584:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3107us (3479us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619902.004878:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 383 00000100:00100000:3.0:1788619902.011920:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619902.012090:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571267712 00000100:00100000:2.0:1788619902.012105:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571267712 00000100:00100000:2.0:1788619902.012253:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 384 00000100:00100000:2.0:1788619902.012265:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa14a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571267712:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619902.012382:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619902.012408:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001b62f519/0x1cd7fe5368733653 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ace8 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619902.012495:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df8b3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589019008:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619902.012621:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619902.016537:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df8b3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589019008:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619902.016591:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001b62f519/0x1cd7fe5368733653 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ace8 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619902.016622:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001b62f519/0x1cd7fe5368733653 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ace8 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619902.016639:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505ce00, rc 0) 00000100:00100000:2.0:1788619902.016745:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa14a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571267712:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4481us (4840us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619902.016775:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 384 00000100:00100000:3.0:1788619902.036704:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571267840 00000100:00100000:3.0:1788619902.036721:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571267840 00002000:00100000:3.0:1788619902.036813:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5024380 x1875503571267840/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:753/0 lens 504/0 e 0 to 0 dl 1788619913 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619902.036826:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5024380 with x1875503571267840 00000100:00100000:3.0:1788619902.036878:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 266 00000100:00100000:3.0:1788619902.036891:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5024380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571267840:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619902.043393:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5024380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571267840:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 6487us (6764us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619902.043414:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 266 00002000:00100000:0.0:1788619902.043435:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5024380 x1875503571267840/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:753/0 lens 504/440 e 0 to 0 dl 1788619913 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619902.043456:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5024380 with x1875503571267840 00000100:00100000:0.0:1788619902.054714:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619902.062664:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619902.070055:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571268096 00000100:00100000:0.0:1788619902.070079:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571268096 00000100:00100000:0.0:1788619902.070184:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 386 00000100:00100000:0.0:1788619902.070197:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571268096:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619902.070223:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619902.070247:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008db03736/0x1cd7fe5368733661 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1acf6 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619902.070381:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b10380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589019264:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619902.070512:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfcbc840 going to sleep for 11 seconds 00000100:00100000:2.0:1788619902.077530:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b10380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589019264:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619902.077591:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008db03736/0x1cd7fe5368733661 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1acf6 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619902.077624:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008db03736/0x1cd7fe5368733661 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1acf6 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619902.077641:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4b000, rc 0) 00000100:00100000:2.0:1788619902.077784:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571268096:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7589us (15346us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619902.077795:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 386 00000100:00100000:0.0:1788619902.095513:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619902.098476:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571268224 00000100:00100000:2.0:1788619902.098502:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571268224 00002000:00100000:2.0:1788619902.098585:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b11500 x1875503571268224/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:753/0 lens 488/0 e 0 to 0 dl 1788619913 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619902.098598:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b11500 with x1875503571268224 00000100:00100000:2.0:1788619902.098714:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 267 00000100:00100000:2.0:1788619902.098727:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b11500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571268224:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619902.099129:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b11500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571268224:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 403us (3630us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619902.099138:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 267 00002000:00100000:2.0:1788619902.099146:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b11500 x1875503571268224/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:753/0 lens 488/440 e 0 to 0 dl 1788619913 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619902.099176:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b11500 with x1875503571268224 00000100:00100000:0.0:1788619902.110705:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571268352 00000100:00100000:0.0:1788619902.110719:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571268352 00000100:00100000:0.0:1788619902.110796:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 387 00000100:00100000:0.0:1788619902.110808:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5024700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571268352:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619902.110831:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619902.110855:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e52c06f9/0x1cd7fe5368733668 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1acfd expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619902.111563:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b13100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589019392:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619902.111736:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfcbc840 going to sleep for 11 seconds 00000100:00100000:3.0:1788619902.123742:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b13100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589019392:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619902.123815:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e52c06f9/0x1cd7fe5368733668 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1acfd expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619902.123848:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e52c06f9/0x1cd7fe5368733668 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1acfd expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619902.123863:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4ba00, rc 0) 00000100:00100000:3.0:1788619902.128535:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5024700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571268352:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 17707us (17938us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619902.128583:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 387 00000100:00100000:2.0:1788619902.136545:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619902.136686:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571268480 00000100:00100000:3.0:1788619902.136700:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571268480 00000100:00100000:3.0:1788619902.136773:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 388 00000100:00100000:3.0:1788619902.136786:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571268480:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619902.136812:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619902.136835:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007736e663/0x1cd7fe536873366f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ad04 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619902.137145:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589019520:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619902.137250:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48ccc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619902.147281:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589019520:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619902.147404:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007736e663/0x1cd7fe536873366f lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ad04 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619902.147439:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007736e663/0x1cd7fe536873366f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ad04 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619902.147456:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d400, rc 0) 00000100:00100000:3.0:1788619902.147542:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571268480:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 10758us (11012us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619902.147555:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 388 00000100:00100000:0.0:1788619902.156542:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571268608 00000100:00100000:0.0:1788619902.156553:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571268608 00002000:00100000:0.0:1788619902.156614:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5025f80 x1875503571268608/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:753/0 lens 488/0 e 0 to 0 dl 1788619913 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619902.156626:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5025f80 with x1875503571268608 00000100:00100000:0.0:1788619902.156662:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 268 00000100:00100000:0.0:1788619902.156671:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5025f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571268608:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619902.156947:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5025f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571268608:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 276us (641us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619902.156954:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 268 00002000:00100000:0.0:1788619902.156962:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5025f80 x1875503571268608/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:753/0 lens 488/440 e 0 to 0 dl 1788619913 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619902.156982:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5025f80 with x1875503571268608 00000100:00100000:2.0:1788619902.166699:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619902.190177:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619902.190316:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571268864 00000100:00100000:0.0:1788619902.190367:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571268864 00000100:00100000:0.0:1788619902.190453:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 390 00000100:00100000:0.0:1788619902.190466:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7582300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571268864:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619902.190490:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619902.190515:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071bcbab9/0x1cd7fe536873367d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ad12 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619902.190608:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b11180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589019776:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619902.190728:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfcbc840 going to sleep for 11 seconds 00000100:00100000:0.0:1788619902.205974:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b11180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589019776:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619902.206049:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000071bcbab9/0x1cd7fe536873367d lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ad12 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619902.206084:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071bcbab9/0x1cd7fe536873367d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ad12 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619902.206100:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4ac00, rc 0) 00000100:00100000:0.0:1788619902.206171:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7582300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571268864:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 15707us (16009us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619902.206182:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 390 00000100:00100000:2.0:1788619902.217808:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619902.240321:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619902.240909:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571269120 00000100:00100000:3.0:1788619902.240924:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571269120 00000100:00100000:3.0:1788619902.241026:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 391 00000100:00100000:3.0:1788619902.241050:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5771180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571269120:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619902.241075:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619902.241106:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007736e663/0x1cd7fe5368733684 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ad19 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619902.241185:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589019904:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619902.241302:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48ccc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619902.254742:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589019904:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619902.254802:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007736e663/0x1cd7fe5368733684 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ad19 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619902.254836:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007736e663/0x1cd7fe5368733684 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ad19 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619902.254853:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d400, rc 0) 00000100:00100000:3.0:1788619902.254936:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5771180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571269120:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 13898us (14642us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619902.254946:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 391 00000100:00100000:2.0:1788619902.274955:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619902.275067:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571269248 00000100:00100000:3.0:1788619902.275080:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571269248 00000100:00100000:3.0:1788619902.275150:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 392 00000100:00100000:3.0:1788619902.275161:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5770700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571269248:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619902.275183:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619902.275204:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007736e663/0x1cd7fe536873368b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ad20 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619902.275297:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589020032:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619902.275445:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48ccc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619902.287558:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589020032:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619902.287613:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007736e663/0x1cd7fe536873368b lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ad20 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619902.287645:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007736e663/0x1cd7fe536873368b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ad20 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619902.287660:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d400, rc 0) 00000100:00100000:3.0:1788619902.288013:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5770700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571269248:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 12850us (13069us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619902.288025:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 392 00000100:00100000:2.0:1788619902.296481:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619902.296587:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571269376 00000100:00100000:0.0:1788619902.296604:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571269376 00002000:00100000:0.0:1788619902.296705:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5772d80 x1875503571269376/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:753/0 lens 488/0 e 0 to 0 dl 1788619913 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619902.296718:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5772d80 with x1875503571269376 00000100:00100000:0.0:1788619902.296768:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 270 00000100:00100000:0.0:1788619902.296789:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5772d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571269376:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619902.297201:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5772d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571269376:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 423us (899us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619902.297210:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 270 00002000:00100000:0.0:1788619902.297217:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5772d80 x1875503571269376/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:753/0 lens 488/440 e 0 to 0 dl 1788619913 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619902.297226:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5772d80 with x1875503571269376 00000100:00100000:2.0:1788619902.318754:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619902.337872:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619902.338011:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571269632 00000100:00100000:0.0:1788619902.338024:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571269632 00000100:00100000:0.0:1788619902.338109:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 394 00000100:00100000:0.0:1788619902.338121:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5770380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571269632:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619902.338143:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619902.338167:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071bcbab9/0x1cd7fe5368733699 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ad2e expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619902.338247:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b11180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589020288:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619902.341949:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfcbc840 going to sleep for 11 seconds 00000100:00100000:0.0:1788619902.347478:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b11180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589020288:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619902.347539:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000071bcbab9/0x1cd7fe5368733699 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ad2e expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619902.347601:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071bcbab9/0x1cd7fe5368733699 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ad2e expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619902.347617:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4ac00, rc 0) 00000100:00100000:0.0:1788619902.347748:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5770380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571269632:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 9627us (9899us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619902.347758:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 394 00000100:00100000:2.0:1788619902.358714:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619902.385382:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571269888 00000100:00100000:3.0:1788619902.385396:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571269888 00000100:00100000:3.0:1788619902.385482:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 395 00000100:00100000:3.0:1788619902.385496:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5027100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571269888:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619902.385521:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619902.385546:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007736e663/0x1cd7fe53687336a0 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ad35 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619902.385634:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589020416:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619902.385756:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48ccc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619902.390576:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589020416:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619902.390637:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007736e663/0x1cd7fe53687336a0 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ad35 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619902.390672:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007736e663/0x1cd7fe53687336a0 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ad35 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619902.390689:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d400, rc 0) 00000100:00100000:3.0:1788619902.390776:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5027100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571269888:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5283us (5619us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619902.390788:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 395 00000100:00100000:2.0:1788619902.396315:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619902.398522:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571270016 00000100:00100000:3.0:1788619902.398545:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571270016 00000100:00100000:3.0:1788619902.398637:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 396 00000100:00100000:3.0:1788619902.398651:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed61880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571270016:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619902.398677:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619902.398703:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007736e663/0x1cd7fe53687336a7 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ad3c expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619902.398793:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589020544:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619902.402214:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48ccc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619902.415725:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589020544:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619902.415788:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007736e663/0x1cd7fe53687336a7 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ad3c expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619902.415822:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007736e663/0x1cd7fe53687336a7 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ad3c expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619902.415840:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d400, rc 0) 00000100:00100000:2.0:1788619902.415952:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed61880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571270016:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 17302us (19662us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619902.415962:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 396 00000100:00100000:0.0:1788619902.419213:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619902.419302:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571270144 00000100:00100000:3.0:1788619902.419448:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571270144 00002000:00100000:2.0:1788619902.419676:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b11180 x1875503571270144/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:753/0 lens 488/0 e 0 to 0 dl 1788619913 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619902.419713:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b11180 with x1875503571270144 00000100:00100000:2.0:1788619902.419759:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 272 00000100:00100000:2.0:1788619902.419769:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b11180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571270144:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619902.420224:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b11180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571270144:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 455us (1025us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619902.420233:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 272 00002000:00100000:2.0:1788619902.420240:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b11180 x1875503571270144/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:753/0 lens 488/440 e 0 to 0 dl 1788619913 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619902.420248:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b11180 with x1875503571270144 00000100:00100000:2.0:1788619902.434711:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619902.439706:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c8597480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589020672:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619902.439760:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008238155e/0x1cd7fe53687336ae lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ad43 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619902.439795:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008238155e/0x1cd7fe53687336ae lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ad43 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619902.439814:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b51c00, rc 0) 00000100:00100000:2.0:1788619902.439917:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed60a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571270272:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4317us (5219us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619902.439927:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 397 00000100:00100000:2.0:1788619902.445208:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619902.445426:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571270400 00000100:00100000:0.0:1788619902.445439:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571270400 00000100:00100000:0.0:1788619902.445526:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 398 00000100:00100000:0.0:1788619902.445538:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed62300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571270400:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619902.445562:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619902.445585:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5e4548d/0x1cd7fe53687336b5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ad4a expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619902.445665:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b13100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589020800:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619902.445774:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfcbc840 going to sleep for 11 seconds 00000100:00100000:0.0:1788619902.451213:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b13100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589020800:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619902.451274:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d5e4548d/0x1cd7fe53687336b5 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ad4a expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619902.451310:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5e4548d/0x1cd7fe53687336b5 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ad4a expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619902.451373:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4ae00, rc 0) 00000100:00100000:0.0:1788619902.451475:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed62300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571270400:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5938us (6280us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619902.451486:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 398 00000100:00100000:2.0:1788619902.456287:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571270528 00000100:00100000:2.0:1788619902.456361:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571270528 00002000:00100000:2.0:1788619902.456434:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c8594380 x1875503571270528/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:753/0 lens 488/0 e 0 to 0 dl 1788619913 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619902.456447:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c8594380 with x1875503571270528 00000100:00100000:2.0:1788619902.456497:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 273 00000100:00100000:2.0:1788619902.456509:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8594380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571270528:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619902.456864:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8594380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571270528:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 356us (689us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619902.456873:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 273 00002000:00100000:2.0:1788619902.456880:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c8594380 x1875503571270528/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:753/0 lens 488/440 e 0 to 0 dl 1788619913 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619902.456889:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c8594380 with x1875503571270528 00000100:00100000:2.0:1788619902.465292:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619902.465407:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571270656 00000100:00100000:3.0:1788619902.465424:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571270656 00000100:00100000:3.0:1788619902.465507:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 399 00000100:00100000:3.0:1788619902.465522:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010fc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571270656:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619902.465570:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619902.465595:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000485aa315/0x1cd7fe53687336bc lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ad51 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619902.465678:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589020928:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619902.465810:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c000 going to sleep for 11 seconds 00000100:00100000:2.0:1788619902.491060:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571270784 00000100:00100000:2.0:1788619902.491076:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571270784 00000100:00100000:2.0:1788619902.491212:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 400 00000100:00100000:2.0:1788619902.491225:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8597100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571270784:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619902.491250:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619902.491275:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008b900302/0x1cd7fe53687336c3 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ad58 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619902.491416:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589021056:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619902.491540:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619902.502071:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589021056:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619902.502142:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008b900302/0x1cd7fe53687336c3 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ad58 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619902.502177:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008b900302/0x1cd7fe53687336c3 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ad58 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619902.502196:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505c000, rc 0) 00000100:00100000:0.0:1788619902.502282:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8597100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571270784:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 11058us (11642us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619902.502292:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 400 00000100:00100000:3.0:1788619902.515471:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619902.540201:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571271040 00000100:00100000:2.0:1788619902.540214:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571271040 00000100:00100000:2.0:1788619902.540279:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 401 00000100:00100000:2.0:1788619902.540315:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8594000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571271040:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619902.540383:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619902.540408:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071354985/0x1cd7fe53687336ca lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ad5f expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619902.540493:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589021184:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619902.540615:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccf00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619902.553104:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589021184:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619902.553176:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000071354985/0x1cd7fe53687336ca lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ad5f expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619902.553211:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071354985/0x1cd7fe53687336ca lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ad5f expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619902.553228:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505c800, rc 0) 00000100:00100000:2.0:1788619902.553394:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8594000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571271040:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 13087us (13411us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619902.553404:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 401 00000100:00100000:0.0:1788619902.559496:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619902.559616:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571271168 00000100:00100000:2.0:1788619902.559629:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571271168 00000100:00100000:2.0:1788619902.559707:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 402 00000100:00100000:2.0:1788619902.559720:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b13100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571271168:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619902.559745:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619902.559768:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071354985/0x1cd7fe53687336d1 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ad66 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619902.559886:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589021312:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619902.559998:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccf00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619902.567216:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589021312:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619902.567279:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000071354985/0x1cd7fe53687336d1 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ad66 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619902.567323:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071354985/0x1cd7fe53687336d1 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ad66 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619902.567386:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505c800, rc 0) 00000100:00100000:2.0:1788619902.567456:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b13100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571271168:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7737us (7976us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619902.567466:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 402 00000100:00100000:3.0:1788619902.579101:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619902.579212:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571271296 00000100:00100000:2.0:1788619902.579226:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571271296 00002000:00100000:2.0:1788619902.579295:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be0e00 x1875503571271296/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:753/0 lens 488/0 e 0 to 0 dl 1788619913 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619902.579319:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be0e00 with x1875503571271296 00000100:00100000:2.0:1788619902.579402:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 275 00000100:00100000:2.0:1788619902.579414:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571271296:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619902.579786:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571271296:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 372us (702us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619902.579795:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 275 00002000:00100000:2.0:1788619902.579822:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be0e00 x1875503571271296/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:753/0 lens 488/440 e 0 to 0 dl 1788619913 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619902.579830:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be0e00 with x1875503571271296 00000100:00100000:0.0:1788619902.587485:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619902.590376:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c8595500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589021440:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619902.590427:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d0cadab6/0x1cd7fe53687336d8 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ad6d expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619902.590456:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d0cadab6/0x1cd7fe53687336d8 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ad6d expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619902.590471:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b51400, rc 0) 00000100:00100000:3.0:1788619902.590529:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b10380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571271424:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2800us (3061us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619902.590538:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 403 00000100:00100000:3.0:1788619902.598520:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619902.598633:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571271552 00000100:00100000:0.0:1788619902.598648:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571271552 00000100:00100000:0.0:1788619902.598724:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 404 00000100:00100000:0.0:1788619902.598736:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6531880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571271552:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619902.598763:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619902.598787:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5e4548d/0x1cd7fe53687336df lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ad74 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619902.598890:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4ba4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589021568:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619902.599023:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfcbc840 going to sleep for 11 seconds 00000100:00100000:2.0:1788619902.624496:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571271680 00000100:00100000:2.0:1788619902.624520:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571271680 00002000:00100000:2.0:1788619902.624606:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c8595880 x1875503571271680/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:753/0 lens 488/0 e 0 to 0 dl 1788619913 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619902.624620:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c8595880 with x1875503571271680 00000100:00100000:2.0:1788619902.624740:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 276 00000100:00100000:2.0:1788619902.624752:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8595880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571271680:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619902.625160:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8595880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571271680:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 408us (3139us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619902.625170:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 276 00002000:00100000:2.0:1788619902.625179:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c8595880 x1875503571271680/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:753/0 lens 488/440 e 0 to 0 dl 1788619913 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619902.625189:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c8595880 with x1875503571271680 00000100:00100000:3.0:1788619902.640509:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619902.654743:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619902.654880:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571271936 00000100:00100000:2.0:1788619902.654894:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571271936 00000100:00100000:2.0:1788619902.654978:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 406 00000100:00100000:2.0:1788619902.654991:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6533100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571271936:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619902.655015:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619902.655048:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071354985/0x1cd7fe53687336ed lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ad82 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619902.655131:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589021824:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619902.655265:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccf00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619902.669554:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589021824:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619902.669620:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000071354985/0x1cd7fe53687336ed lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ad82 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619902.669652:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071354985/0x1cd7fe53687336ed lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ad82 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619902.669669:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505c800, rc 0) 00000100:00100000:2.0:1788619902.671182:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6533100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571271936:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 16176us (16437us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619902.671198:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 406 00000100:00100000:3.0:1788619902.675486:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619902.675696:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571272064 00000100:00100000:0.0:1788619902.675710:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571272064 00002000:00100000:0.0:1788619902.675785:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6531500 x1875503571272064/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:753/0 lens 488/0 e 0 to 0 dl 1788619913 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619902.675799:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6531500 with x1875503571272064 00000100:00100000:0.0:1788619902.675890:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 277 00000100:00100000:0.0:1788619902.675912:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6531500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571272064:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619902.676484:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6531500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571272064:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 581us (1014us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619902.676495:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 277 00002000:00100000:0.0:1788619902.676504:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6531500 x1875503571272064/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:753/0 lens 488/440 e 0 to 0 dl 1788619913 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619902.676513:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6531500 with x1875503571272064 00000100:00100000:3.0:1788619902.680247:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619902.680396:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571272192 00000100:00100000:2.0:1788619902.680410:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571272192 00000100:00100000:2.0:1788619902.680480:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 407 00000100:00100000:2.0:1788619902.680492:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6532300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571272192:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619902.680519:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619902.680544:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007f93e9f3/0x1cd7fe53687336f4 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ad89 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619902.680625:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589021952:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619902.680736:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccf00 going to sleep for 11 seconds 00000100:00100000:3.0:1788619902.683679:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589021952:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619902.683764:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007f93e9f3/0x1cd7fe53687336f4 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ad89 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619902.683801:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f93e9f3/0x1cd7fe53687336f4 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ad89 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619902.683819:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505de00, rc 0) 00000100:00100000:3.0:1788619902.683914:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6532300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571272192:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3422us (3680us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619902.683927:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 407 00000100:00100000:3.0:1788619902.688413:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619902.699356:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619902.699593:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571272448 00000100:00100000:0.0:1788619902.699606:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571272448 00002000:00100000:0.0:1788619902.699671:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5019180 x1875503571272448/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:753/0 lens 488/0 e 0 to 0 dl 1788619913 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619902.699683:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5019180 with x1875503571272448 00000100:00100000:0.0:1788619902.699760:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 278 00000100:00100000:0.0:1788619902.699771:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5019180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571272448:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619902.700488:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5019180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571272448:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 714us (1168us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619902.700518:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 278 00002000:00100000:0.0:1788619902.700527:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5019180 x1875503571272448/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:753/0 lens 488/440 e 0 to 0 dl 1788619913 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619902.700536:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5019180 with x1875503571272448 00000100:00100000:2.0:1788619902.704765:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619902.704855:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571272576 00000100:00100000:3.0:1788619902.704877:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571272576 00000100:00100000:3.0:1788619902.704968:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 409 00000100:00100000:3.0:1788619902.704980:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571272576:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619902.705006:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619902.705044:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062c34eb6/0x1cd7fe5368733702 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ad97 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619902.705128:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589022208:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619902.705247:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c000 going to sleep for 11 seconds 00000100:00100000:3.0:1788619902.708551:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589022208:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619902.708622:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000062c34eb6/0x1cd7fe5368733702 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ad97 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619902.708718:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062c34eb6/0x1cd7fe5368733702 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ad97 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619902.708736:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c800, rc 0) 00000100:00100000:3.0:1788619902.708826:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571272576:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3847us (4077us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619902.708836:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 409 00000100:00100000:3.0:1788619902.714196:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619902.714384:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571272704 00000100:00100000:2.0:1788619902.714397:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571272704 00000100:00100000:2.0:1788619902.714492:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 410 00000100:00100000:2.0:1788619902.714504:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571272704:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619902.714529:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619902.714554:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000611bb050/0x1cd7fe5368733709 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ad9e expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619902.714654:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589022336:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619902.714757:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619902.718774:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589022336:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619902.718838:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000611bb050/0x1cd7fe5368733709 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ad9e expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619902.718884:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000611bb050/0x1cd7fe5368733709 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ad9e expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619902.718899:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505d600, rc 0) 00000100:00100000:0.0:1788619902.719014:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571272704:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4511us (4832us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619902.719025:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 410 00000100:00100000:0.0:1788619902.723702:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619902.728938:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619902.729054:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571272960 00000100:00100000:3.0:1788619902.729068:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571272960 00000100:00100000:3.0:1788619902.729147:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 411 00000100:00100000:3.0:1788619902.729160:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571272960:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619902.729184:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619902.729206:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001fc792ba/0x1cd7fe5368733710 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ada5 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619902.729291:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6001880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589022464:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619902.729450:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c000 going to sleep for 11 seconds 00000100:00100000:2.0:1788619902.732779:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6001880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589022464:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619902.732836:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001fc792ba/0x1cd7fe5368733710 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ada5 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619902.732883:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001fc792ba/0x1cd7fe5368733710 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ada5 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619902.732901:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d800, rc 0) 00000100:00100000:2.0:1788619902.732982:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571272960:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3823us (4058us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619902.732993:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 411 00000100:00100000:0.0:1788619902.737558:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619902.737841:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571273088 00000100:00100000:3.0:1788619902.737857:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571273088 00000100:00100000:3.0:1788619902.737941:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 412 00000100:00100000:3.0:1788619902.737953:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571273088:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619902.737979:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619902.738001:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062c34eb6/0x1cd7fe5368733717 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1adac expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619902.738114:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6002a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589022592:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619902.738230:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48ccc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619902.745915:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619902.746026:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571273216 00000100:00100000:0.0:1788619902.746050:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571273216 00002000:00100000:0.0:1788619902.746116:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e67f4700 x1875503571273216/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:753/0 lens 488/0 e 0 to 0 dl 1788619913 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619902.746129:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e67f4700 with x1875503571273216 00000100:00100000:0.0:1788619902.746174:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 280 00000100:00100000:0.0:1788619902.746186:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571273216:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619902.746615:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571273216:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 429us (712us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619902.746624:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 280 00002000:00100000:0.0:1788619902.746632:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e67f4700 x1875503571273216/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:753/0 lens 488/440 e 0 to 0 dl 1788619913 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619902.746642:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e67f4700 with x1875503571273216 00010000:00010000:3.0:1788619902.754849:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788619902.755266:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00010000:00010000:1.0:1788619902.759279:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002e280df3/0x1cd7fe536873371e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1adb3 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619902.759375:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b50400, rc 0) 00000100:00100000:1.0:1788619902.759452:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571273344:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3958us (4192us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619902.759463:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 413 00000100:00100000:3.0:1788619902.765172:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619902.765376:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571273472 00000100:00100000:1.0:1788619902.765390:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571273472 00002000:00100000:1.0:1788619902.765457:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6002680 x1875503571273472/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:753/0 lens 488/0 e 0 to 0 dl 1788619913 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619902.765470:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6002680 with x1875503571273472 00000100:00100000:1.0:1788619902.765503:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 281 00000100:00100000:1.0:1788619902.765514:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6002680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571273472:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:1.0:1788619902.770422:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 714697 -> 876544 00000004:00010000:1.0:1788619902.770437:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619901 -> 1788619902 00000004:00010000:1.0:1788619902.770440:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619901 -> 1788619901 00000004:00010000:1.0:1788619902.770444:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619901 -> 1788619902 00000004:00010000:1.0:1788619902.770447:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 902 -> 902 00000100:00100000:1.0:1788619902.770556:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6002680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2353:x1875503571273472:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 5034us (5391us total) trans 4294967543 rc 0/0 00000100:00100000:1.0:1788619902.770571:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 281 00002000:00100000:1.0:1788619902.770591:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6002680 x1875503571273472/t4294967543(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:753/0 lens 488/448 e 0 to 0 dl 1788619913 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619902.770603:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6002680 with x1875503571273472 00000100:00100000:3.0:1788619902.776025:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619902.776214:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571273600 00000100:00100000:2.0:1788619902.776228:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571273600 00000100:00100000:2.0:1788619902.776375:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 617 00000100:00100000:2.0:1788619902.776389:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6001180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2353:x1875503571273600:12345-192.168.201.5@tcp:16:fsx.0 00000100:00100000:3.0:1788619902.846458:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6001180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2353:x1875503571273600:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 70054us (70432us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619902.846478:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 617 00000001:00080000:0.0:1788619902.847366:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967543 is committed 00000100:00100000:0.0:1788619902.851170:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619902.851709:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571273728 00000100:00100000:3.0:1788619902.851724:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571273728 00000100:00100000:3.0:1788619902.851835:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 618 00000100:00100000:3.0:1788619902.851849:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2355:x1875503571273728:12345-192.168.201.5@tcp:101:fsx.0 00000100:00080000:1.0:1788619902.852022:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:3.0:1788619902.852178:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619902.852208:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe5368733725 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1adba expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619902.852260:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049200/0x1cd7fe53687335c0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ac55 expref: 9 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619902.852812:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6000000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589022848:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619902.853165:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c0c0 going to sleep for 11 seconds 00000004:00010000:3.0:1788619902.856676:0:5845:0:(mdt_lvb.c:201:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks: 902 -> 1158 00000100:00100000:3.0:1788619902.856728:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6000000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589022848:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619902.856763:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 876544 00010000:00010000:3.0:1788619902.856792:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe5368733725 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1adba expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619902.856845:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619902.856849:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619902.856932:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe5368733725 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1adba expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619902.857024:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683da00, rc 0) 00000100:00100000:3.0:1788619902.857186:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2355:x1875503571273728:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5339us (6030us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619902.857197:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 618 00000100:00100000:2.0:1788619902.862771:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619902.863086:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571273856 00000100:00100000:3.0:1788619902.863101:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571273856 00000100:00100000:3.0:1788619902.863227:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 619 00000100:00100000:3.0:1788619902.863241:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dc8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2353:x1875503571273856:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619902.863269:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619902.863305:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe536873372c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1adc1 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619902.863377:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049200/0x1cd7fe53687335c0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ac55 expref: 9 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619902.863432:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6000000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589022976:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619902.863550:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619902.867021:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6000000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589022976:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619902.867069:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 876544 00010000:00010000:2.0:1788619902.867093:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe536873372c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1adc1 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619902.867131:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619902.867134:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619902.867141:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe536873372c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1adc1 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619902.867158:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683ce00, rc 0) 00000100:00100000:2.0:1788619902.867238:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dc8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2353:x1875503571273856:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4000us (4482us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619902.867249:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 619 00000100:00100000:3.0:1788619902.871624:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619902.871823:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571274112 00000100:00100000:0.0:1788619902.871836:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571274112 00000100:00100000:0.0:1788619902.871934:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 620 00000100:00100000:0.0:1788619902.871946:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6000380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:14908:x1875503571274112:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:0.0:1788619902.872022:0:5846:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619902.872068:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d400/0x1cd7fe53687335ab lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ac40 expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619902.872094:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619902.872101:0:5846:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788619902.872840:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4a600/0x1cd7fe5368733733 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619902.872864:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4be00/0x1cd7fe5368733549 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1abe5 expref: 9 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619902.872875:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4a600/0x1cd7fe5368733733 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619902.872952:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4be00/0x1cd7fe5368733549 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1abe5 expref: 9 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619902.872974:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4be00/0x1cd7fe5368733549 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1abe5 expref: 9 pid: 11397 timeout: 495 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619902.872993:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b15c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589023104:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619902.873128:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfcbc840 going to sleep for 11 seconds 00000100:00100000:3.0:1788619902.878261:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619902.878610:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b15c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589023104:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619902.878650:0:5846:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4a600/0x1cd7fe5368733733 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619902.878846:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571274240 00000100:00100000:1.0:1788619902.878856:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571274240 00010000:00010000:1.0:1788619902.878926:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4be00/0x1cd7fe5368733549 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1abe5 expref: 10 pid: 11397 timeout: 495 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619902.878978:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 198 00000100:00100000:1.0:1788619902.878991:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6000e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571274240:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619902.879011:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619902.879019:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4be00/0x1cd7fe5368733549 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1abe5 expref: 10 pid: 11397 timeout: 495 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619902.879045:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4be00/0x1cd7fe5368733549 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1abe5 expref: 10 pid: 11397 timeout: 495 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619902.879075:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4be00/0x1cd7fe5368733549 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1abe5 expref: 10 pid: 11397 timeout: 495 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619902.879096:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619902.879101:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4a600/0x1cd7fe5368733733 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619902.879114:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4a600/0x1cd7fe5368733733 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619902.879125:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4a600/0x1cd7fe5368733733 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619902.879153:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619902.879235:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6000e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571274240:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 246us (991us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619902.879263:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 198 00010000:00010000:3.0:1788619902.879937:0:5846:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4a600/0x1cd7fe5368733733 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619902.879959:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4a600/0x1cd7fe5368733733 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788619902.880160:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967544 00000004:00010000:3.0:1788619902.880235:0:5846:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 876544 -> 876544 00000004:00010000:3.0:1788619902.880239:0:5846:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619902 -> 1788619902 00000004:00010000:3.0:1788619902.880243:0:5846:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619901 -> 1788619901 00000004:00010000:3.0:1788619902.880245:0:5846:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619902 -> 1788619902 00000004:00010000:3.0:1788619902.880248:0:5846:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1158 -> 1158 00000004:00010000:3.0:1788619902.880258:0:5846:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000002458907a reply state 0000000035f742b9 transno 4294967545 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4a600/0x1cd7fe5368733733 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619902.880270:0:5846:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788619902.880448:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6000380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571274112:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 8500us (8847us total) trans 4294967545 rc 0/0 00000100:00100000:3.0:1788619902.880457:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 620 00000020:00080000:3.0:1788619902.880463:0:5846:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:2.0:1788619902.884143:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4a600/0x1cd7fe5368733733 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619902.884167:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4a600/0x1cd7fe5368733733 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619902.884176:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4a600/0x1cd7fe5368733733 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619902.884183:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c4b4a600 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:2.0:1788619902.884190:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4a600/0x1cd7fe5368733733 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619902.884200:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4a600/0x1cd7fe5368733733 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619902.884244:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619902.884247:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788619902.884254:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4a600/0x1cd7fe5368733733 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619902.884262:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4a600/0x1cd7fe5368733733 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619902.889921:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619902.890089:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571274368 00000100:00100000:3.0:1788619902.890103:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571274368 00000100:00100000:3.0:1788619902.890188:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 621 00000100:00100000:3.0:1788619902.890200:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dcbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571274368:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619902.890233:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619902.890262:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe536873373a lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1adc8 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619902.890306:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049200/0x1cd7fe53687335c0 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ac55 expref: 8 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619902.890380:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe536873373a lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1adc8 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619902.890430:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049200/0x1cd7fe53687335c0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ac55 expref: 8 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619902.890447:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049200/0x1cd7fe53687335c0 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ac55 expref: 8 pid: 5845 timeout: 495 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619902.890465:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6000000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589023232:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619902.890581:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c780 going to sleep for 11 seconds 00000100:00100000:3.0:1788619902.896885:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619902.897076:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571274496 00000100:00100000:1.0:1788619902.897090:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571274496 00010000:00010000:1.0:1788619902.897145:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049200/0x1cd7fe53687335c0 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ac55 expref: 9 pid: 5845 timeout: 495 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619902.897198:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 199 00000100:00100000:1.0:1788619902.897210:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6002a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571274496:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619902.897229:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619902.897263:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049200/0x1cd7fe53687335c0 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ac55 expref: 9 pid: 5845 timeout: 495 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619902.897281:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049200/0x1cd7fe53687335c0 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ac55 expref: 9 pid: 5845 timeout: 495 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619902.897322:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619902.897378:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe536873373a lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1adc8 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619902.897392:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe536873373a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1adc8 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619902.897405:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe536873373a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1adc8 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619902.897458:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe536873373a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1adc8 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619902.897494:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c8595500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589023360:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619902.898220:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963180 going to sleep for 11 seconds 00010000:00010000:0.0:1788619902.898443:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5049200/0x1cd7fe53687335c0 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ac55 expref: 9 pid: 5845 timeout: 495 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619902.898467:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6000000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589023232:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619902.898503:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe536873373a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1adc8 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619902.898519:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619902.898523:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619902.898526:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683da00, rc 0) 00000100:00100000:0.0:1788619902.898603:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dcbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571274368:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8396us (8696us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619902.898614:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 621 00000100:00100000:1.0:1788619902.903207:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c8595500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589023360:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619902.903246:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619902.903397:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6002a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+8:2354:x1875503571274496:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 6188us (6526us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619902.903412:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 199 00000100:00100000:1.0:1788619902.908527:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619902.908844:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571274624 00000100:00100000:2.0:1788619902.908928:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571274624 00002000:00010000:2.0:1788619902.908992:0:5858:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:2.0:1788619902.909000:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c8595500 with x1875503571274624 00000100:00100000:2.0:1788619902.909114:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 282 00000100:00100000:2.0:1788619902.909128:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8595500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571274624:12345-192.168.201.5@tcp:10:fsx.0 00000004:00010000:2.0:1788619902.909848:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 876544 -> 291318 00000004:00010000:2.0:1788619902.909855:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619902 -> 1788619902 00000004:00010000:2.0:1788619902.909859:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619901 -> 1788619901 00000004:00010000:2.0:1788619902.909873:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619902 -> 1788619902 00000004:00010000:2.0:1788619902.909876:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1158 -> 1158 00000100:00100000:2.0:1788619902.909996:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8595500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571274624:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 868us (1481us total) trans 4294967546 rc 0/0 00000100:00100000:2.0:1788619902.910006:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 282 00002000:00010000:2.0:1788619902.910011:0:5858:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:2.0:1788619902.910014:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c8595500 with x1875503571274624 00000100:00100000:2.0:1788619902.912978:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619902.913141:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571274752 00000100:00100000:3.0:1788619902.913156:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571274752 00002000:00010000:3.0:1788619902.913230:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:3.0:1788619902.913236:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5dcb100 with x1875503571274752 ext(0->18446744073709551615) 00002000:00010000:3.0:1788619902.913252:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5dcb100. 00000100:00100000:3.0:1788619902.913303:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 222 00000100:00100000:3.0:1788619902.913315:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dcb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571274752:12345-192.168.201.5@tcp:10:fsx.0 00000100:00080000:1.0:1788619902.913752:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:3.0:1788619902.913879:0:6703:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1472871 -> 0 00002000:00010000:3.0:1788619902.913884:0:6703:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619901 -> 1788619902 00002000:00010000:3.0:1788619902.913887:0:6703:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619901 -> 1788619899 00002000:00010000:3.0:1788619902.913889:0:6703:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619901 -> 1788619902 00002000:00010000:3.0:1788619902.913891:0:6703:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 391 -> 391 00000100:00100000:3.0:1788619902.913993:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dcb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571274752:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 678us (1029us total) trans 4294967356 rc 0/0 00000100:00100000:3.0:1788619902.914023:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 222 00002000:00010000:3.0:1788619902.914039:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:3.0:1788619902.914043:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5dcb100 with x1875503571274752 ext(0->18446744073709551615) 00002000:00010000:3.0:1788619902.914049:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5dcb100. 00000100:00100000:3.0:1788619902.919788:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619902.920168:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571274880 00000100:00100000:2.0:1788619902.920204:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571274880 00000100:00100000:2.0:1788619902.920287:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 622 00000100:00100000:2.0:1788619902.920413:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6001880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:14908:x1875503571274880:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619902.920446:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619902.920476:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d000/0x1cd7fe5368733741 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1adcf expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619902.920671:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c200/0x1cd7fe5368733748 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619902.920691:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c200/0x1cd7fe5368733748 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619902.920705:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c200/0x1cd7fe5368733748 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788619902.920742:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 291318 00000004:00200000:2.0:1788619902.920748:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788619902.920771:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c200/0x1cd7fe5368733748 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619902.920804:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d000/0x1cd7fe5368733741 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1adcf expref: 17 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619902.920825:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c200/0x1cd7fe5368733748 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1adcf expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619902.920835:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619902.920838:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788619902.920842:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505c200, rc 0) 00000100:00100000:2.0:1788619902.921303:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6001880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:14908:x1875503571274880:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 994us (1520us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619902.921314:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 622 00000100:00100000:1.0:1788619902.926733:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619902.926896:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571275008 00000100:00100000:3.0:1788619902.926911:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571275008 00000100:00100000:3.0:1788619902.926989:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 623 00000100:00100000:3.0:1788619902.927002:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8597480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+8:14908:x1875503571275008:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619902.927061:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619902.927092:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe536873374f lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1add6 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619902.927214:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368733756 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619902.927233:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368733756 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619902.927248:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368733756 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788619902.927285:0:5846:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 291318 00000004:00200000:3.0:1788619902.927291:0:5846:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788619902.927439:0:5846:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368733756 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619902.927477:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe536873374f lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1add6 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619902.927526:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368733756 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1add6 expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619902.927537:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619902.927540:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788619902.927545:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683ca00, rc 0) 00000100:00100000:3.0:1788619902.927878:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8597480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571275008:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 876us (1157us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619902.927887:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 623 00000100:00100000:0.0:1788619902.932056:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619902.932112:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619902.932244:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571275264 00000100:00100000:2.0:1788619902.932258:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571275264 00000100:00100000:2.0:1788619902.932372:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 414 00000100:00100000:2.0:1788619902.932387:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b15c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571275264:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619902.932415:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619902.932441:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff0b0db7/0x1cd7fe536873375d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ade4 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619902.932524:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5697100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589023488:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619902.932652:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc3c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619902.935820:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571275136 00000100:00100000:3.0:1788619902.935842:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571275136 00002000:00010000:2.0:1788619902.936142:0:6698:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619899 -> 1788619901 00000100:00100000:2.0:1788619902.936179:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5697100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589023488:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619902.936230:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ff0b0db7/0x1cd7fe536873375d lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ade4 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619902.936261:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff0b0db7/0x1cd7fe536873375d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ade4 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619902.936278:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505dc00, rc 0) 00000100:00100000:2.0:1788619902.936532:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b15c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571275264:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4145us (4504us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619902.936543:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 414 00000100:00100000:3.0:1788619902.936785:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 624 00000100:00100000:3.0:1788619902.936797:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b15f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571275136:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619902.936819:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619902.936844:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368733764 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1addd expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619902.936907:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe536873373a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1adc8 expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619902.936962:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6000000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589023616:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619902.937053:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c240 going to sleep for 11 seconds 00000100:00100000:3.0:1788619902.941762:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6000000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589023616:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619902.941804:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 291318 00010000:00010000:3.0:1788619902.941831:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368733764 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1addd expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619902.942040:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619902.942045:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619902.942052:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368733764 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1addd expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619902.942072:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d600, rc 0) 00000100:00100000:3.0:1788619902.942200:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b15f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571275136:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5405us (10085us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619902.942211:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 624 00000100:00100000:0.0:1788619902.947228:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619902.947362:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619902.947519:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571275392 00000100:00100000:3.0:1788619902.947535:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571275392 00000100:00100000:3.0:1788619902.947700:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 625 00000100:00100000:3.0:1788619902.947714:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b17480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571275392:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619902.947742:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619902.947770:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe536873376b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1adeb expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619902.947818:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe536873373a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1adc8 expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619902.947890:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6000000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589023744:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619902.948009:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c240 going to sleep for 11 seconds 00000100:00100000:3.0:1788619902.952732:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6000000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589023744:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619902.952771:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 291318 00010000:00010000:3.0:1788619902.952799:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe536873376b lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1adeb expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619902.953096:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571275520 00000100:00100000:2.0:1788619902.953107:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571275520 00000100:00100000:2.0:1788619902.953238:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 415 00000100:00100000:2.0:1788619902.953250:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b14000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571275520:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619902.953270:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619902.953290:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff0b0db7/0x1cd7fe5368733772 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1adf2 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619902.953402:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5697100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589023872:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619902.953505:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc3c0 going to sleep for 11 seconds 00010000:00010000:3.0:1788619902.953584:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619902.953588:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619902.953608:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe536873376b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1adeb expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619902.953627:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c600, rc 0) 00000100:00100000:3.0:1788619902.953668:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b17480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571275392:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5958us (6306us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619902.953676:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 625 00000100:00100000:2.0:1788619902.958091:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5697100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589023872:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619902.958146:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ff0b0db7/0x1cd7fe5368733772 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1adf2 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619902.958178:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff0b0db7/0x1cd7fe5368733772 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1adf2 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619902.958194:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505dc00, rc 0) 00000100:00100000:2.0:1788619902.958266:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b14000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571275520:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5017us (11054us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619902.958277:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 415 00000100:00100000:0.0:1788619902.965695:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619902.965833:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571275776 00000100:00100000:1.0:1788619902.965847:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571275776 00000100:00100000:1.0:1788619902.966244:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 626 00000100:00100000:1.0:1788619902.966258:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b14700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:14908:x1875503571275776:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:1.0:1788619902.966384:0:9417:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619902.966419:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c200/0x1cd7fe5368733748 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1adcf expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619902.966445:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619902.966452:0:9417:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788619902.966756:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50400/0x1cd7fe5368733779 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619902.966775:0:9417:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368733756 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1add6 expref: 8 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619902.966809:0:9417:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50400/0x1cd7fe5368733779 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619902.966855:0:9417:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368733756 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1add6 expref: 8 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619902.966874:0:9417:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368733756 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1add6 expref: 8 pid: 5846 timeout: 495 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619902.966891:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78cd3c8700 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589024000:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619902.967479:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963180 going to sleep for 11 seconds 00000100:00100000:1.0:1788619902.971553:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78cd3c8700 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589024000:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619902.971611:0:9417:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50400/0x1cd7fe5368733779 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619902.972416:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619902.973086:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571275904 00000100:00100000:1.0:1788619902.973098:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571275904 00010000:00010000:1.0:1788619902.973147:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368733756 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1add6 expref: 9 pid: 5846 timeout: 495 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619902.973196:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 200 00000100:00100000:1.0:1788619902.973207:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d01180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571275904:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619902.973226:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619902.973234:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368733756 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1add6 expref: 9 pid: 5846 timeout: 495 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619902.973273:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368733756 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1add6 expref: 9 pid: 5846 timeout: 495 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619902.973312:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368733756 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1add6 expref: 9 pid: 5846 timeout: 495 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619902.973356:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619902.973363:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50400/0x1cd7fe5368733779 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619902.973374:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50400/0x1cd7fe5368733779 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619902.973383:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50400/0x1cd7fe5368733779 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619902.973416:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619902.973477:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d01180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+8:2354:x1875503571275904:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 271us (1066us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619902.973485:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 200 00010000:00010000:3.0:1788619902.973613:0:9417:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50400/0x1cd7fe5368733779 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619902.973622:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50400/0x1cd7fe5368733779 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788619902.973767:0:9417:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967547 00000004:00010000:3.0:1788619902.973839:0:9417:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 291318 -> 291318 00000004:00010000:3.0:1788619902.973843:0:9417:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619902 -> 1788619902 00000004:00010000:3.0:1788619902.973846:0:9417:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619901 -> 1788619901 00000004:00010000:3.0:1788619902.973849:0:9417:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619902 -> 1788619902 00000004:00010000:3.0:1788619902.973851:0:9417:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1158 -> 1158 00000004:00010000:3.0:1788619902.973859:0:9417:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000094be3ec reply state 0000000020f629a0 transno 4294967548 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50400/0x1cd7fe5368733779 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619902.973868:0:9417:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788619902.974186:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b14700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+18:14908:x1875503571275776:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 7929us (8506us total) trans 4294967548 rc 0/0 00000100:00100000:3.0:1788619902.974207:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 626 00000020:00080000:3.0:1788619902.974211:0:9417:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1788619902.982382:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619902.983077:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571276032 00000100:00100000:3.0:1788619902.983093:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571276032 00002000:00010000:3.0:1788619902.983157:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1347905->18446744073709551615) 00002000:00100000:3.0:1788619902.983164:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6d02300 with x1875503571276032 ext(1347905->18446744073709551615) 00002000:00010000:3.0:1788619902.983180:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6d02300. 00000100:00100000:3.0:1788619902.983224:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 223 00000100:00100000:3.0:1788619902.983236:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d02300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571276032:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:3.0:1788619902.983786:0:6702:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 0 -> 1347905 00002000:00010000:3.0:1788619902.983793:0:6702:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619902 -> 1788619902 00002000:00010000:3.0:1788619902.983796:0:6702:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619901 -> 1788619899 00002000:00010000:3.0:1788619902.983799:0:6702:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619902 -> 1788619902 00002000:00010000:3.0:1788619902.983801:0:6702:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 391 -> 391 00000100:00100000:3.0:1788619902.983905:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d02300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2353:x1875503571276032:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 668us (1538us total) trans 4294967357 rc 0/0 00000100:00100000:3.0:1788619902.983916:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 223 00002000:00010000:3.0:1788619902.983921:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1347905->18446744073709551615) 00002000:00100000:3.0:1788619902.983925:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6d02300 with x1875503571276032 ext(1347905->18446744073709551615) 00002000:00010000:3.0:1788619902.983931:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6d02300. 00010000:00010000:2.0:1788619902.984081:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50400/0x1cd7fe5368733779 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619902.984095:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50400/0x1cd7fe5368733779 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619902.984133:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50400/0x1cd7fe5368733779 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619902.984140:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c4b50400 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:2.0:1788619902.984147:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50400/0x1cd7fe5368733779 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619902.984158:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50400/0x1cd7fe5368733779 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619902.984180:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619902.984184:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788619902.984190:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50400/0x1cd7fe5368733779 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619902.984197:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50400/0x1cd7fe5368733779 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619902.990574:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619902.990754:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571276160 00000100:00100000:2.0:1788619902.990768:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571276160 00002000:00100000:2.0:1788619902.990838:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1282048->1286143) req@ffff8e78c74f7100 x1875503571276160/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:753/0 lens 488/0 e 0 to 0 dl 1788619913 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619902.990917:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c74f7100 with x1875503571276160 ext(1282048->1286143) 00002000:00010000:2.0:1788619902.990924:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c74f7100 00000100:00100000:2.0:1788619902.990994:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 224 00000100:00100000:2.0:1788619902.991006:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2353:x1875503571276160:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619902.991400:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2353:x1875503571276160:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 394us (839us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619902.991418:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 224 00002000:00100000:2.0:1788619902.991427:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1282048->1286143) req@ffff8e78c74f7100 x1875503571276160/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:753/0 lens 488/4536 e 0 to 0 dl 1788619913 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619902.991436:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c74f7100 with x1875503571276160 ext(1282048->1286143) 00002000:00010000:2.0:1788619902.991440:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c74f7100 00000100:00100000:0.0:1788619902.996577:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619902.996757:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571276288 00000100:00100000:3.0:1788619902.996772:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571276288 00002000:00100000:3.0:1788619902.996836:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1286144->1290239) req@ffff8e78c74f5f80 x1875503571276288/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:753/0 lens 488/0 e 0 to 0 dl 1788619913 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619902.996848:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c74f5f80 with x1875503571276288 ext(1286144->1290239) 00002000:00010000:3.0:1788619902.996883:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c74f5f80 00000100:00100000:3.0:1788619902.996926:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 225 00000100:00100000:3.0:1788619902.996937:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2352:x1875503571276288:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619902.997403:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2352:x1875503571276288:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 465us (840us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619902.997412:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 225 00002000:00100000:3.0:1788619902.997420:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1286144->1290239) req@ffff8e78c74f5f80 x1875503571276288/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:753/0 lens 488/4536 e 0 to 0 dl 1788619913 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619902.997429:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c74f5f80 with x1875503571276288 ext(1286144->1290239) 00002000:00010000:3.0:1788619902.997434:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c74f5f80 00000100:00100000:0.0:1788619903.002648:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619903.002835:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571276416 00000100:00100000:2.0:1788619903.002850:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571276416 00002000:00100000:2.0:1788619903.002924:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1290240->1294335) req@ffff8e77c50c3480 x1875503571276416/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/0 e 0 to 0 dl 1788619914 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619903.002938:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c50c3480 with x1875503571276416 ext(1290240->1294335) 00002000:00010000:2.0:1788619903.002946:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c50c3480 00000100:00100000:2.0:1788619903.002987:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 226 00000100:00100000:2.0:1788619903.002998:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2353:x1875503571276416:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619903.003408:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2353:x1875503571276416:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 409us (773us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619903.003417:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 226 00002000:00100000:2.0:1788619903.003425:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1290240->1294335) req@ffff8e77c50c3480 x1875503571276416/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/4536 e 0 to 0 dl 1788619914 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619903.003435:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c50c3480 with x1875503571276416 ext(1290240->1294335) 00002000:00010000:2.0:1788619903.003439:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c50c3480 00000100:00100000:1.0:1788619903.008488:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619903.009023:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571276544 00000100:00100000:2.0:1788619903.009050:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571276544 00002000:00100000:2.0:1788619903.009126:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1294336->1298431) req@ffff8e78cd3c8700 x1875503571276544/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/0 e 0 to 0 dl 1788619914 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619903.009164:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78cd3c8700 with x1875503571276544 ext(1294336->1298431) 00002000:00010000:2.0:1788619903.009171:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78cd3c8700 00000100:00100000:2.0:1788619903.009219:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 227 00000100:00100000:2.0:1788619903.009231:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cd3c8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2352:x1875503571276544:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619903.009826:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cd3c8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2352:x1875503571276544:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 592us (1346us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619903.009838:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 227 00002000:00100000:2.0:1788619903.009846:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1294336->1298431) req@ffff8e78cd3c8700 x1875503571276544/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/4536 e 0 to 0 dl 1788619914 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619903.009867:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78cd3c8700 with x1875503571276544 ext(1294336->1298431) 00002000:00010000:2.0:1788619903.009872:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78cd3c8700 00000100:00100000:0.0:1788619903.017840:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619903.018310:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571276672 00000100:00100000:3.0:1788619903.018367:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571276672 00002000:00100000:3.0:1788619903.018439:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1298432->1302527) req@ffff8e78c7430000 x1875503571276672/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/0 e 0 to 0 dl 1788619914 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619903.018455:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7430000 with x1875503571276672 ext(1298432->1302527) 00002000:00010000:3.0:1788619903.018463:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c7430000 00000100:00100000:3.0:1788619903.018509:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 228 00000100:00100000:3.0:1788619903.018523:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7430000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2353:x1875503571276672:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619903.018874:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7430000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2353:x1875503571276672:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 352us (1051us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619903.018883:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 228 00002000:00100000:3.0:1788619903.018890:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1298432->1302527) req@ffff8e78c7430000 x1875503571276672/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/4536 e 0 to 0 dl 1788619914 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619903.018899:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7430000 with x1875503571276672 ext(1298432->1302527) 00002000:00010000:3.0:1788619903.018920:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c7430000 00000100:00100000:0.0:1788619903.027108:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619903.027511:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571276800 00000100:00100000:2.0:1788619903.027526:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571276800 00002000:00100000:2.0:1788619903.027597:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1302528->1306623) req@ffff8e78fd938e00 x1875503571276800/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/0 e 0 to 0 dl 1788619914 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619903.027610:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fd938e00 with x1875503571276800 ext(1302528->1306623) 00002000:00010000:2.0:1788619903.027618:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fd938e00 00000100:00100000:2.0:1788619903.027719:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 229 00000100:00100000:2.0:1788619903.027731:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd938e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2352:x1875503571276800:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619903.028110:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd938e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2352:x1875503571276800:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 379us (1013us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619903.028118:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 229 00002000:00100000:2.0:1788619903.028146:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1302528->1306623) req@ffff8e78fd938e00 x1875503571276800/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/4536 e 0 to 0 dl 1788619914 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619903.028155:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fd938e00 with x1875503571276800 ext(1302528->1306623) 00002000:00010000:2.0:1788619903.028160:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fd938e00 00000100:00100000:1.0:1788619903.036929:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619903.037049:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571276928 00000100:00100000:3.0:1788619903.037063:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571276928 00002000:00100000:3.0:1788619903.037132:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1306624->1310719) req@ffff8e78cd3c8e00 x1875503571276928/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/0 e 0 to 0 dl 1788619914 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619903.037146:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78cd3c8e00 with x1875503571276928 ext(1306624->1310719) 00002000:00010000:3.0:1788619903.037153:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78cd3c8e00 00000100:00100000:3.0:1788619903.037199:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 230 00000100:00100000:3.0:1788619903.037211:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cd3c8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571276928:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619903.037656:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cd3c8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571276928:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 444us (740us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619903.037666:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 230 00002000:00100000:3.0:1788619903.037675:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1306624->1310719) req@ffff8e78cd3c8e00 x1875503571276928/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/4536 e 0 to 0 dl 1788619914 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619903.037685:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78cd3c8e00 with x1875503571276928 ext(1306624->1310719) 00002000:00010000:3.0:1788619903.037689:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78cd3c8e00 00000100:00100000:0.0:1788619903.042952:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619903.043214:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571277056 00000100:00100000:2.0:1788619903.043229:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571277056 00002000:00100000:2.0:1788619903.043299:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1310720->1314815) req@ffff8e77c5ff6d80 x1875503571277056/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/0 e 0 to 0 dl 1788619914 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619903.043311:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5ff6d80 with x1875503571277056 ext(1310720->1314815) 00002000:00010000:2.0:1788619903.043319:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5ff6d80 00000100:00100000:2.0:1788619903.043403:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 231 00000100:00100000:2.0:1788619903.043415:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571277056:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619903.044691:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571277056:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 1264us (1742us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619903.044708:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 231 00002000:00100000:2.0:1788619903.044721:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1310720->1314815) req@ffff8e77c5ff6d80 x1875503571277056/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/4536 e 0 to 0 dl 1788619914 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619903.044732:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5ff6d80 with x1875503571277056 ext(1310720->1314815) 00002000:00010000:2.0:1788619903.044738:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5ff6d80 00000100:00100000:0.0:1788619903.050756:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619903.050952:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571277184 00000100:00100000:1.0:1788619903.050965:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571277184 00002000:00100000:1.0:1788619903.051048:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1314816->1318911) req@ffff8e77c4be5f80 x1875503571277184/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/0 e 0 to 0 dl 1788619914 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619903.051062:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4be5f80 with x1875503571277184 ext(1314816->1318911) 00002000:00010000:1.0:1788619903.051092:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4be5f80 00000100:00100000:1.0:1788619903.051139:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 232 00000100:00100000:1.0:1788619903.051153:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2355:x1875503571277184:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619903.052037:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2355:x1875503571277184:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 872us (1281us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619903.052049:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 232 00002000:00100000:1.0:1788619903.052059:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1314816->1318911) req@ffff8e77c4be5f80 x1875503571277184/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/4536 e 0 to 0 dl 1788619914 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619903.052068:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4be5f80 with x1875503571277184 ext(1314816->1318911) 00002000:00010000:1.0:1788619903.052073:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4be5f80 00000100:00100000:1.0:1788619903.059555:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619903.059701:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571277312 00000100:00100000:3.0:1788619903.059716:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571277312 00002000:00100000:3.0:1788619903.059783:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1318912->1323007) req@ffff8e78cd3c9180 x1875503571277312/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/0 e 0 to 0 dl 1788619914 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619903.059796:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78cd3c9180 with x1875503571277312 ext(1318912->1323007) 00002000:00010000:3.0:1788619903.059803:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78cd3c9180 00000100:00100000:3.0:1788619903.059844:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 233 00000100:00100000:3.0:1788619903.059857:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cd3c9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571277312:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619903.060651:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cd3c9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571277312:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 793us (1110us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619903.060663:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 233 00002000:00100000:3.0:1788619903.060673:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1318912->1323007) req@ffff8e78cd3c9180 x1875503571277312/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/4536 e 0 to 0 dl 1788619914 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619903.060684:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78cd3c9180 with x1875503571277312 ext(1318912->1323007) 00002000:00010000:3.0:1788619903.060689:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78cd3c9180 00000100:00100000:0.0:1788619903.066536:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619903.066667:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571277440 00000100:00100000:1.0:1788619903.066682:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571277440 00002000:00100000:1.0:1788619903.066753:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1323008->1327103) req@ffff8e78fe061180 x1875503571277440/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/0 e 0 to 0 dl 1788619914 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619903.066766:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fe061180 with x1875503571277440 ext(1323008->1327103) 00002000:00010000:1.0:1788619903.066773:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fe061180 00000100:00100000:1.0:1788619903.066812:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 234 00000100:00100000:1.0:1788619903.066823:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe061180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571277440:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619903.067196:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe061180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571277440:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 374us (676us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619903.067205:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 234 00002000:00100000:1.0:1788619903.067213:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1323008->1327103) req@ffff8e78fe061180 x1875503571277440/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/4536 e 0 to 0 dl 1788619914 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619903.067247:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fe061180 with x1875503571277440 ext(1323008->1327103) 00002000:00010000:1.0:1788619903.067253:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fe061180 00000100:00100000:0.0:1788619903.073706:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619903.073976:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571277568 00000100:00100000:3.0:1788619903.073990:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571277568 00002000:00100000:3.0:1788619903.074068:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1327104->1331199) req@ffff8e77c6b35500 x1875503571277568/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/0 e 0 to 0 dl 1788619914 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619903.074081:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6b35500 with x1875503571277568 ext(1327104->1331199) 00002000:00010000:3.0:1788619903.074088:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6b35500 00000100:00100000:3.0:1788619903.074131:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 235 00000100:00100000:3.0:1788619903.074142:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b35500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2355:x1875503571277568:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619903.074779:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b35500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2355:x1875503571277568:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 633us (1085us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619903.074789:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 235 00002000:00100000:3.0:1788619903.074798:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1327104->1331199) req@ffff8e77c6b35500 x1875503571277568/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/4536 e 0 to 0 dl 1788619914 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619903.074808:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6b35500 with x1875503571277568 ext(1327104->1331199) 00002000:00010000:3.0:1788619903.074813:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6b35500 00000100:00100000:0.0:1788619903.081201:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619903.081507:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571277696 00000100:00100000:2.0:1788619903.081522:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571277696 00002000:00100000:2.0:1788619903.081595:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1331200->1335295) req@ffff8e77c6b36d80 x1875503571277696/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/0 e 0 to 0 dl 1788619914 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619903.081608:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6b36d80 with x1875503571277696 ext(1331200->1335295) 00002000:00010000:2.0:1788619903.081633:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6b36d80 00000100:00100000:2.0:1788619903.081676:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 236 00000100:00100000:2.0:1788619903.081689:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b36d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2353:x1875503571277696:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619903.082794:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b36d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2353:x1875503571277696:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 1096us (1608us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619903.082807:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 236 00002000:00100000:2.0:1788619903.082818:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1331200->1335295) req@ffff8e77c6b36d80 x1875503571277696/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/4536 e 0 to 0 dl 1788619914 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619903.082830:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6b36d80 with x1875503571277696 ext(1331200->1335295) 00002000:00010000:2.0:1788619903.082835:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6b36d80 00000100:00100000:0.0:1788619903.087954:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619903.088086:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571277824 00000100:00100000:1.0:1788619903.088101:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571277824 00002000:00100000:1.0:1788619903.088169:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1335296->1339391) req@ffff8e77c6a89f80 x1875503571277824/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/0 e 0 to 0 dl 1788619914 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619903.088192:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a89f80 with x1875503571277824 ext(1335296->1339391) 00002000:00010000:1.0:1788619903.088199:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a89f80 00000100:00100000:1.0:1788619903.088235:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 237 00000100:00100000:1.0:1788619903.088247:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a89f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2355:x1875503571277824:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619903.088895:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a89f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2355:x1875503571277824:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 646us (954us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619903.088905:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 237 00002000:00100000:1.0:1788619903.088913:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1335296->1339391) req@ffff8e77c6a89f80 x1875503571277824/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/4536 e 0 to 0 dl 1788619914 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619903.088923:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a89f80 with x1875503571277824 ext(1335296->1339391) 00002000:00010000:1.0:1788619903.088927:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a89f80 00000100:00100000:0.0:1788619903.095062:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619903.095262:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571277952 00000100:00100000:2.0:1788619903.095276:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571277952 00002000:00100000:2.0:1788619903.095570:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1339392->1343487) req@ffff8e77c6a8b800 x1875503571277952/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/0 e 0 to 0 dl 1788619914 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619903.095584:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a8b800 with x1875503571277952 ext(1339392->1343487) 00002000:00010000:2.0:1788619903.095592:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a8b800 00000100:00100000:2.0:1788619903.095635:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 238 00000100:00100000:2.0:1788619903.095647:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a8b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2352:x1875503571277952:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619903.096002:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a8b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2352:x1875503571277952:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 355us (953us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619903.096010:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 238 00002000:00100000:2.0:1788619903.096017:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1339392->1343487) req@ffff8e77c6a8b800 x1875503571277952/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/4536 e 0 to 0 dl 1788619914 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619903.096037:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a8b800 with x1875503571277952 ext(1339392->1343487) 00002000:00010000:2.0:1788619903.096061:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a8b800 00000100:00100000:0.0:1788619903.102510:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619903.102667:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571278080 00000100:00100000:3.0:1788619903.102698:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571278080 00002000:00100000:3.0:1788619903.102766:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1343488->1347583) req@ffff8e78dfa3b480 x1875503571278080/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/0 e 0 to 0 dl 1788619914 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619903.102779:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa3b480 with x1875503571278080 ext(1343488->1347583) 00002000:00010000:3.0:1788619903.102787:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa3b480 00000100:00100000:3.0:1788619903.102829:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 239 00000100:00100000:3.0:1788619903.102841:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa3b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2355:x1875503571278080:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619903.103412:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa3b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2355:x1875503571278080:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 569us (912us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619903.103421:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 239 00002000:00100000:3.0:1788619903.103430:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1343488->1347583) req@ffff8e78dfa3b480 x1875503571278080/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/4536 e 0 to 0 dl 1788619914 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619903.103439:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa3b480 with x1875503571278080 ext(1343488->1347583) 00002000:00010000:3.0:1788619903.103444:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa3b480 00000100:00100000:0.0:1788619903.111643:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619903.111808:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571278208 00000100:00100000:1.0:1788619903.111823:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571278208 00002000:00100000:1.0:1788619903.111898:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1347584->1351679) req@ffff8e78dfa3b100 x1875503571278208/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/0 e 0 to 0 dl 1788619914 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619903.111912:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa3b100 with x1875503571278208 ext(1347584->1351679) 00002000:00010000:1.0:1788619903.111920:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa3b100 00000100:00100000:1.0:1788619903.111986:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 240 00000100:00100000:1.0:1788619903.111999:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa3b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2352:x1875503571278208:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619903.112664:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa3b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2352:x1875503571278208:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 663us (1032us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619903.112675:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 240 00002000:00100000:1.0:1788619903.112685:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1347584->1351679) req@ffff8e78dfa3b100 x1875503571278208/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/4536 e 0 to 0 dl 1788619914 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619903.112695:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa3b100 with x1875503571278208 ext(1347584->1351679) 00002000:00010000:1.0:1788619903.112700:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa3b100 00000100:00100000:0.0:1788619903.117927:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619903.118453:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571278336 00000100:00100000:2.0:1788619903.118467:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571278336 00002000:00100000:2.0:1788619903.118533:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1282048->1347904) req@ffff8e78c75f7480 x1875503571278336/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/0 e 0 to 0 dl 1788619914 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619903.118547:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c75f7480 with x1875503571278336 ext(1282048->1347904) 00002000:00010000:2.0:1788619903.118553:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c75f7480 00000100:00100000:2.0:1788619903.118593:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 241 00000100:00100000:2.0:1788619903.118602:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2352:x1875503571278336:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:3.0:1788619903.131806:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+11:2352:x1875503571278336:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 13187us (13875us total) trans 4294967358 rc 0/0 00000100:00100000:3.0:1788619903.131827:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 241 00002000:00100000:3.0:1788619903.131845:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1282048->1347904) req@ffff8e78c75f7480 x1875503571278336/t4294967358(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/448 e 0 to 0 dl 1788619914 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619903.131858:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c75f7480 with x1875503571278336 ext(1282048->1347904) 00002000:00010000:3.0:1788619903.131866:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c75f7480 00000100:00100000:0.0:1788619903.136664:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619903.136822:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571278464 00000100:00100000:1.0:1788619903.136835:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571278464 00000100:00100000:1.0:1788619903.136922:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 416 00000100:00100000:1.0:1788619903.136935:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+11:2354:x1875503571278464:12345-192.168.201.5@tcp:16:fsx.0 00000001:00080000:2.0:1788619903.182059:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967355 is committed 00000001:00080000:2.0:1788619903.182231:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967358 is committed 00000100:00100000:1.0:1788619903.185155:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571278464:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 48203us (48485us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619903.185175:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 416 00000100:00100000:0.0:1788619903.190790:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619903.190962:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571278592 00000100:00100000:3.0:1788619903.190978:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571278592 00000100:00100000:3.0:1788619903.191108:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 627 00000100:00100000:3.0:1788619903.191123:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+17:14908:x1875503571278592:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619903.191476:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619903.191505:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368733780 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1adf9 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.191689:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368733787 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.191710:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368733787 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.191799:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368733787 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788619903.191839:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1347905 00000004:00200000:3.0:1788619903.191845:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788619903.191948:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368733787 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.191985:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368733780 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1adf9 expref: 19 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.192009:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368733787 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1adf9 expref: 18 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.192020:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619903.192033:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788619903.192039:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d600, rc 0) 00000100:00100000:3.0:1788619903.192129:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+18:14908:x1875503571278592:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 1010us (1417us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619903.192138:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 627 00000100:00100000:2.0:1788619903.206468:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619903.206702:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571278720 00000100:00100000:1.0:1788619903.206717:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571278720 00002000:00100000:1.0:1788619903.206793:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5697100 x1875503571278720/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/0 e 0 to 0 dl 1788619914 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619903.206806:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5697100 with x1875503571278720 00000100:00100000:0.0:1788619903.206952:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 283 00000100:00100000:0.0:1788619903.206968:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5697100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2354:x1875503571278720:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619903.211988:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619903.217668:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571278848 00000100:00100000:3.0:1788619903.217695:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571278848 00002000:00100000:3.0:1788619903.217777:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1282047) req@ffff8e78c5695880 x1875503571278848/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/0 e 0 to 0 dl 1788619914 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619903.217792:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5695880 with x1875503571278848 ext(1048576->1282047) 00002000:00010000:3.0:1788619903.217799:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5695880 00000100:00100000:3.0:1788619903.217861:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 242 00000100:00100000:3.0:1788619903.217874:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5695880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571278848:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619903.250082:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5695880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571278848:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 32194us (38565us total) trans 0 rc 233472/233472 00000100:00100000:3.0:1788619903.250099:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 242 00002000:00100000:3.0:1788619903.250117:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1282047) req@ffff8e78c5695880 x1875503571278848/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/440 e 0 to 0 dl 1788619914 ref 1 fl Complete:/600/0 rc 233472/233472 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619903.250129:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5695880 with x1875503571278848 ext(1048576->1282047) 00002000:00010000:3.0:1788619903.250137:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5695880 00000100:00100000:1.0:1788619903.252636:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5697100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2354:x1875503571278720:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 45656us (46166us total) trans 0 rc 294912/294912 00000100:00100000:1.0:1788619903.252655:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 283 00002000:00100000:1.0:1788619903.252671:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5697100 x1875503571278720/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/440 e 0 to 0 dl 1788619914 ref 1 fl Complete:/600/0 rc 294912/294912 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619903.252704:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5697100 with x1875503571278720 00000100:00100000:2.0:1788619903.258645:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619903.258755:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571278976 00000100:00100000:3.0:1788619903.258770:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571278976 00000100:00100000:3.0:1788619903.258943:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 628 00000100:00100000:3.0:1788619903.258957:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c33800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+8:14908:x1875503571278976:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619903.258985:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619903.259014:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe536873378e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ae00 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.259064:0:11397:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe536873373a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1adc8 expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.259099:0:11397:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe536873378e lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae00 expref: 9 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.259144:0:11397:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe536873373a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1adc8 expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.259161:0:11397:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe536873373a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1adc8 expref: 17 pid: 5845 timeout: 495 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619903.259179:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6000000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589024128:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619903.259410:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78ce7f3000 going to sleep for 11 seconds 00000100:00100000:3.0:1788619903.266163:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6000000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589024128:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619903.266218:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe536873378e lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae00 expref: 9 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.266249:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619903.266252:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619903.266258:0:11397:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe536873378e lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae00 expref: 9 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.266272:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c200, rc 0) 00000100:00100000:3.0:1788619903.266574:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c33800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571278976:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7618us (7942us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619903.266587:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 628 00000100:00100000:0.0:1788619903.269720:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619903.270266:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571279104 00000100:00100000:3.0:1788619903.270310:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571279104 00010000:00010000:3.0:1788619903.270422:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe536873373a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1adc8 expref: 18 pid: 5845 timeout: 495 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619903.270770:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 201 00000100:00100000:2.0:1788619903.270788:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c80e8850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2355:x1875503571279104:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619903.270807:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619903.270856:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe536873373a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1adc8 expref: 18 pid: 5845 timeout: 495 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619903.270872:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe536873373a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1adc8 expref: 18 pid: 5845 timeout: 495 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619903.270912:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe536873373a lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1adc8 expref: 18 pid: 5845 timeout: 495 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619903.270951:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619903.270960:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe536873378e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae00 expref: 8 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619903.270974:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe536873378e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae00 expref: 8 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619903.270986:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe536873378e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae00 expref: 8 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619903.271054:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe536873378e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae00 expref: 8 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619903.271072:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c33b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589024256:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619903.271196:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccb40 going to sleep for 11 seconds 00000100:00100000:1.0:1788619903.274733:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c33b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589024256:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619903.274775:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619903.274845:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c80e8850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2355:x1875503571279104:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 4063us (5140us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619903.274872:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 201 00000100:00100000:0.0:1788619903.278995:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619903.279144:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571279232 00000100:00100000:1.0:1788619903.279158:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571279232 00000100:00100000:1.0:1788619903.279284:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 417 00000100:00100000:1.0:1788619903.279307:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571279232:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619903.279357:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619903.279391:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000850190e5/0x1cd7fe5368733795 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ae07 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619903.279481:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78cd3caa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589024384:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619903.279597:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd9639c0 going to sleep for 11 seconds 00002000:00010000:1.0:1788619903.282085:0:7654:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime: 1788619902 -> 1788619903 00002000:00010000:1.0:1788619903.282096:0:7654:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619899 -> 1788619903 00002000:00010000:1.0:1788619903.282099:0:7654:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime: 1788619902 -> 1788619903 00000100:00100000:1.0:1788619903.282130:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78cd3caa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589024384:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619903.282177:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000850190e5/0x1cd7fe5368733795 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae07 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619903.282208:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000850190e5/0x1cd7fe5368733795 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae07 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619903.282244:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b51800, rc 0) 00000100:00100000:1.0:1788619903.282445:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571279232:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3033us (3483us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619903.282456:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 417 00000100:00100000:0.0:1788619903.287130:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619903.287655:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571279360 00000100:00100000:2.0:1788619903.287670:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571279360 00002000:00100000:2.0:1788619903.287747:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c501c380 x1875503571279360/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:754/0 lens 504/0 e 0 to 0 dl 1788619914 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619903.287760:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c501c380 with x1875503571279360 00000100:00100000:2.0:1788619903.287800:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 284 00000100:00100000:2.0:1788619903.287812:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571279360:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619903.297299:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571279360:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 9465us (10160us total) trans 0 rc 294912/294912 00000100:00100000:2.0:1788619903.297513:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 284 00002000:00100000:2.0:1788619903.297530:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c501c380 x1875503571279360/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:754/0 lens 504/440 e 0 to 0 dl 1788619914 ref 1 fl Complete:/600/0 rc 294912/294912 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619903.297542:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c501c380 with x1875503571279360 00000100:00100000:0.0:1788619903.302391:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619903.302549:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571279488 00000100:00100000:1.0:1788619903.302562:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571279488 00002000:00100000:1.0:1788619903.302632:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c501ea00 x1875503571279488/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:754/0 lens 488/0 e 0 to 0 dl 1788619914 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619903.302646:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c501ea00 with x1875503571279488 00000100:00100000:1.0:1788619903.302697:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 285 00000100:00100000:1.0:1788619903.302709:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571279488:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619903.303108:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571279488:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 399us (797us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619903.303117:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 285 00002000:00100000:1.0:1788619903.303125:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c501ea00 x1875503571279488/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:754/0 lens 488/440 e 0 to 0 dl 1788619914 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619903.303134:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c501ea00 with x1875503571279488 00000100:00100000:0.0:1788619903.306857:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619903.307158:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571279616 00000100:00100000:2.0:1788619903.307172:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571279616 00002000:00100000:2.0:1788619903.307246:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900d5d180 x1875503571279616/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:754/0 lens 488/0 e 0 to 0 dl 1788619914 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619903.307259:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900d5d180 with x1875503571279616 00000100:00100000:2.0:1788619903.307313:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 286 00000100:00100000:2.0:1788619903.307348:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571279616:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619903.307772:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571279616:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 447us (940us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619903.307782:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 286 00002000:00100000:2.0:1788619903.307790:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900d5d180 x1875503571279616/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:754/0 lens 488/440 e 0 to 0 dl 1788619914 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619903.307798:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900d5d180 with x1875503571279616 00000100:00100000:2.0:1788619903.311270:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619903.311671:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571279744 00000100:00100000:0.0:1788619903.311686:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571279744 00002000:00100000:0.0:1788619903.311759:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5c33b80 x1875503571279744/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:754/0 lens 488/0 e 0 to 0 dl 1788619914 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619903.311773:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5c33b80 with x1875503571279744 00000100:00100000:0.0:1788619903.311817:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 287 00000100:00100000:0.0:1788619903.311828:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c33b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571279744:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619903.312249:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c33b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571279744:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 421us (993us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619903.312258:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 287 00002000:00100000:0.0:1788619903.312267:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5c33b80 x1875503571279744/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:754/0 lens 488/440 e 0 to 0 dl 1788619914 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619903.312276:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5c33b80 with x1875503571279744 00000100:00100000:0.0:1788619903.316820:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619903.316934:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571279872 00000100:00100000:3.0:1788619903.316948:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571279872 00000100:00100000:3.0:1788619903.317042:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 629 00000100:00100000:3.0:1788619903.317055:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571279872:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619903.317084:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619903.317112:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe536873379c lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ae0e expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.317236:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687337a3 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.317256:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687337a3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.317271:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687337a3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788619903.317321:0:9417:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1347905 00000004:00200000:3.0:1788619903.317382:0:9417:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788619903.317412:0:9417:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687337a3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.317451:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe536873379c lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae0e expref: 11 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.317475:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687337a3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae0e expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.317512:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619903.317516:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788619903.317522:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c000, rc 0) 00000100:00100000:3.0:1788619903.317603:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:14908:x1875503571279872:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 551us (799us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619903.317611:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 629 00000100:00100000:1.0:1788619903.321761:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619903.321900:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571280000 00000100:00100000:2.0:1788619903.321914:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571280000 00000100:00100000:2.0:1788619903.321998:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 418 00000100:00100000:2.0:1788619903.322011:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cd3caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571280000:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619903.322049:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619903.322075:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003bf815d1/0x1cd7fe53687337aa lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ae15 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619903.322182:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589024512:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619903.322307:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc780 going to sleep for 11 seconds 00000100:00100000:0.0:1788619903.326249:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589024512:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619903.326662:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003bf815d1/0x1cd7fe53687337aa lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae15 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619903.326697:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003bf815d1/0x1cd7fe53687337aa lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae15 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619903.326714:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505c400, rc 0) 00000100:00100000:1.0:1788619903.327064:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cd3caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571280000:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5052us (5317us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619903.327076:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 418 00000100:00100000:0.0:1788619903.331273:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619903.331587:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571280128 00000100:00100000:2.0:1788619903.331600:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571280128 00000100:00100000:2.0:1788619903.331712:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 419 00000100:00100000:2.0:1788619903.331727:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c09f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571280128:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619903.331752:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619903.331776:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c461731d/0x1cd7fe53687337b1 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ae1c expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619903.331867:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589024640:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619903.332143:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619903.335461:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589024640:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619903.335523:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c461731d/0x1cd7fe53687337b1 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae1c expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619903.335558:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c461731d/0x1cd7fe53687337b1 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae1c expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619903.335575:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505cc00, rc 0) 00000100:00100000:2.0:1788619903.335653:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c09f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571280128:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3929us (4393us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619903.335664:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 419 00000100:00100000:0.0:1788619903.342060:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619903.342170:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571280256 00000100:00100000:3.0:1788619903.342184:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571280256 00000100:00100000:3.0:1788619903.342302:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 630 00000100:00100000:3.0:1788619903.342314:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2352:x1875503571280256:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619903.342360:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619903.342541:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe53687337b8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ae23 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.342595:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe53687337b8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae23 expref: 18 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788619903.342614:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 291318 00010000:00010000:3.0:1788619903.342624:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe53687337b8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae23 expref: 18 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.342642:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619903.342645:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619903.342651:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c600, rc 0) 00000100:00100000:3.0:1788619903.342734:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2352:x1875503571280256:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 422us (691us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619903.342762:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 630 00000100:00100000:0.0:1788619903.348250:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619903.348525:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571280384 00000100:00100000:1.0:1788619903.348538:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571280384 00000100:00100000:1.0:1788619903.348609:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 420 00000100:00100000:1.0:1788619903.348640:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571280384:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619903.348665:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619903.348688:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c6f8c6a2/0x1cd7fe53687337bf lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ae2a expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619903.348768:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78cd3ca300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589024768:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619903.348904:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd9639c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619903.353204:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78cd3ca300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589024768:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619903.353274:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c6f8c6a2/0x1cd7fe53687337bf lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae2a expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619903.353531:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c6f8c6a2/0x1cd7fe53687337bf lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae2a expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619903.353549:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b50800, rc 0) 00000100:00100000:1.0:1788619903.353621:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571280384:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4983us (5385us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619903.353631:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 420 00000100:00100000:0.0:1788619903.358279:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619903.358619:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571280512 00000100:00100000:1.0:1788619903.358634:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571280512 00000100:00100000:1.0:1788619903.358720:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 421 00000100:00100000:1.0:1788619903.358735:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571280512:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619903.358759:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619903.358781:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a7bead88/0x1cd7fe53687337c6 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ae31 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619903.358898:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78cd3ca300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589024896:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619903.359003:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd9639c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619903.363182:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78cd3ca300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589024896:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619903.363238:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a7bead88/0x1cd7fe53687337c6 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae31 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619903.363286:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a7bead88/0x1cd7fe53687337c6 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae31 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619903.363313:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b51000, rc 0) 00000100:00100000:1.0:1788619903.363531:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571280512:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4798us (5277us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619903.363542:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 421 00000100:00100000:0.0:1788619903.370007:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619903.370242:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571280640 00000100:00100000:2.0:1788619903.370255:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571280640 00000100:00100000:2.0:1788619903.370377:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 422 00000100:00100000:2.0:1788619903.370391:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571280640:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619903.370415:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619903.370439:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c461731d/0x1cd7fe53687337cd lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ae38 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619903.370523:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589025024:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619903.370632:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619903.374228:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589025024:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619903.374661:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c461731d/0x1cd7fe53687337cd lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae38 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619903.374698:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c461731d/0x1cd7fe53687337cd lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae38 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619903.374716:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505cc00, rc 0) 00000100:00100000:2.0:1788619903.374789:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571280640:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4401us (4800us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619903.374799:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 422 00000100:00100000:0.0:1788619903.380481:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619903.380648:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571280768 00000100:00100000:3.0:1788619903.380661:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571280768 00000100:00100000:3.0:1788619903.380744:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 423 00000100:00100000:3.0:1788619903.380758:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571280768:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619903.380783:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619903.380807:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f980cda4/0x1cd7fe53687337d4 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ae3f expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619903.380900:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6000000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589025152:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619903.381039:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78ce7f3000 going to sleep for 11 seconds 00000100:00100000:3.0:1788619903.384834:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6000000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589025152:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619903.384903:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f980cda4/0x1cd7fe53687337d4 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae3f expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.384935:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f980cda4/0x1cd7fe53687337d4 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae3f expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.384951:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d200, rc 0) 00000100:00100000:3.0:1788619903.385033:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571280768:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4269us (4558us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619903.385043:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 423 00000100:00100000:0.0:1788619903.394741:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619903.394844:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571280896 00000100:00100000:1.0:1788619903.394869:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571280896 00000100:00100000:1.0:1788619903.394942:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 424 00000100:00100000:1.0:1788619903.394954:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571280896:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619903.394977:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619903.394999:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a7bead88/0x1cd7fe53687337db lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ae46 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619903.395085:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78cd3ca300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589025280:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619903.395192:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd9639c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619903.399205:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78cd3ca300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589025280:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619903.399272:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a7bead88/0x1cd7fe53687337db lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae46 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619903.399316:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a7bead88/0x1cd7fe53687337db lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae46 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619903.399543:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b51000, rc 0) 00000100:00100000:1.0:1788619903.399616:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571280896:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4663us (4890us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619903.399627:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 424 00000100:00100000:0.0:1788619903.405552:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619903.406049:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571281152 00000100:00100000:2.0:1788619903.406064:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571281152 00000100:00100000:2.0:1788619903.406199:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 631 00000100:00100000:2.0:1788619903.406213:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+10:14908:x1875503571281152:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:2.0:1788619903.406361:0:11397:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619903.406520:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687337a3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae0e expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619903.406545:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619903.406551:0:11397:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788619903.406862:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505cc00/0x1cd7fe53687337e2 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619903.406883:0:11397:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368733787 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1adf9 expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619903.406894:0:11397:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505cc00/0x1cd7fe53687337e2 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619903.406953:0:11397:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368733787 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1adf9 expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619903.406971:0:11397:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368733787 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1adf9 expref: 17 pid: 5845 timeout: 496 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619903.406988:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c4380 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589025408:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619903.407117:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619903.412639:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619903.412741:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571281280 00000100:00100000:1.0:1788619903.412756:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571281280 00010000:00010000:1.0:1788619903.412813:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368733787 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1adf9 expref: 18 pid: 5845 timeout: 496 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619903.412874:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 202 00000100:00100000:1.0:1788619903.412887:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd4700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2355:x1875503571281280:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619903.412920:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619903.412930:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368733787 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1adf9 expref: 18 pid: 5845 timeout: 496 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619903.412947:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368733787 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1adf9 expref: 18 pid: 5845 timeout: 496 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619903.412977:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619903.413001:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505cc00/0x1cd7fe53687337e2 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619903.413013:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505cc00/0x1cd7fe53687337e2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x50290000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619903.413032:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505cc00/0x1cd7fe53687337e2 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x50290000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619903.413060:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619903.413412:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd4700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2355:x1875503571281280:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 526us (788us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619903.413421:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 202 00010000:00010000:2.0:1788619903.413837:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368733787 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1adf9 expref: 17 pid: 5845 timeout: 496 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619903.413869:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c4380 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589025408:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619903.413898:0:11397:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505cc00/0x1cd7fe53687337e2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619903.414085:0:11397:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505cc00/0x1cd7fe53687337e2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619903.414092:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505cc00/0x1cd7fe53687337e2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788619903.414540:0:11397:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967549 00000004:00010000:2.0:1788619903.414619:0:11397:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 291318 -> 291318 00000004:00010000:2.0:1788619903.414624:0:11397:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619902 -> 1788619903 00000004:00010000:2.0:1788619903.414627:0:11397:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619901 -> 1788619901 00000004:00010000:2.0:1788619903.414629:0:11397:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619902 -> 1788619903 00000004:00010000:2.0:1788619903.414632:0:11397:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1158 -> 1158 00000004:00010000:2.0:1788619903.414641:0:11397:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000007bf66142 reply state 00000000e688136a transno 4294967550 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505cc00/0x1cd7fe53687337e2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619903.414653:0:11397:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788619903.414782:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+12:14908:x1875503571281152:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 8570us (9243us total) trans 4294967550 rc 0/0 00000100:00100000:2.0:1788619903.414792:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 631 00000020:00080000:2.0:1788619903.414797:0:11397:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:2.0:1788619903.420155:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505cc00/0x1cd7fe53687337e2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619903.420176:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505cc00/0x1cd7fe53687337e2 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619903.420185:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505cc00/0x1cd7fe53687337e2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619903.420216:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c505cc00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:2.0:1788619903.420223:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505cc00/0x1cd7fe53687337e2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619903.420232:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505cc00/0x1cd7fe53687337e2 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619903.420458:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619903.420463:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788619903.420471:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505cc00/0x1cd7fe53687337e2 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619903.420477:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505cc00/0x1cd7fe53687337e2 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619903.421267:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619903.421757:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571281408 00000100:00100000:3.0:1788619903.421770:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571281408 00000100:00100000:3.0:1788619903.421908:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 632 00000100:00100000:3.0:1788619903.421933:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571281408:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619903.421965:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619903.421969:0:5846:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1788619903.422036:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe536873378e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae00 expref: 11 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.422059:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619903.422065:0:5846:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788619903.422083:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687337e9 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ae4d expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.422115:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe53687337b8 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae23 expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.422126:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687337e9 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae4d expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.422171:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe53687337b8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae23 expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.422188:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe53687337b8 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae23 expref: 16 pid: 5845 timeout: 496 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619903.422205:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6000000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589025536:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619903.423041:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78ce7f3000 going to sleep for 11 seconds 00000100:00100000:3.0:1788619903.427489:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619903.427794:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571281536 00000100:00100000:1.0:1788619903.427809:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571281536 00010000:00010000:1.0:1788619903.427878:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe53687337b8 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae23 expref: 17 pid: 5845 timeout: 496 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619903.427934:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 203 00000100:00100000:1.0:1788619903.427946:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6000e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2355:x1875503571281536:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619903.427964:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619903.427996:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe53687337b8 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae23 expref: 17 pid: 5845 timeout: 496 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619903.428012:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe53687337b8 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae23 expref: 17 pid: 5845 timeout: 496 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619903.428050:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619903.428057:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687337e9 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae4d expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619903.428087:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687337e9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae4d expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619903.428098:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687337e9 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae4d expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619903.428150:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687337e9 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae4d expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619903.428165:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78cd3ca300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589025664:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619903.428280:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd9639c0 going to sleep for 11 seconds 00010000:00010000:0.0:1788619903.428753:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe53687337b8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae23 expref: 17 pid: 5845 timeout: 496 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619903.428778:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6000000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589025536:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619903.428815:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687337e9 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae4d expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619903.428830:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619903.428833:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619903.428837:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d200, rc 0) 00000100:00100000:0.0:1788619903.428894:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571281408:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6965us (7644us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619903.428903:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 632 00000100:00100000:1.0:1788619903.435638:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619903.435752:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571281664 00000100:00100000:2.0:1788619903.435765:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571281664 00000100:00100000:2.0:1788619903.435858:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 425 00000100:00100000:2.0:1788619903.435871:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd939f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:14908:x1875503571281664:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619903.435898:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619903.435932:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c461731d/0x1cd7fe53687337f0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ae54 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619903.435975:0:7654:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000574a4e4c/0x1cd7fe5368733438 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab05 expref: 5 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619903.435986:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=4s v=(1 11) 00010000:00010000:2.0:1788619903.435991:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(1 1) 00010000:00010000:2.0:1788619903.436015:0:7654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c461731d/0x1cd7fe53687337f0 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae54 expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619903.436069:0:7654:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000574a4e4c/0x1cd7fe5368733438 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab05 expref: 5 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619903.436086:0:7654:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000574a4e4c/0x1cd7fe5368733438 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab05 expref: 5 pid: 6696 timeout: 496 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619903.436102:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589025792:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619903.436201:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc3c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619903.441952:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78cd3ca300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589025664:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619903.442006:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619903.442304:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6000e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571281536:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 14358us (14830us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619903.442317:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 203 00000100:00100000:0.0:1788619903.443033:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619903.444867:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571281792 00000100:00100000:3.0:1788619903.444888:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571281792 00010000:00010000:3.0:1788619903.444965:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000574a4e4c/0x1cd7fe5368733438 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab05 expref: 6 pid: 6696 timeout: 496 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619903.445030:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 204 00000100:00100000:3.0:1788619903.445042:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571281792:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619903.445063:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619903.445095:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000574a4e4c/0x1cd7fe5368733438 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab05 expref: 6 pid: 6696 timeout: 496 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.445112:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000574a4e4c/0x1cd7fe5368733438 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab05 expref: 6 pid: 6696 timeout: 496 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.445148:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(1 1) 00010000:00010000:3.0:1788619903.445153:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(1 1) 00010000:00010000:3.0:1788619903.445164:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000c461731d/0x1cd7fe53687337f0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae54 expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.445175:0:5829:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c461731d/0x1cd7fe53687337f0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae54 expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.445660:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000c461731d/0x1cd7fe53687337f0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae54 expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619903.445680:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6002a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1875503589025920:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619903.445794:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78ce7f3a80 going to sleep for 11 seconds 00010000:00010000:2.0:1788619903.448664:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000574a4e4c/0x1cd7fe5368733438 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ab05 expref: 6 pid: 6696 timeout: 496 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619903.448697:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589025792:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619903.448742:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c461731d/0x1cd7fe53687337f0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae54 expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619903.448763:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505cc00, rc 0) 00000100:00100000:2.0:1788619903.448857:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd939f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:14908:x1875503571281664:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 12988us (13234us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619903.448868:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 425 00000100:00100000:2.0:1788619903.450543:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6002a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1875503589025920:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619903.450593:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619903.450669:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571281792:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 5628us (7650us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619903.450681:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 204 00000100:00100000:2.0:1788619903.456088:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619903.456174:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571281920 00000100:00100000:3.0:1788619903.456191:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571281920 00002000:00010000:3.0:1788619903.456510:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1609382->18446744073709551615) 00002000:00100000:3.0:1788619903.456517:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c50c4380 with x1875503571281920 ext(1609382->18446744073709551615) 00002000:00010000:3.0:1788619903.456532:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c50c4380. 00000100:00100000:0.0:1788619903.456791:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 243 00000100:00100000:0.0:1788619903.456808:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571281920:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:0.0:1788619903.457239:0:6702:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1347905 -> 1609382 00002000:00010000:0.0:1788619903.457269:0:6702:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619903 -> 1788619903 00002000:00010000:0.0:1788619903.457272:0:6702:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619903 -> 1788619899 00002000:00010000:0.0:1788619903.457275:0:6702:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619903 -> 1788619903 00002000:00010000:0.0:1788619903.457278:0:6702:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 391 -> 263 00000100:00100000:0.0:1788619903.457503:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571281920:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 698us (1423us total) trans 4294967359 rc 0/0 00000100:00100000:0.0:1788619903.457513:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 243 00002000:00010000:0.0:1788619903.457519:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1609382->18446744073709551615) 00002000:00100000:0.0:1788619903.457523:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c50c4380 with x1875503571281920 ext(1609382->18446744073709551615) 00002000:00010000:0.0:1788619903.457530:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c50c4380. 00000100:00100000:1.0:1788619903.468006:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619903.468129:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571282048 00000100:00100000:0.0:1788619903.468142:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571282048 00002000:00100000:0.0:1788619903.468207:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1609727) req@ffff8e78fe060380 x1875503571282048/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:754/0 lens 488/0 e 0 to 0 dl 1788619914 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619903.468220:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fe060380 with x1875503571282048 ext(1048576->1609727) 00002000:00010000:0.0:1788619903.468227:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fe060380 00000100:00100000:0.0:1788619903.468265:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 244 00000100:00100000:0.0:1788619903.468277:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe060380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571282048:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619903.476814:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe060380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571282048:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 8519us (8804us total) trans 0 rc 561152/561152 00000100:00100000:0.0:1788619903.476833:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 244 00002000:00100000:0.0:1788619903.476852:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1609727) req@ffff8e78fe060380 x1875503571282048/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:754/0 lens 488/440 e 0 to 0 dl 1788619914 ref 1 fl Complete:/600/0 rc 561152/561152 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619903.476865:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fe060380 with x1875503571282048 ext(1048576->1609727) 00002000:00010000:0.0:1788619903.476875:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fe060380 00000100:00100000:1.0:1788619903.483878:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619903.483968:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619903.484090:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571282304 00000100:00100000:0.0:1788619903.484103:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571282304 00002000:00100000:0.0:1788619903.484174:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1585152->1609381) req@ffff8e78e647b800 x1875503571282304/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:754/0 lens 488/0 e 0 to 0 dl 1788619914 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619903.484188:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78e647b800 with x1875503571282304 ext(1585152->1609381) 00002000:00010000:0.0:1788619903.484195:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78e647b800 00000100:00100000:0.0:1788619903.484237:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 245 00000100:00100000:0.0:1788619903.484250:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e647b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571282304:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:2.0:1788619903.485247:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571282176 00000100:00100000:2.0:1788619903.485262:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571282176 00002000:00100000:2.0:1788619903.485363:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6d01180 x1875503571282176/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:754/0 lens 488/0 e 0 to 0 dl 1788619914 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619903.485385:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6d01180 with x1875503571282176 00000100:00100000:2.0:1788619903.485413:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 288 00000100:00100000:2.0:1788619903.485422:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d01180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571282176:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619903.490564:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e647b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571282304:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 6297us (6576us total) trans 4294967360 rc 0/0 00000100:00100000:0.0:1788619903.490586:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 245 00002000:00100000:0.0:1788619903.490605:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1585152->1609381) req@ffff8e78e647b800 x1875503571282304/t4294967360(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:754/0 lens 488/448 e 0 to 0 dl 1788619914 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619903.490618:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78e647b800 with x1875503571282304 ext(1585152->1609381) 00002000:00010000:0.0:1788619903.490626:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78e647b800 00000100:00100000:2.0:1788619903.494398:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d01180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571282176:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 8903us (10464us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619903.494422:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 288 00002000:00100000:2.0:1788619903.494442:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6d01180 x1875503571282176/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:754/0 lens 488/440 e 0 to 0 dl 1788619914 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619903.494456:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6d01180 with x1875503571282176 00000100:00100000:2.0:1788619903.499067:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619903.499256:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571282432 00000100:00100000:1.0:1788619903.499273:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571282432 00000100:00100000:1.0:1788619903.499418:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 426 00000100:00100000:1.0:1788619903.499431:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df387100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571282432:12345-192.168.201.5@tcp:16:fsx.0 00000001:00080000:0.0:1788619903.547570:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967360 is committed 00000100:00100000:2.0:1788619903.548467:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df387100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571282432:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 49026us (49407us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619903.548485:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 426 00000100:00100000:3.0:1788619903.552373:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619903.552656:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571282560 00000100:00100000:0.0:1788619903.552670:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571282560 00000100:00100000:0.0:1788619903.552746:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 633 00000100:00100000:0.0:1788619903.552759:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6002a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571282560:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619903.552789:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619903.552819:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4ae00/0x1cd7fe53687337f7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ae5b expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619903.552942:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4b600/0x1cd7fe53687337fe lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619903.552981:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4b600/0x1cd7fe53687337fe lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619903.552997:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4b600/0x1cd7fe53687337fe lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619903.553045:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1609382 00000004:00200000:0.0:1788619903.553051:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619903.553077:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4b600/0x1cd7fe53687337fe lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619903.553110:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4ae00/0x1cd7fe53687337f7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae5b expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619903.553132:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4b600/0x1cd7fe53687337fe lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae5b expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619903.553143:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619903.553146:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619903.553151:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4b600, rc 0) 00000100:00100000:0.0:1788619903.553248:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6002a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:14908:x1875503571282560:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 490us (941us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619903.553256:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 633 00000100:00100000:0.0:1788619903.559478:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619903.559602:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571282688 00000100:00100000:2.0:1788619903.559615:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571282688 00000100:00100000:2.0:1788619903.559743:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 289 00000100:00100000:2.0:1788619903.559757:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2353:x1875503571282688:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:2.0:1788619903.560157:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 291318 -> 291318 00000004:00010000:2.0:1788619903.560178:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619903 -> 1788619903 00000004:00010000:2.0:1788619903.560181:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619901 -> 1788619901 00000004:00010000:2.0:1788619903.560184:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619903 -> 1788619903 00000004:00010000:2.0:1788619903.560187:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1158 -> 1158 00000100:00100000:2.0:1788619903.560509:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+13:2353:x1875503571282688:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 753us (1042us total) trans 4294967551 rc 0/0 00000100:00100000:2.0:1788619903.560519:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 289 00000100:00100000:2.0:1788619903.566076:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619903.566406:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571282816 00000100:00100000:0.0:1788619903.566421:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571282816 00000100:00100000:0.0:1788619903.566502:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 634 00000100:00100000:0.0:1788619903.566516:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df384700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+16:14908:x1875503571282816:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619903.566569:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619903.566601:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4ae00/0x1cd7fe5368733805 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ae62 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619903.566643:0:11397:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687337e9 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae4d expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619903.566656:0:11397:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4ae00/0x1cd7fe5368733805 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae62 expref: 17 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619903.566701:0:11397:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687337e9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae4d expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619903.566718:0:11397:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687337e9 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae4d expref: 12 pid: 5846 timeout: 496 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619903.566736:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c4a4f480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589026048:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619903.566869:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa240 going to sleep for 11 seconds 00000100:00100000:0.0:1788619903.569530:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c4a4f480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589026048:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619903.569581:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4ae00/0x1cd7fe5368733805 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae62 expref: 17 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619903.569607:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619903.569611:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619903.569615:0:11397:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4ae00/0x1cd7fe5368733805 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae62 expref: 17 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619903.569647:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4ae00, rc 0) 00000100:00100000:0.0:1788619903.569777:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df384700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+17:14908:x1875503571282816:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3263us (3715us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619903.569788:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 634 00000100:00100000:2.0:1788619903.574028:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619903.574420:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571282944 00000100:00100000:3.0:1788619903.574435:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571282944 00010000:00010000:3.0:1788619903.574493:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687337e9 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae4d expref: 13 pid: 5846 timeout: 496 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619903.574548:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 205 00000100:00100000:3.0:1788619903.574561:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df384000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+13:2355:x1875503571282944:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619903.574582:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619903.574619:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687337e9 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae4d expref: 13 pid: 5846 timeout: 496 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.574636:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687337e9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae4d expref: 13 pid: 5846 timeout: 496 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.574693:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687337e9 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae4d expref: 13 pid: 5846 timeout: 496 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.574715:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619903.574723:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4ae00/0x1cd7fe5368733805 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae62 expref: 16 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.574737:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4ae00/0x1cd7fe5368733805 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae62 expref: 16 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.574749:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4ae00/0x1cd7fe5368733805 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae62 expref: 16 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.574809:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4ae00/0x1cd7fe5368733805 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae62 expref: 16 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619903.574827:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6000000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589026176:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619903.575582:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c306b3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619903.581714:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619903.582516:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6000000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589026176:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619903.582554:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619903.582645:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df384000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571282944:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 8085us (8641us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619903.582659:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 205 00000100:00100000:2.0:1788619903.584168:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571283072 00000100:00100000:2.0:1788619903.584188:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571283072 00000100:00100000:2.0:1788619903.584671:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 290 00000100:00100000:2.0:1788619903.584686:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e6478a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2355:x1875503571283072:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:2.0:1788619903.584974:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 291318 -> 291318 00000004:00010000:2.0:1788619903.584979:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619903 -> 1788619903 00000004:00010000:2.0:1788619903.584983:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619901 -> 1788619901 00000004:00010000:2.0:1788619903.584985:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619903 -> 1788619903 00000004:00010000:2.0:1788619903.584988:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1158 -> 1158 00000100:00100000:2.0:1788619903.585099:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e6478a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2355:x1875503571283072:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 416us (3400us total) trans 4294967552 rc 0/0 00000100:00100000:2.0:1788619903.585107:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 290 00000100:00100000:1.0:1788619903.589469:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619903.589928:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571283200 00000100:00100000:3.0:1788619903.589943:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571283200 00000100:00100000:3.0:1788619903.590056:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 635 00000100:00100000:3.0:1788619903.590070:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e647ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2353:x1875503571283200:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619903.590104:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619903.590131:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe536873380c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ae69 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.590181:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4ae00/0x1cd7fe5368733805 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae62 expref: 17 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619903.590248:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6000000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589026304:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619903.590511:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c306b3c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619903.593648:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6000000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589026304:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619903.593692:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 291318 00010000:00010000:3.0:1788619903.593720:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe536873380c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae69 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.593747:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619903.593751:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619903.593758:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe536873380c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae69 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.593778:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d200, rc 0) 00000100:00100000:3.0:1788619903.593872:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e647ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2353:x1875503571283200:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3804us (4416us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619903.593882:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 635 00000100:00100000:1.0:1788619903.595231:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78dfa42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589026816:0@lo:400:kworker.0 00000100:00100000:1.0:1788619903.595339:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619903.595383:0:3318:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:1.0:1788619903.595389:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6000000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:3318:1875503589026432:0@lo:400:kworker.0 00000100:00100000:1.0:1788619903.595433:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619903.595464:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78dfa41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3318:1875503589026944:0@lo:400:kworker.0 00000100:00100000:1.0:1788619903.595494:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619903.595533:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589026432 00000100:00100000:1.0:1788619903.595542:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589026432 00000100:00100000:1.0:1788619903.595596:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284 00000100:00100000:1.0:1788619903.595606:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5ce1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+19:3318:x1875503589026432:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788619903.595661:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5ce1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+19:3318:x1875503589026432:12345-0@lo:400:kworker.0 Request processed in 55us (226us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619903.595668:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284 00000100:00100000:1.0:1788619903.595721:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6000000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:3318:1875503589026432:0@lo:400:kworker.0 00000100:00100000:0.0:1788619903.596371:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78cd3ca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3316:1875503589026560:0@lo:400:kworker.0 00000100:00100000:0.0:1788619903.596422:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788619903.596458:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78dfa40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589027072:0@lo:400:kworker.0 00000100:00100000:0.0:1788619903.596489:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788619903.596509:0:3316:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1788619903.596515:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78dfa43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3316:1875503589026688:0@lo:400:kworker.0 00000100:00100000:0.0:1788619903.596538:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788619903.596574:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589026816 00000100:00100000:0.0:1788619903.596581:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589026816 00000100:00100000:0.0:1788619903.596633:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589027072 00000100:00100000:0.0:1788619903.596636:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589027072 00000100:00100000:0.0:1788619903.596659:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 119 00000100:00100000:0.0:1788619903.596667:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e647a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589026816:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788619903.596716:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e647a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589026816:12345-0@lo:400:kworker.0 Request processed in 50us (1392us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619903.596724:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 119 00000100:00100000:0.0:1788619903.596741:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 120 00000100:00100000:0.0:1788619903.596747:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe13b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503589027072:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788619903.596783:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe13b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503589027072:12345-0@lo:400:kworker.0 Request processed in 36us (293us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619903.596808:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 120 00000100:00080000:0.0:1788619903.597002:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1788619903.597008:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1788619903.597048:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78dfa40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589027072:0@lo:400:kworker.0 00000100:00100000:0.0:1788619903.597097:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589026944 00000100:00100000:0.0:1788619903.597102:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589026944 00000100:00080000:0.0:1788619903.597120:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1788619903.597144:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589026560 00000100:00100000:0.0:1788619903.597147:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589026560 00000100:00100000:0.0:1788619903.597168:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589026688 00000100:00100000:0.0:1788619903.597170:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589026688 00000100:00100000:0.0:1788619903.597188:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 177 00000100:00100000:2.0:1788619903.597313:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 178 00000100:00100000:2.0:1788619903.597349:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3316:x1875503589026560:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788619903.597400:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3316:x1875503589026560:12345-0@lo:400:kworker.0 Request processed in 79us (978us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619903.597418:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 178 00000100:00100000:2.0:1788619903.597431:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 179 00000100:00100000:2.0:1788619903.597437:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3316:x1875503589026688:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788619903.597468:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3316:x1875503589026688:12345-0@lo:400:kworker.0 Request processed in 31us (929us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619903.597474:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 179 00000100:00100000:0.0:1788619903.597627:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e6479f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3318:x1875503589026944:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788619903.597662:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e6479f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3318:x1875503589026944:12345-0@lo:400:kworker.0 Request processed in 468us (2168us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619903.597669:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 177 00000100:00080000:0.0:1788619903.597683:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1788619903.597705:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78dfa43b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3316:1875503589026688:0@lo:400:kworker.0 00000100:00100000:0.0:1788619903.597717:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78cd3ca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3316:1875503589026560:0@lo:400:kworker.0 00000100:00100000:1.0:1788619903.597820:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78dfa41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3318:1875503589026944:0@lo:400:kworker.0 00000100:00100000:1.0:1788619903.597834:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78dfa42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589026816:0@lo:400:kworker.0 00000100:00100000:1.0:1788619903.601532:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619903.601639:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571283328 00000100:00100000:2.0:1788619903.601651:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571283328 00000100:00100000:2.0:1788619903.601728:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 636 00000100:00100000:2.0:1788619903.601739:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e647b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2352:x1875503571283328:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619903.601767:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619903.601797:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c200/0x1cd7fe5368733813 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ae70 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619903.601850:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4ae00/0x1cd7fe5368733805 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae62 expref: 17 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619903.601903:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df384380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589027200:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619903.602015:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccf00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619903.606009:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df384380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589027200:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619903.606060:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 291318 00010000:00010000:2.0:1788619903.606089:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c200/0x1cd7fe5368733813 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae70 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619903.606118:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619903.606122:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619903.606151:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c200/0x1cd7fe5368733813 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae70 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619903.606170:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505c200, rc 0) 00000100:00100000:2.0:1788619903.606262:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e647b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2352:x1875503571283328:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4523us (4743us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619903.606272:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 636 00000100:00100000:1.0:1788619903.610748:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619903.610842:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571283456 00000100:00100000:0.0:1788619903.610866:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571283456 00000100:00100000:0.0:1788619903.610965:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 637 00000100:00100000:0.0:1788619903.610979:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e6478700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:14908:x1875503571283456:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619903.611006:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619903.611045:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4ac00/0x1cd7fe536873381a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ae77 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619903.611110:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4ae00/0x1cd7fe5368733805 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae62 expref: 17 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619903.611122:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4ac00/0x1cd7fe536873381a lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae77 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619903.611167:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4ae00/0x1cd7fe5368733805 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae62 expref: 17 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619903.611184:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4ae00/0x1cd7fe5368733805 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae62 expref: 17 pid: 11397 timeout: 496 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619903.611211:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c4a4f800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589027328:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619903.611372:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa240 going to sleep for 11 seconds 00000100:00100000:0.0:1788619903.615271:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c4a4f800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589027328:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619903.615529:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4ac00/0x1cd7fe536873381a lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae77 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619903.615553:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619903.615557:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619903.615561:0:5845:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4ac00/0x1cd7fe536873381a lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae77 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619903.615574:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4ac00, rc 0) 00000100:00100000:0.0:1788619903.615655:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e6478700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+13:14908:x1875503571283456:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4679us (4921us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619903.615686:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 637 00000100:00100000:1.0:1788619903.618126:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619903.618770:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571283584 00000100:00100000:3.0:1788619903.618784:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571283584 00010000:00010000:3.0:1788619903.619054:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4ae00/0x1cd7fe5368733805 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae62 expref: 18 pid: 11397 timeout: 496 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619903.619157:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 206 00000100:00100000:3.0:1788619903.619169:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e6478000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2355:x1875503571283584:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619903.619202:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619903.619238:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4ae00/0x1cd7fe5368733805 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae62 expref: 18 pid: 11397 timeout: 496 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.619253:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4ae00/0x1cd7fe5368733805 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae62 expref: 18 pid: 11397 timeout: 496 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.619291:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4ae00/0x1cd7fe5368733805 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae62 expref: 18 pid: 11397 timeout: 496 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.619315:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619903.619362:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4ac00/0x1cd7fe536873381a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae77 expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.619378:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4ac00/0x1cd7fe536873381a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae77 expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.619389:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4ac00/0x1cd7fe536873381a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae77 expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.619443:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4ac00/0x1cd7fe536873381a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae77 expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619903.619474:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78dfa41500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589027456:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619903.619596:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c306b3c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619903.624699:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78dfa41500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589027456:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619903.624738:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619903.624805:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e6478000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2355:x1875503571283584:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 5638us (6691us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619903.624819:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 206 00000100:00100000:0.0:1788619903.630533:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619903.630665:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571283712 00000100:00100000:2.0:1788619903.630680:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571283712 00002000:00100000:2.0:1788619903.630753:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c4a4f800 x1875503571283712/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:754/0 lens 488/0 e 0 to 0 dl 1788619914 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619903.630769:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c4a4f800 with x1875503571283712 00000100:00100000:2.0:1788619903.630865:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 291 00000100:00100000:2.0:1788619903.630878:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571283712:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619903.637792:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571283712:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 6899us (7258us total) trans 0 rc 294912/294912 00000100:00100000:3.0:1788619903.637810:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 291 00002000:00100000:3.0:1788619903.637828:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c4a4f800 x1875503571283712/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:754/0 lens 488/440 e 0 to 0 dl 1788619914 ref 1 fl Complete:/600/0 rc 294912/294912 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619903.637851:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c4a4f800 with x1875503571283712 00000100:00100000:1.0:1788619903.642746:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619903.642934:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571283840 00000100:00100000:0.0:1788619903.642948:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571283840 00000100:00100000:0.0:1788619903.643078:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 638 00000100:00100000:0.0:1788619903.643092:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e6478380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+17:14908:x1875503571283840:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619903.643126:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619903.643153:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4aa00/0x1cd7fe5368733821 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ae7e expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619903.643298:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4a200/0x1cd7fe5368733828 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619903.643317:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4a200/0x1cd7fe5368733828 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619903.643538:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4a200/0x1cd7fe5368733828 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619903.643581:0:9417:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1609382 00000004:00200000:0.0:1788619903.643587:0:9417:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619903.643611:0:9417:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4a200/0x1cd7fe5368733828 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619903.643646:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4aa00/0x1cd7fe5368733821 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae7e expref: 19 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619903.643666:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4a200/0x1cd7fe5368733828 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae7e expref: 18 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619903.643701:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619903.643704:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619903.643709:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4a200, rc 0) 00000100:00100000:0.0:1788619903.643837:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e6478380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+18:14908:x1875503571283840:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 747us (1106us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619903.643854:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 638 00000100:00100000:1.0:1788619903.648420:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619903.648745:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571283968 00000100:00100000:2.0:1788619903.648759:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571283968 00000100:00100000:1.0:1788619903.648896:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619903.649051:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571284096 00000100:00100000:3.0:1788619903.649057:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571284096 00000100:00100000:3.0:1788619903.649111:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 427 00000100:00100000:3.0:1788619903.649125:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e647aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571284096:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619903.649151:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619903.649176:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f980cda4/0x1cd7fe536873382f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ae8c expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619903.649273:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78dfa41500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589027584:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619903.649425:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c306b3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619903.649653:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 639 00000100:00100000:2.0:1788619903.649665:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e6479880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2353:x1875503571283968:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619903.649679:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619903.649698:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c200/0x1cd7fe5368733836 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ae85 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619903.649735:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c200/0x1cd7fe5368733836 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae85 expref: 19 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788619903.649751:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 291318 00010000:00010000:2.0:1788619903.649759:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c200/0x1cd7fe5368733836 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae85 expref: 19 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619903.649784:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619903.649788:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619903.649793:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505c200, rc 0) 00000100:00100000:2.0:1788619903.649832:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e6479880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2353:x1875503571283968:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 171us (1639us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619903.649850:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 639 00002000:00010000:3.0:1788619903.653056:0:6697:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619899 -> 1788619903 00002000:00010000:3.0:1788619903.653068:0:6697:0:(ofd_lvb.c:284:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks: 263 -> 391 00000100:00100000:3.0:1788619903.653098:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78dfa41500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589027584:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619903.653145:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f980cda4/0x1cd7fe536873382f lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae8c expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.653175:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f980cda4/0x1cd7fe536873382f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae8c expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.653200:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d200, rc 0) 00000100:00100000:3.0:1788619903.653273:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e647aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571284096:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4151us (4377us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619903.653288:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 427 00000100:00100000:1.0:1788619903.658778:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619903.658899:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571284224 00000100:00100000:0.0:1788619903.658923:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571284224 00000100:00100000:0.0:1788619903.658989:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 428 00000100:00100000:0.0:1788619903.659002:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e647bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571284224:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619903.659038:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619903.659065:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff761517/0x1cd7fe536873383d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ae93 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619903.659145:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c4a4ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589027712:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619903.659291:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa240 going to sleep for 11 seconds 00000100:00100000:0.0:1788619903.662738:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c4a4ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589027712:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619903.662796:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ff761517/0x1cd7fe536873383d lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae93 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619903.662828:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff761517/0x1cd7fe536873383d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae93 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619903.662856:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4be00, rc 0) 00000100:00100000:0.0:1788619903.662953:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e647bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571284224:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3953us (4190us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619903.662963:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 428 00000100:00100000:3.0:1788619903.667463:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619903.667696:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571284352 00000100:00100000:1.0:1788619903.667709:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571284352 00000100:00100000:1.0:1788619903.667788:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 429 00000100:00100000:1.0:1788619903.667802:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa41500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571284352:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619903.667829:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619903.667883:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000078ef3fe/0x1cd7fe5368733844 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ae9a expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619903.667967:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e647a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589027840:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619903.668109:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e7902158540 going to sleep for 11 seconds 00000100:00100000:0.0:1788619903.672270:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e647a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589027840:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619903.672486:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000078ef3fe/0x1cd7fe5368733844 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae9a expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619903.672520:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000078ef3fe/0x1cd7fe5368733844 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae9a expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619903.672537:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b51600, rc 0) 00000100:00100000:1.0:1788619903.672751:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa41500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571284352:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4949us (5301us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619903.672762:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 429 00000100:00100000:0.0:1788619903.677799:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619903.678079:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571284480 00000100:00100000:1.0:1788619903.678092:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571284480 00000100:00100000:1.0:1788619903.678218:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 640 00000100:00100000:1.0:1788619903.678231:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+19:14908:x1875503571284480:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619903.678261:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619903.678265:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:1.0:1788619903.678362:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c200/0x1cd7fe5368733836 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae85 expref: 19 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619903.678387:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619903.678393:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788619903.678411:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50400/0x1cd7fe536873384b lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1aea1 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619903.678443:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4ac00/0x1cd7fe536873381a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae77 expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619903.678454:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50400/0x1cd7fe536873384b lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aea1 expref: 19 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619903.678523:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4ac00/0x1cd7fe536873381a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae77 expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619903.678538:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4ac00/0x1cd7fe536873381a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae77 expref: 12 pid: 5845 timeout: 496 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619903.678554:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e6479c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589027968:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619903.678674:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e7902158540 going to sleep for 11 seconds 00000100:00100000:2.0:1788619903.688962:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619903.689162:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e6479c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589027968:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619903.689213:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50400/0x1cd7fe536873384b lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aea1 expref: 19 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619903.689239:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619903.689242:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619903.689248:0:5845:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50400/0x1cd7fe536873384b lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aea1 expref: 19 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619903.689262:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b50400, rc 0) 00000100:00100000:0.0:1788619903.689390:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+19:14908:x1875503571284480:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 11160us (11607us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619903.689400:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 640 00000100:00100000:3.0:1788619903.689900:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571284608 00000100:00100000:3.0:1788619903.689912:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571284608 00010000:00010000:3.0:1788619903.689959:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4ac00/0x1cd7fe536873381a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae77 expref: 13 pid: 5845 timeout: 496 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619903.690007:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 207 00000100:00100000:3.0:1788619903.690028:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df384380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571284608:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619903.690045:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619903.690080:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4ac00/0x1cd7fe536873381a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae77 expref: 13 pid: 5845 timeout: 496 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.690096:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4ac00/0x1cd7fe536873381a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae77 expref: 13 pid: 5845 timeout: 496 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.690134:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4ac00/0x1cd7fe536873381a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae77 expref: 13 pid: 5845 timeout: 496 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.690152:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619903.690158:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50400/0x1cd7fe536873384b lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aea1 expref: 18 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.690171:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50400/0x1cd7fe536873384b lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aea1 expref: 18 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.690182:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50400/0x1cd7fe536873384b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aea1 expref: 18 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.690228:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50400/0x1cd7fe536873384b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aea1 expref: 18 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619903.690245:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78dfa43480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589028096:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619903.690372:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c306b3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619903.694679:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619903.695132:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78dfa43480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589028096:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619903.695167:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619903.695246:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df384380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571284608:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 5228us (6298us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619903.695260:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 207 00000100:00100000:0.0:1788619903.697620:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571284736 00000100:00100000:0.0:1788619903.697641:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571284736 00000100:00100000:1.0:1788619903.697877:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 430 00000100:00100000:1.0:1788619903.697891:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df385880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571284736:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619903.697932:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619903.697958:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005295b935/0x1cd7fe5368733852 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1aea8 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619903.698057:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e6478e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589028224:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619903.698181:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e7902158540 going to sleep for 11 seconds 00000100:00100000:1.0:1788619903.701664:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e6478e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589028224:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619903.701721:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005295b935/0x1cd7fe5368733852 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aea8 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619903.701752:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005295b935/0x1cd7fe5368733852 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aea8 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619903.701793:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b50a00, rc 0) 00000100:00100000:1.0:1788619903.701943:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df385880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571284736:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4053us (7278us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619903.701954:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 430 00000100:00100000:2.0:1788619903.705437:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619903.705619:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571284864 00000100:00100000:3.0:1788619903.705633:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571284864 00002000:00100000:3.0:1788619903.705723:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df386a00 x1875503571284864/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/0 e 0 to 0 dl 1788619914 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619903.705737:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df386a00 with x1875503571284864 00000100:00100000:3.0:1788619903.705784:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 292 00000100:00100000:3.0:1788619903.705796:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df386a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2354:x1875503571284864:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619903.706302:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df386a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2354:x1875503571284864:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 505us (877us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619903.706311:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 292 00002000:00100000:3.0:1788619903.706319:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df386a00 x1875503571284864/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/440 e 0 to 0 dl 1788619914 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619903.706453:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df386a00 with x1875503571284864 00000100:00100000:2.0:1788619903.710482:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619903.710932:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571284992 00000100:00100000:3.0:1788619903.710946:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571284992 00000100:00100000:2.0:1788619903.711131:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 431 00000100:00100000:2.0:1788619903.711145:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df387b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571284992:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619903.711170:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619903.711192:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000611bb050/0x1cd7fe5368733859 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1aeaf expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619903.711275:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df386680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589028352:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619903.711497:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccf00 going to sleep for 11 seconds 00000100:00100000:0.0:1788619903.714811:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df386680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589028352:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619903.714933:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000611bb050/0x1cd7fe5368733859 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aeaf expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619903.714971:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000611bb050/0x1cd7fe5368733859 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aeaf expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619903.714987:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505d600, rc 0) 00000100:00100000:0.0:1788619903.715149:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df387b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571284992:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4006us (4681us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619903.715159:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 431 00000100:00100000:2.0:1788619903.724167:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619903.724363:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571285120 00000100:00100000:0.0:1788619903.724377:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571285120 00000100:00100000:0.0:1788619903.724495:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 432 00000100:00100000:0.0:1788619903.724509:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df385c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571285120:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619903.724535:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619903.724562:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008db03736/0x1cd7fe5368733860 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1aeb6 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619903.724648:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c4a4ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589028480:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619903.724759:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa240 going to sleep for 11 seconds 00000100:00100000:0.0:1788619903.727795:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c4a4ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589028480:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619903.727917:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008db03736/0x1cd7fe5368733860 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aeb6 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619903.728037:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008db03736/0x1cd7fe5368733860 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aeb6 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619903.728058:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4b000, rc 0) 00000100:00100000:0.0:1788619903.728139:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df385c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571285120:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3633us (3989us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619903.728150:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 432 00000100:00100000:2.0:1788619903.731778:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619903.731883:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571285248 00000100:00100000:3.0:1788619903.731896:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571285248 00002000:00100000:3.0:1788619903.731977:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df384e00 x1875503571285248/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/0 e 0 to 0 dl 1788619914 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619903.731990:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df384e00 with x1875503571285248 00000100:00100000:3.0:1788619903.732091:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 293 00000100:00100000:3.0:1788619903.732104:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df384e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2352:x1875503571285248:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619903.732457:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df384e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2352:x1875503571285248:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 354us (697us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619903.732467:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 293 00002000:00100000:3.0:1788619903.732476:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df384e00 x1875503571285248/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/440 e 0 to 0 dl 1788619914 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619903.732509:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df384e00 with x1875503571285248 00000100:00100000:2.0:1788619903.736717:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619903.736827:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571285376 00000100:00100000:1.0:1788619903.736965:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571285376 00000100:00100000:1.0:1788619903.737086:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 433 00000100:00100000:1.0:1788619903.737097:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df385500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571285376:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619903.737119:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619903.737142:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005295b935/0x1cd7fe5368733867 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1aebd expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619903.737221:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e6478e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589028608:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619903.737570:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e7902158540 going to sleep for 11 seconds 00000100:00100000:1.0:1788619903.740660:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e6478e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589028608:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619903.740720:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005295b935/0x1cd7fe5368733867 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aebd expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619903.740753:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005295b935/0x1cd7fe5368733867 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aebd expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619903.740770:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b50a00, rc 0) 00000100:00100000:1.0:1788619903.741071:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df385500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571285376:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3973us (4369us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619903.741104:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 433 00000100:00100000:2.0:1788619903.745442:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619903.745542:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571285504 00000100:00100000:1.0:1788619903.745555:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571285504 00000100:00100000:1.0:1788619903.749900:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 434 00000100:00100000:1.0:1788619903.749946:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df387480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571285504:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619903.749975:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619903.750002:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000b633971/0x1cd7fe536873386e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1aec4 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619903.750106:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e6478e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589028736:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619903.750246:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e7902158540 going to sleep for 11 seconds 00000100:00100000:3.0:1788619903.753384:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e6478e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589028736:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619903.753444:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000b633971/0x1cd7fe536873386e lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aec4 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.753480:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000b633971/0x1cd7fe536873386e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aec4 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.753497:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b51e00, rc 0) 00000100:00100000:3.0:1788619903.753578:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df387480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571285504:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3652us (8301us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619903.753589:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 434 00000100:00100000:2.0:1788619903.757594:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619903.757700:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571285632 00000100:00100000:3.0:1788619903.757714:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571285632 00002000:00100000:3.0:1788619903.757783:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df385180 x1875503571285632/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/0 e 0 to 0 dl 1788619914 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619903.757796:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df385180 with x1875503571285632 00000100:00100000:3.0:1788619903.757933:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 294 00000100:00100000:3.0:1788619903.757946:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df385180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2355:x1875503571285632:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619903.758384:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df385180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2355:x1875503571285632:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 438us (805us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619903.758394:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 294 00002000:00100000:3.0:1788619903.758402:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df385180 x1875503571285632/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/440 e 0 to 0 dl 1788619914 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619903.758429:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df385180 with x1875503571285632 00000100:00100000:2.0:1788619903.763558:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619903.763644:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571285760 00000100:00100000:3.0:1788619903.763659:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571285760 00000100:00100000:3.0:1788619903.763765:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 435 00000100:00100000:3.0:1788619903.763780:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df387800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571285760:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619903.763805:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619903.763830:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f980cda4/0x1cd7fe5368733875 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1aecb expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619903.763939:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78dfa43480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589028864:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619903.764076:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c306b3c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619903.767034:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78dfa43480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589028864:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619903.767089:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f980cda4/0x1cd7fe5368733875 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aecb expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.767123:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f980cda4/0x1cd7fe5368733875 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aecb expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.767139:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d200, rc 0) 00000100:00100000:3.0:1788619903.767257:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df387800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571285760:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3479us (3711us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619903.767267:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 435 00000100:00100000:2.0:1788619903.774145:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619903.774291:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571285888 00000100:00100000:0.0:1788619903.774306:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571285888 00000100:00100000:0.0:1788619903.774414:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 436 00000100:00100000:0.0:1788619903.774428:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df384a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571285888:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619903.774452:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619903.774479:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008db03736/0x1cd7fe536873387c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1aed2 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619903.774573:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c4a4ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589028992:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619903.774696:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa240 going to sleep for 11 seconds 00000100:00100000:0.0:1788619903.778906:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c4a4ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589028992:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619903.778965:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008db03736/0x1cd7fe536873387c lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aed2 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619903.779001:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008db03736/0x1cd7fe536873387c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aed2 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619903.779051:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4b000, rc 0) 00000100:00100000:0.0:1788619903.779137:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df384a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571285888:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4710us (5006us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619903.779148:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 436 00010000:00010000:1.0:1788619903.780066:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788619903.783788:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619903.783955:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571286016 00000100:00100000:3.0:1788619903.783969:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571286016 00002000:00100000:3.0:1788619903.784069:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df386d80 x1875503571286016/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/0 e 0 to 0 dl 1788619914 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619903.784084:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df386d80 with x1875503571286016 00000100:00100000:3.0:1788619903.784188:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 295 00000100:00100000:3.0:1788619903.784202:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df386d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2353:x1875503571286016:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619903.784572:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df386d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2353:x1875503571286016:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 371us (797us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619903.784581:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 295 00002000:00100000:3.0:1788619903.784590:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df386d80 x1875503571286016/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/440 e 0 to 0 dl 1788619914 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619903.784599:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df386d80 with x1875503571286016 00000100:00100000:2.0:1788619903.788653:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619903.788778:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571286144 00000100:00100000:1.0:1788619903.788792:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571286144 00000100:00100000:1.0:1788619903.788969:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 437 00000100:00100000:1.0:1788619903.788983:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df386300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571286144:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619903.789012:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619903.789047:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005295b935/0x1cd7fe5368733883 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1aed9 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619903.789134:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e6479500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589029120:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619903.789303:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e7902158540 going to sleep for 11 seconds 00000100:00100000:1.0:1788619903.792172:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e6479500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589029120:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619903.792231:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005295b935/0x1cd7fe5368733883 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aed9 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619903.792388:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005295b935/0x1cd7fe5368733883 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aed9 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619903.792405:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b50a00, rc 0) 00000100:00100000:1.0:1788619903.792529:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df386300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571286144:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3547us (3891us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619903.792540:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 437 00000100:00100000:2.0:1788619903.796276:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619903.796545:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571286272 00000100:00100000:3.0:1788619903.796558:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571286272 00000100:00100000:3.0:1788619903.796678:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 438 00000100:00100000:3.0:1788619903.796692:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df385f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571286272:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619903.796720:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619903.796745:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe536873388a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1aee0 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619903.796847:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78dfa43480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589029248:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619903.796975:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c306b3c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619903.800525:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78dfa43480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589029248:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619903.800584:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe536873388a lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aee0 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.800635:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe536873388a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aee0 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.800653:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683dc00, rc 0) 00000100:00100000:3.0:1788619903.800732:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df385f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571286272:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4043us (4471us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619903.800741:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 438 00000100:00100000:2.0:1788619903.805452:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619903.805668:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571286400 00000100:00100000:3.0:1788619903.805685:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571286400 00002000:00100000:3.0:1788619903.805764:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df386680 x1875503571286400/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/0 e 0 to 0 dl 1788619914 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619903.805778:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df386680 with x1875503571286400 00000100:00100000:3.0:1788619903.805818:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 296 00000100:00100000:3.0:1788619903.805830:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df386680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2354:x1875503571286400:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619903.806494:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df386680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2354:x1875503571286400:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 661us (1051us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619903.806506:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 296 00002000:00100000:3.0:1788619903.806517:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df386680 x1875503571286400/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/440 e 0 to 0 dl 1788619914 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619903.806528:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df386680 with x1875503571286400 00000100:00100000:2.0:1788619903.811544:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619903.811651:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571286528 00000100:00100000:3.0:1788619903.811666:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571286528 00000100:00100000:3.0:1788619903.811743:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 439 00000100:00100000:3.0:1788619903.811756:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df387100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571286528:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619903.811802:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619903.811827:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe5368733891 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1aee7 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619903.811942:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78dfa43480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589029376:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619903.812074:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c306b3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619903.815610:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619903.815666:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619903.815820:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571286784 00000100:00100000:0.0:1788619903.815833:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571286784 00000100:00100000:0.0:1788619903.815921:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 121 00000100:00100000:0.0:1788619903.815932:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010fc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571286784:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:0.0:1788619903.816045:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010fc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571286784:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 114us (377us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619903.816054:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 121 00000100:00100000:1.0:1788619903.816464:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78dfa43480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589029376:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619903.816515:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe5368733891 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aee7 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619903.816549:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe5368733891 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aee7 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619903.816567:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683dc00, rc 0) 00000100:00100000:1.0:1788619903.816619:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df387100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571286528:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4866us (5092us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619903.816628:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 439 00000100:00100000:3.0:1788619903.816966:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571286656 00000100:00100000:3.0:1788619903.816972:0:5836:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571286656 00000100:00100000:3.0:1788619903.817028:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 285 00000100:00100000:3.0:1788619903.817038:0:5836:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5ce3050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:597450a2-e90c-408c-b401-a0ce48eed204+13:2355:x1875503571286656:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:3.0:1788619903.817095:0:5836:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5ce3050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:597450a2-e90c-408c-b401-a0ce48eed204+13:2355:x1875503571286656:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 59us (1502us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619903.817101:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 285 00000100:00100000:0.0:1788619903.823954:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619903.824080:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571286912 00000100:00100000:1.0:1788619903.824097:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571286912 00000100:00100000:1.0:1788619903.824174:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 440 00000100:00100000:1.0:1788619903.824187:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571286912:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619903.824213:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619903.824257:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005295b935/0x1cd7fe5368733898 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1aeee expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619903.824405:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e6479500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589029504:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619903.824523:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e7902158540 going to sleep for 11 seconds 00000100:00100000:1.0:1788619903.827400:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e6479500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589029504:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619903.827457:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005295b935/0x1cd7fe5368733898 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aeee expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619903.827494:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005295b935/0x1cd7fe5368733898 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aeee expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619903.827509:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b50a00, rc 0) 00000100:00100000:1.0:1788619903.827587:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571286912:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3402us (3647us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619903.827598:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 440 00000100:00100000:2.0:1788619903.835307:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619903.835594:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571287040 00000100:00100000:3.0:1788619903.835610:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571287040 00002000:00100000:3.0:1788619903.835689:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6531880 x1875503571287040/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/0 e 0 to 0 dl 1788619914 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619903.835704:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6531880 with x1875503571287040 00000100:00100000:3.0:1788619903.835752:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 297 00000100:00100000:3.0:1788619903.835765:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6531880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2352:x1875503571287040:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619903.836126:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6531880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2352:x1875503571287040:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 362us (848us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619903.836135:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 297 00002000:00100000:3.0:1788619903.836144:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6531880 x1875503571287040/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/440 e 0 to 0 dl 1788619914 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619903.836179:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6531880 with x1875503571287040 00000100:00100000:0.0:1788619903.841165:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619903.841296:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571287168 00000100:00100000:1.0:1788619903.841311:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571287168 00000100:00100000:1.0:1788619903.841438:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 441 00000100:00100000:1.0:1788619903.841451:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571287168:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619903.841477:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619903.841501:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005295b935/0x1cd7fe536873389f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1aef5 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619903.841585:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e6479500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589029632:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619903.841720:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e7902158540 going to sleep for 11 seconds 00000100:00100000:1.0:1788619903.845816:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e6479500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589029632:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619903.845875:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005295b935/0x1cd7fe536873389f lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aef5 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619903.845984:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005295b935/0x1cd7fe536873389f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aef5 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619903.846003:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b50a00, rc 0) 00000100:00100000:1.0:1788619903.846154:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571287168:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4704us (5007us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619903.846166:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 441 00000100:00100000:0.0:1788619903.851096:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619903.851195:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571287296 00000100:00100000:3.0:1788619903.851209:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571287296 00000100:00100000:3.0:1788619903.851298:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 442 00000100:00100000:3.0:1788619903.851309:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571287296:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619903.851367:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619903.851394:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f980cda4/0x1cd7fe53687338a6 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1aefc expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619903.851479:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78dfa41180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589029760:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619903.851600:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c306b6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619903.855151:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78dfa41180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589029760:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619903.855216:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f980cda4/0x1cd7fe53687338a6 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aefc expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.855250:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f980cda4/0x1cd7fe53687338a6 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aefc expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.855268:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d200, rc 0) 00000100:00100000:2.0:1788619903.856580:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571287296:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5259us (5485us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619903.856598:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 442 00000100:00100000:2.0:1788619903.860492:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619903.860621:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571287424 00000100:00100000:3.0:1788619903.860635:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571287424 00002000:00100000:3.0:1788619903.860719:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6531f80 x1875503571287424/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/0 e 0 to 0 dl 1788619914 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619903.860733:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6531f80 with x1875503571287424 00000100:00100000:3.0:1788619903.860812:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 298 00000100:00100000:3.0:1788619903.860826:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6531f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2352:x1875503571287424:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619903.861217:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6531f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2352:x1875503571287424:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 392us (743us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619903.861227:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 298 00002000:00100000:3.0:1788619903.861235:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6531f80 x1875503571287424/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/440 e 0 to 0 dl 1788619914 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619903.861244:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6531f80 with x1875503571287424 00000100:00100000:0.0:1788619903.865276:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619903.865475:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571287552 00000100:00100000:2.0:1788619903.865489:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571287552 00000100:00100000:2.0:1788619903.865570:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 443 00000100:00100000:2.0:1788619903.865584:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571287552:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619903.865608:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619903.865633:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007770e595/0x1cd7fe53687338ad lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1af03 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619903.865716:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6533100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589029888:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619903.865829:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccc00 going to sleep for 11 seconds 00000100:00100000:3.0:1788619903.869231:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6533100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589029888:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619903.869465:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007770e595/0x1cd7fe53687338ad lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af03 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.869499:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007770e595/0x1cd7fe53687338ad lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af03 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619903.869515:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505d000, rc 0) 00000100:00100000:0.0:1788619903.869723:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571287552:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4139us (4459us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619903.869735:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 443 00000100:00100000:0.0:1788619903.876084:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619903.876490:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571287680 00000100:00100000:1.0:1788619903.876506:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571287680 00000100:00100000:1.0:1788619903.876642:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 444 00000100:00100000:1.0:1788619903.876657:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571287680:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619903.876682:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619903.876708:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005295b935/0x1cd7fe53687338b4 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1af0a expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619903.876797:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e6479500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589030016:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619903.876948:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e7902158540 going to sleep for 11 seconds 00000100:00100000:1.0:1788619903.879905:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e6479500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589030016:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619903.879962:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005295b935/0x1cd7fe53687338b4 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af0a expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619903.880038:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005295b935/0x1cd7fe53687338b4 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af0a expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619903.880056:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b50a00, rc 0) 00000100:00100000:1.0:1788619903.880141:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571287680:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3487us (4072us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619903.880152:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 444 00000100:00100000:0.0:1788619903.883774:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619903.884039:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571287808 00000100:00100000:2.0:1788619903.884054:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571287808 00002000:00100000:2.0:1788619903.884129:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c4a4c000 x1875503571287808/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/0 e 0 to 0 dl 1788619914 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619903.884142:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c4a4c000 with x1875503571287808 00000100:00100000:2.0:1788619903.884210:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 299 00000100:00100000:2.0:1788619903.884223:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2352:x1875503571287808:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619903.884745:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2352:x1875503571287808:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 523us (985us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619903.884756:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 299 00002000:00100000:2.0:1788619903.884764:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c4a4c000 x1875503571287808/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/440 e 0 to 0 dl 1788619914 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619903.884773:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c4a4c000 with x1875503571287808 00000100:00100000:2.0:1788619903.889105:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619903.889196:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571287936 00000100:00100000:1.0:1788619903.889209:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571287936 00000100:00100000:1.0:1788619903.889296:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 445 00000100:00100000:1.0:1788619903.889309:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6532300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571287936:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619903.889366:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619903.889394:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d0cadab6/0x1cd7fe53687338bb lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1af11 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619903.889478:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e6479500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589030144:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619903.889594:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e7902158540 going to sleep for 11 seconds 00000100:00100000:1.0:1788619903.892564:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e6479500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589030144:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619903.892624:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d0cadab6/0x1cd7fe53687338bb lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af11 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619903.892659:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d0cadab6/0x1cd7fe53687338bb lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af11 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619903.892705:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b51400, rc 0) 00000100:00100000:1.0:1788619903.892777:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6532300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571287936:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3470us (3686us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619903.892787:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 445 00000100:00100000:2.0:1788619903.897094:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619903.897228:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571288064 00000100:00100000:0.0:1788619903.897244:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571288064 00000100:00100000:0.0:1788619903.897373:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 446 00000100:00100000:0.0:1788619903.897387:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6533100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571288064:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619903.897440:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619903.897468:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf7745d2/0x1cd7fe53687338c2 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1af18 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619903.897553:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c4a4c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589030272:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619903.897921:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa240 going to sleep for 11 seconds 00000100:00100000:1.0:1788619903.900768:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c4a4c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589030272:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619903.900827:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000cf7745d2/0x1cd7fe53687338c2 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af18 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619903.900876:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf7745d2/0x1cd7fe53687338c2 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af18 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619903.900894:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4a000, rc 0) 00000100:00100000:1.0:1788619903.900977:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6533100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571288064:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3592us (3900us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619903.900988:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 446 00000100:00100000:3.0:1788619903.905584:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619903.905813:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571288192 00000100:00100000:1.0:1788619903.905827:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571288192 00002000:00100000:1.0:1788619903.905917:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78dfa41180 x1875503571288192/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/0 e 0 to 0 dl 1788619914 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619903.905930:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78dfa41180 with x1875503571288192 00000100:00100000:1.0:1788619903.905971:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 300 00000100:00100000:1.0:1788619903.905982:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa41180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2352:x1875503571288192:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619903.906406:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa41180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2352:x1875503571288192:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 424us (839us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619903.906432:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 300 00002000:00100000:1.0:1788619903.906440:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78dfa41180 x1875503571288192/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/440 e 0 to 0 dl 1788619914 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619903.906448:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78dfa41180 with x1875503571288192 00000100:00100000:2.0:1788619903.919619:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619903.919752:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571288320 00000100:00100000:0.0:1788619903.919766:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571288320 00000100:00100000:0.0:1788619903.919853:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 447 00000100:00100000:0.0:1788619903.919866:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b13100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571288320:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619903.919891:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619903.919925:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e36287d4/0x1cd7fe53687338c9 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1af1f expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619903.920007:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c4a4d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589030400:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619903.920178:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:0.0:1788619903.924347:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c4a4d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589030400:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619903.924411:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e36287d4/0x1cd7fe53687338c9 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af1f expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619903.924444:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e36287d4/0x1cd7fe53687338c9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af1f expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619903.924461:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4a800, rc 0) 00000100:00100000:0.0:1788619903.925679:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b13100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571288320:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5802us (6062us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619903.925730:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 447 00000100:00100000:3.0:1788619903.931023:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619903.931141:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571288448 00000100:00100000:2.0:1788619903.931154:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571288448 00000100:00100000:2.0:1788619903.931241:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 448 00000100:00100000:2.0:1788619903.931255:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa43800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571288448:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619903.931282:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619903.931318:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e60363e3/0x1cd7fe53687338d0 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1af26 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619903.931438:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b10380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589030528:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619903.931573:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619903.936257:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b10380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589030528:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619903.936537:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e60363e3/0x1cd7fe53687338d0 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af26 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619903.936573:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e60363e3/0x1cd7fe53687338d0 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af26 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619903.936591:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505d800, rc 0) 00000100:00100000:0.0:1788619903.936945:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa43800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571288448:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5689us (5945us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619903.936958:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 448 00000100:00100000:1.0:1788619903.941900:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619903.942073:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571288576 00000100:00100000:3.0:1788619903.942087:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571288576 00002000:00100000:3.0:1788619903.942154:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e6479500 x1875503571288576/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/0 e 0 to 0 dl 1788619914 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619903.942167:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e6479500 with x1875503571288576 00000100:00100000:3.0:1788619903.942209:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 301 00000100:00100000:3.0:1788619903.942220:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e6479500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2353:x1875503571288576:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619903.942787:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e6479500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2353:x1875503571288576:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 566us (898us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619903.942797:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 301 00002000:00100000:3.0:1788619903.942807:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e6479500 x1875503571288576/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/440 e 0 to 0 dl 1788619914 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619903.942816:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e6479500 with x1875503571288576 00000100:00100000:1.0:1788619903.948156:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619903.948457:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571288704 00000100:00100000:3.0:1788619903.948471:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571288704 00000100:00100000:2.0:1788619903.948697:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 449 00000100:00100000:2.0:1788619903.948781:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e647b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571288704:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619903.948811:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619903.948906:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000da933269/0x1cd7fe53687338d7 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1af2d expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619903.949032:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b10380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589030656:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619903.949200:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc780 going to sleep for 11 seconds 00000100:00100000:1.0:1788619903.953083:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b10380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589030656:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619903.953140:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000da933269/0x1cd7fe53687338d7 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af2d expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619903.953175:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000da933269/0x1cd7fe53687338d7 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af2d expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619903.953193:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505c600, rc 0) 00000100:00100000:1.0:1788619903.953283:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e647b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571288704:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4566us (5451us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619903.953316:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 449 00000100:00100000:1.0:1788619903.957608:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619903.957747:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571288832 00000100:00100000:0.0:1788619903.957760:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571288832 00000100:00100000:0.0:1788619903.957845:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 450 00000100:00100000:0.0:1788619903.957858:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e6479180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571288832:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619903.957884:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619903.957907:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e36287d4/0x1cd7fe53687338de lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1af34 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619903.957994:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c4a4d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589030784:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619903.958137:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:0.0:1788619903.966777:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c4a4d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589030784:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619903.966853:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e36287d4/0x1cd7fe53687338de lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af34 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619903.966888:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e36287d4/0x1cd7fe53687338de lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af34 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619903.966906:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4a800, rc 0) 00000100:00100000:0.0:1788619903.967064:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e6479180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571288832:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 9207us (9468us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619903.967099:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 450 00000100:00100000:3.0:1788619903.975548:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619903.976235:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571288960 00000100:00100000:2.0:1788619903.976252:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571288960 00002000:00100000:2.0:1788619903.976518:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78dfa40a80 x1875503571288960/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/0 e 0 to 0 dl 1788619914 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619903.976534:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78dfa40a80 with x1875503571288960 00000100:00100000:2.0:1788619903.976645:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 302 00000100:00100000:2.0:1788619903.976658:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa40a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2355:x1875503571288960:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619903.977150:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa40a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2355:x1875503571288960:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 492us (1616us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619903.977160:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 302 00002000:00100000:2.0:1788619903.977168:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78dfa40a80 x1875503571288960/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/440 e 0 to 0 dl 1788619914 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619903.977178:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78dfa40a80 with x1875503571288960 00000100:00100000:3.0:1788619903.980969:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619903.981144:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571289088 00000100:00100000:0.0:1788619903.981159:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571289088 00000100:00100000:0.0:1788619903.981248:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 451 00000100:00100000:0.0:1788619903.981261:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa42d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571289088:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619903.981297:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619903.981323:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e36287d4/0x1cd7fe53687338e5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1af3b expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619903.981442:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c4a4d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589030912:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619903.981550:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:0.0:1788619903.984814:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c4a4d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589030912:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619903.984885:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e36287d4/0x1cd7fe53687338e5 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af3b expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619903.984918:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e36287d4/0x1cd7fe53687338e5 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af3b expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619903.984933:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4a800, rc 0) 00000100:00100000:0.0:1788619903.985019:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa42d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571289088:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3752us (4057us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619903.985030:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 451 00000100:00100000:3.0:1788619903.990164:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619903.990289:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571289216 00000100:00100000:1.0:1788619903.990302:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571289216 00000100:00100000:1.0:1788619903.990408:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 452 00000100:00100000:1.0:1788619903.990419:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa42300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571289216:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619903.990446:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619903.990471:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a7bead88/0x1cd7fe53687338ec lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1af42 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619903.990554:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e6478e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589031040:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619903.990662:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e7902158540 going to sleep for 11 seconds 00000100:00100000:1.0:1788619903.992489:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e6478e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589031040:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619903.992530:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a7bead88/0x1cd7fe53687338ec lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af42 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619903.992555:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a7bead88/0x1cd7fe53687338ec lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af42 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619903.992566:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b51000, rc 0) 00000100:00100000:1.0:1788619903.992634:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa42300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571289216:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2217us (2487us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619903.992642:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 452 00000100:00100000:3.0:1788619903.994846:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619903.994940:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571289344 00000100:00100000:2.0:1788619903.994954:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571289344 00002000:00100000:2.0:1788619903.995035:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78dfa41f80 x1875503571289344/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/0 e 0 to 0 dl 1788619914 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619903.995048:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78dfa41f80 with x1875503571289344 00000100:00100000:2.0:1788619903.995129:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 303 00000100:00100000:2.0:1788619903.995139:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa41f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2355:x1875503571289344:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619903.995509:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa41f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2355:x1875503571289344:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 370us (674us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619903.995533:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 303 00002000:00100000:2.0:1788619903.995539:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78dfa41f80 x1875503571289344/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:754/0 lens 488/440 e 0 to 0 dl 1788619914 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619903.995546:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78dfa41f80 with x1875503571289344 00000100:00100000:3.0:1788619903.998028:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619903.998477:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571289472 00000100:00100000:1.0:1788619903.998509:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571289472 00000100:00100000:1.0:1788619903.998623:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 453 00000100:00100000:1.0:1788619903.998636:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa43100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571289472:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619903.998660:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619903.998685:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a7bead88/0x1cd7fe53687338f3 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1af49 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619903.998843:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e6478e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589031168:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619903.999005:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e7902158540 going to sleep for 11 seconds 00000100:00100000:1.0:1788619904.001915:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e6478e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589031168:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619904.001976:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a7bead88/0x1cd7fe53687338f3 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af49 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619904.002025:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a7bead88/0x1cd7fe53687338f3 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af49 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619904.002044:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b51000, rc 0) 00000100:00100000:1.0:1788619904.002134:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa43100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571289472:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3500us (4131us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619904.002144:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 453 00000100:00100000:3.0:1788619904.006798:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619904.006932:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571289600 00000100:00100000:0.0:1788619904.006948:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571289600 00000100:00100000:0.0:1788619904.007083:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 454 00000100:00100000:0.0:1788619904.007096:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa40700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571289600:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619904.007120:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619904.007144:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e36287d4/0x1cd7fe53687338fa lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1af50 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619904.007225:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c4a4d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589031296:192.168.201.5@tcp:106: 00000100:00080000:1.0:1788619904.007463:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1788619904.007604:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:0.0:1788619904.011533:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c4a4d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589031296:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619904.011592:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e36287d4/0x1cd7fe53687338fa lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af50 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619904.011628:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e36287d4/0x1cd7fe53687338fa lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af50 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619904.011641:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4a800, rc 0) 00000100:00100000:0.0:1788619904.011706:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa40700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571289600:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4615us (5003us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619904.013183:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 454 00000100:00100000:2.0:1788619904.015804:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619904.015971:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571289728 00000100:00100000:3.0:1788619904.015985:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571289728 00002000:00100000:3.0:1788619904.016152:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b10380 x1875503571289728/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:0/0 lens 488/0 e 0 to 0 dl 1788619915 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619904.016166:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b10380 with x1875503571289728 00000100:00100000:3.0:1788619904.016219:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 304 00000100:00100000:3.0:1788619904.016232:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b10380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2352:x1875503571289728:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619904.016674:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b10380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2352:x1875503571289728:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 443us (883us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619904.016683:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 304 00002000:00100000:3.0:1788619904.016691:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b10380 x1875503571289728/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:0/0 lens 488/440 e 0 to 0 dl 1788619915 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619904.016729:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b10380 with x1875503571289728 00000100:00080000:1.0:1788619904.016828:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788619904.022182:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619904.023787:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571289856 00000100:00100000:0.0:1788619904.023805:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571289856 00000100:00100000:0.0:1788619904.024021:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 455 00000100:00100000:0.0:1788619904.024035:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa40000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571289856:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619904.024060:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619904.024086:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e36287d4/0x1cd7fe5368733901 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1af57 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619904.024196:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c4a4d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589031424:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619904.024560:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:2.0:1788619904.027750:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c4a4d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589031424:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619904.027807:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e36287d4/0x1cd7fe5368733901 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af57 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619904.027848:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e36287d4/0x1cd7fe5368733901 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af57 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619904.027865:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4a800, rc 0) 00000100:00100000:2.0:1788619904.027943:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa40000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571289856:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3911us (5778us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619904.027954:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 455 00000100:00100000:2.0:1788619904.031783:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619904.031874:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571289984 00000100:00100000:1.0:1788619904.031888:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571289984 00000100:00100000:1.0:1788619904.031976:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 456 00000100:00100000:1.0:1788619904.031991:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8597100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571289984:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619904.032049:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619904.032075:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a7bead88/0x1cd7fe5368733908 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1af5e expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619904.032157:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e6478e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589031552:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619904.032291:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e7902158540 going to sleep for 11 seconds 00000100:00100000:1.0:1788619904.035187:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e6478e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589031552:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619904.035240:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a7bead88/0x1cd7fe5368733908 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af5e expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619904.035271:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a7bead88/0x1cd7fe5368733908 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af5e expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619904.035464:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b51000, rc 0) 00000100:00100000:1.0:1788619904.035562:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8597100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571289984:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3572us (3803us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619904.035571:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 456 00000100:00100000:2.0:1788619904.057773:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619904.058046:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571290112 00000100:00100000:3.0:1788619904.058062:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571290112 00002000:00100000:3.0:1788619904.058138:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c8594000 x1875503571290112/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:0/0 lens 488/0 e 0 to 0 dl 1788619915 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619904.058152:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c8594000 with x1875503571290112 00000100:00100000:3.0:1788619904.058203:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 305 00000100:00100000:3.0:1788619904.058215:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8594000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2353:x1875503571290112:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619904.059845:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8594000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2353:x1875503571290112:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 1609us (2066us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619904.059865:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 305 00002000:00100000:0.0:1788619904.059878:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c8594000 x1875503571290112/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:0/0 lens 488/440 e 0 to 0 dl 1788619915 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619904.059889:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c8594000 with x1875503571290112 00000100:00100000:3.0:1788619904.064132:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619904.064241:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571290240 00000100:00100000:1.0:1788619904.064254:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571290240 00000100:00100000:1.0:1788619904.064373:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 457 00000100:00100000:1.0:1788619904.064386:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571290240:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619904.064412:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619904.064437:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a7bead88/0x1cd7fe536873390f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1af65 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619904.064550:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e6478e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589031680:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619904.065215:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e7902158540 going to sleep for 11 seconds 00000100:00100000:1.0:1788619904.069100:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e6478e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589031680:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619904.069156:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a7bead88/0x1cd7fe536873390f lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af65 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619904.069189:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a7bead88/0x1cd7fe536873390f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af65 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619904.069205:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b51000, rc 0) 00000100:00100000:1.0:1788619904.069296:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571290240:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4911us (5179us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619904.069306:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 457 00000100:00100000:2.0:1788619904.075787:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619904.076166:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571290368 00000100:00100000:1.0:1788619904.076180:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571290368 00002000:00100000:1.0:1788619904.076247:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c501b800 x1875503571290368/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:0/0 lens 488/0 e 0 to 0 dl 1788619915 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619904.076260:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c501b800 with x1875503571290368 00000100:00100000:1.0:1788619904.076314:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 306 00000100:00100000:1.0:1788619904.076380:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2353:x1875503571290368:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:1.0:1788619904.083591:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 291318 -> 352256 00000004:00010000:1.0:1788619904.083604:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619903 -> 1788619903 00000004:00010000:1.0:1788619904.083607:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619901 -> 1788619901 00000004:00010000:1.0:1788619904.083620:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619903 -> 1788619903 00000004:00010000:1.0:1788619904.083622:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1158 -> 1158 00000100:00100000:1.0:1788619904.084217:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2353:x1875503571290368:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 7884us (8439us total) trans 4294967553 rc 0/0 00000100:00100000:1.0:1788619904.084231:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 306 00002000:00100000:1.0:1788619904.084250:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c501b800 x1875503571290368/t4294967553(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:0/0 lens 488/448 e 0 to 0 dl 1788619915 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619904.084260:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c501b800 with x1875503571290368 00000100:00100000:3.0:1788619904.089463:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619904.089623:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571290496 00000100:00100000:0.0:1788619904.089636:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571290496 00000100:00100000:0.0:1788619904.089787:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 641 00000100:00100000:0.0:1788619904.089800:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa41880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2355:x1875503571290496:12345-192.168.201.5@tcp:16:fsx.0 00000100:00100000:0.0:1788619904.098990:0:3317:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c4a4ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3317:1875503589031808:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1788619904.099102:0:3317:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788619904.099178:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589031808 00000100:00100000:0.0:1788619904.099187:0:6699:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589031808 00000100:00100000:0.0:1788619904.099246:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 82 00000100:00100000:0.0:1788619904.099255:0:6699:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3317:x1875503589031808:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1788619904.099307:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3856662528 avail 3854565376 00000100:00100000:0.0:1788619904.099936:0:6699:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3317:x1875503589031808:12345-0@lo:13:osp-pre-1-0.0 Request processed in 678us (831us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619904.099945:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 82 00000100:00100000:0.0:1788619904.100037:0:3317:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c4a4ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3317:1875503589031808:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1788619904.122763:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa41880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2355:x1875503571290496:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 32949us (33298us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619904.122783:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 641 00000001:00080000:0.0:1788619904.123020:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967544 is committed 00000001:00080000:0.0:1788619904.123035:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967553 is committed 00000001:00080000:0.0:1788619904.123056:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967551 is committed 00000100:00100000:3.0:1788619904.126667:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619904.126796:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571290624 00000100:00100000:0.0:1788619904.126809:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571290624 00000100:00100000:0.0:1788619904.126944:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 458 00000100:00100000:0.0:1788619904.126958:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571290624:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619904.126982:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619904.127005:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008db03736/0x1cd7fe5368733916 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1af6c expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619904.127104:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c4a4ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589031936:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619904.127222:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619904.130106:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c4a4ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589031936:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619904.130167:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008db03736/0x1cd7fe5368733916 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af6c expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619904.130202:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008db03736/0x1cd7fe5368733916 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af6c expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619904.130240:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4b000, rc 0) 00000100:00100000:2.0:1788619904.130396:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571290624:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3440us (3742us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619904.130407:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 458 00000100:00100000:3.0:1788619904.134069:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619904.134196:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571290752 00000100:00100000:0.0:1788619904.134210:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571290752 00000100:00100000:0.0:1788619904.134296:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 459 00000100:00100000:0.0:1788619904.134311:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571290752:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619904.134390:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619904.134426:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071bcbab9/0x1cd7fe536873391d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1af73 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619904.134541:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c4a4e680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589032064:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619904.134698:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa180 going to sleep for 11 seconds 00000100:00100000:0.0:1788619904.138238:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c4a4e680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589032064:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619904.138312:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000071bcbab9/0x1cd7fe536873391d lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af73 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619904.138402:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071bcbab9/0x1cd7fe536873391d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af73 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619904.138419:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4ac00, rc 0) 00000100:00100000:0.0:1788619904.138492:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571290752:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4184us (4445us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619904.138503:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 459 00000100:00100000:2.0:1788619904.143198:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619904.143543:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571290880 00000100:00100000:3.0:1788619904.143559:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571290880 00000100:00100000:3.0:1788619904.143656:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 642 00000100:00100000:3.0:1788619904.143669:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571290880:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619904.143700:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619904.143733:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368733924 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1af7a expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619904.143777:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50400/0x1cd7fe536873384b lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aea1 expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619904.143789:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368733924 lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af7a expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619904.143866:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50400/0x1cd7fe536873384b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aea1 expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619904.143882:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50400/0x1cd7fe536873384b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aea1 expref: 12 pid: 5845 timeout: 496 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619904.143899:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78dfa42680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589032192:192.168.201.5@tcp:104: 00000100:00080000:1.0:1788619904.144073:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788619904.144216:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619904.149528:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619904.149671:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78dfa42680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589032192:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619904.149729:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368733924 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af7a expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619904.149755:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619904.149759:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619904.149764:0:5844:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368733924 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af7a expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619904.149780:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d200, rc 0) 00000100:00100000:0.0:1788619904.150542:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571291008 00000100:00100000:0.0:1788619904.150554:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571291008 00010000:00010000:0.0:1788619904.150609:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50400/0x1cd7fe536873384b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aea1 expref: 13 pid: 5845 timeout: 496 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619904.150655:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 208 00000100:00100000:0.0:1788619904.150668:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8021c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571291008:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619904.150686:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619904.150720:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50400/0x1cd7fe536873384b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aea1 expref: 13 pid: 5845 timeout: 496 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619904.150734:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50400/0x1cd7fe536873384b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aea1 expref: 13 pid: 5845 timeout: 496 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619904.150763:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50400/0x1cd7fe536873384b lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aea1 expref: 13 pid: 5845 timeout: 496 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619904.150817:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619904.150824:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368733924 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af7a expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619904.150845:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368733924 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af7a expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619904.150857:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368733924 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af7a expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619904.150917:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368733924 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af7a expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619904.150934:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c4a4e680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589032320:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619904.151119:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:14908:x1875503571290880:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7450us (7935us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619904.151130:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 642 00000100:00100000:0.0:1788619904.152896:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa180 going to sleep for 11 seconds 00000100:00100000:2.0:1788619904.160133:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619904.160234:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571291136 00000100:00100000:1.0:1788619904.160249:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571291136 00002000:00100000:1.0:1788619904.160356:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b15c00 x1875503571291136/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/0 e 0 to 0 dl 1788619915 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619904.160369:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b15c00 with x1875503571291136 00000100:00100000:1.0:1788619904.160411:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 307 00000100:00100000:1.0:1788619904.160423:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b15c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571291136:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619904.163737:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c4a4e680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589032320:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619904.163780:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619904.171807:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8021c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571291008:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 21121us (22278us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619904.171828:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 208 00000100:00100000:1.0:1788619904.178731:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b15c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571291136:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 18292us (18594us total) trans 0 rc 352256/352256 00000100:00100000:1.0:1788619904.178751:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 307 00002000:00100000:1.0:1788619904.178770:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b15c00 x1875503571291136/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/440 e 0 to 0 dl 1788619915 ref 1 fl Complete:/600/0 rc 352256/352256 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619904.178783:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b15c00 with x1875503571291136 00000100:00100000:0.0:1788619904.185821:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619904.185984:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619904.186092:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571291392 00000100:00100000:2.0:1788619904.186107:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571291392 00000100:00100000:2.0:1788619904.186192:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 460 00000100:00100000:2.0:1788619904.186205:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571291392:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619904.186236:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619904.186291:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007f93e9f3/0x1cd7fe536873392b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1af88 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619904.186422:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b14000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589032448:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619904.186546:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc000 going to sleep for 11 seconds 00002000:00010000:2.0:1788619904.189516:0:6698:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime: 1788619903 -> 1788619904 00002000:00010000:2.0:1788619904.189530:0:6698:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619903 -> 1788619904 00002000:00010000:2.0:1788619904.189533:0:6698:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime: 1788619903 -> 1788619904 00000100:00100000:2.0:1788619904.189577:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b14000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589032448:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619904.189623:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007f93e9f3/0x1cd7fe536873392b lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af88 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619904.189655:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f93e9f3/0x1cd7fe536873392b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af88 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619904.189671:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505de00, rc 0) 00000100:00100000:2.0:1788619904.189741:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571291392:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3538us (3756us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619904.189752:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 460 00000100:00100000:1.0:1788619904.191759:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571291264 00000100:00100000:1.0:1788619904.191782:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571291264 00000100:00100000:1.0:1788619904.192068:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 643 00000100:00100000:1.0:1788619904.192082:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2355:x1875503571291264:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619904.192107:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619904.192136:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50800/0x1cd7fe5368733932 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1af81 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619904.192184:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50800/0x1cd7fe5368733932 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af81 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788619904.192199:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 352256 00010000:00010000:1.0:1788619904.192208:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50800/0x1cd7fe5368733932 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af81 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619904.192226:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619904.192229:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619904.192235:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b50800, rc 0) 00000100:00100000:1.0:1788619904.192359:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571291264:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 281us (6555us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619904.192367:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 643 00000100:00100000:0.0:1788619904.195851:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619904.195969:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571291520 00000100:00100000:3.0:1788619904.195986:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571291520 00000100:00100000:3.0:1788619904.196074:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 461 00000100:00100000:3.0:1788619904.196086:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571291520:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619904.196109:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619904.196134:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe5368733939 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1af8f expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619904.196214:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78dfa41c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589032576:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619904.196485:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a900 going to sleep for 11 seconds 00000100:00100000:3.0:1788619904.199478:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78dfa41c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589032576:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619904.199540:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe5368733939 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af8f expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619904.199575:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe5368733939 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af8f expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619904.199612:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683ca00, rc 0) 00000100:00100000:3.0:1788619904.199741:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571291520:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3656us (3915us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619904.199751:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 461 00000100:00100000:0.0:1788619904.204720:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619904.204814:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571291648 00000100:00100000:2.0:1788619904.204827:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571291648 00000100:00100000:2.0:1788619904.204901:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 462 00000100:00100000:2.0:1788619904.204913:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571291648:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619904.204937:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619904.204963:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008b900302/0x1cd7fe5368733940 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1af96 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619904.205086:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b14000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589032704:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619904.205196:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc000 going to sleep for 11 seconds 00000100:00100000:2.0:1788619904.209771:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b14000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589032704:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619904.209828:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008b900302/0x1cd7fe5368733940 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af96 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619904.209870:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008b900302/0x1cd7fe5368733940 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af96 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619904.209886:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505c000, rc 0) 00000100:00100000:2.0:1788619904.209976:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571291648:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5063us (5273us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619904.209987:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 462 00000100:00100000:0.0:1788619904.215246:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619904.216015:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571291776 00000100:00100000:2.0:1788619904.216031:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571291776 00000100:00100000:2.0:1788619904.216172:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 644 00000100:00100000:2.0:1788619904.216184:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571291776:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619904.216212:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619904.216216:0:9417:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1788619904.216287:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b50800/0x1cd7fe5368733932 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af81 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619904.216314:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619904.216388:0:9417:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788619904.216409:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c000/0x1cd7fe5368733947 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1af9d expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619904.216445:0:9417:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368733924 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af7a expref: 9 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619904.216457:0:9417:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c000/0x1cd7fe5368733947 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af9d expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619904.216502:0:9417:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368733924 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af7a expref: 9 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619904.216518:0:9417:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368733924 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af7a expref: 9 pid: 5844 timeout: 496 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619904.216535:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b14000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589032832:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619904.216717:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc000 going to sleep for 11 seconds 00000100:00100000:0.0:1788619904.226648:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619904.226854:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b14000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589032832:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619904.226920:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c000/0x1cd7fe5368733947 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af9d expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619904.226946:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619904.226950:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619904.226973:0:9417:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c000/0x1cd7fe5368733947 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af9d expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619904.226988:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505c000, rc 0) 00000100:00100000:1.0:1788619904.227077:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571291776:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 10894us (11847us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619904.227088:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 644 00000100:00100000:3.0:1788619904.228377:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571291904 00000100:00100000:3.0:1788619904.228396:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571291904 00010000:00010000:3.0:1788619904.228450:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368733924 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af7a expref: 10 pid: 5844 timeout: 496 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619904.228498:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 209 00000100:00100000:3.0:1788619904.228509:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa15500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571291904:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619904.228526:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619904.228562:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368733924 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af7a expref: 10 pid: 5844 timeout: 496 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619904.228577:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368733924 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af7a expref: 10 pid: 5844 timeout: 496 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619904.228604:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368733924 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af7a expref: 10 pid: 5844 timeout: 496 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619904.228627:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619904.228634:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c000/0x1cd7fe5368733947 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af9d expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619904.228668:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c000/0x1cd7fe5368733947 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af9d expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619904.228681:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c000/0x1cd7fe5368733947 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af9d expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619904.228732:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c000/0x1cd7fe5368733947 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af9d expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619904.228751:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78dfa41c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589032960:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619904.230047:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a900 going to sleep for 11 seconds 00000100:00100000:0.0:1788619904.235093:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619904.235269:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571292032 00000100:00100000:1.0:1788619904.235292:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571292032 00002000:00100000:1.0:1788619904.235412:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78dfa14a80 x1875503571292032/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:0/0 lens 488/0 e 0 to 0 dl 1788619915 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619904.235426:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78dfa14a80 with x1875503571292032 00000100:00100000:1.0:1788619904.235474:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 308 00000100:00100000:1.0:1788619904.235485:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa14a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571292032:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619904.235803:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa14a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571292032:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 318us (724us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619904.235812:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 308 00002000:00100000:1.0:1788619904.235820:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78dfa14a80 x1875503571292032/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:0/0 lens 488/4536 e 0 to 0 dl 1788619915 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619904.235829:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78dfa14a80 with x1875503571292032 00000100:00100000:3.0:1788619904.236237:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78dfa41c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589032960:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619904.236271:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619904.236306:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa15500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571291904:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 7799us (9674us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619904.236317:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 209 00000100:00100000:0.0:1788619904.241593:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619904.242085:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571292160 00000100:00100000:1.0:1788619904.242100:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571292160 00002000:00100000:1.0:1788619904.242170:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c71f1500 x1875503571292160/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:0/0 lens 488/0 e 0 to 0 dl 1788619915 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619904.242183:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c71f1500 with x1875503571292160 00000100:00100000:1.0:1788619904.242232:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 309 00000100:00100000:1.0:1788619904.242252:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571292160:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619904.242623:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571292160:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 372us (1045us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619904.242632:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 309 00002000:00100000:1.0:1788619904.242639:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c71f1500 x1875503571292160/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:0/0 lens 488/4536 e 0 to 0 dl 1788619915 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619904.242647:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c71f1500 with x1875503571292160 00000100:00100000:0.0:1788619904.253122:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619904.253555:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571292288 00000100:00100000:2.0:1788619904.253570:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571292288 00002000:00100000:2.0:1788619904.253648:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be0a80 x1875503571292288/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:0/0 lens 488/0 e 0 to 0 dl 1788619915 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619904.253662:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be0a80 with x1875503571292288 00000100:00100000:2.0:1788619904.253699:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 310 00000100:00100000:2.0:1788619904.253711:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571292288:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619904.254042:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571292288:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 331us (935us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619904.254724:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 310 00002000:00100000:2.0:1788619904.254737:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be0a80 x1875503571292288/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:0/0 lens 488/4536 e 0 to 0 dl 1788619915 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619904.254746:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be0a80 with x1875503571292288 00000100:00100000:0.0:1788619904.271588:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619904.273538:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571292416 00000100:00100000:2.0:1788619904.273563:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571292416 00002000:00100000:2.0:1788619904.273655:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df8b3100 x1875503571292416/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:0/0 lens 488/0 e 0 to 0 dl 1788619915 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619904.273669:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df8b3100 with x1875503571292416 00000100:00100000:2.0:1788619904.273730:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 311 00000100:00100000:2.0:1788619904.273742:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571292416:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619904.274112:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571292416:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 371us (2537us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619904.274121:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 311 00002000:00100000:2.0:1788619904.274129:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df8b3100 x1875503571292416/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:0/0 lens 488/4536 e 0 to 0 dl 1788619915 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619904.274138:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df8b3100 with x1875503571292416 00000100:00100000:0.0:1788619904.281883:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619904.282146:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571292544 00000100:00100000:1.0:1788619904.282159:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571292544 00002000:00100000:1.0:1788619904.282225:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df8b3480 x1875503571292544/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:0/0 lens 488/0 e 0 to 0 dl 1788619915 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619904.282238:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df8b3480 with x1875503571292544 00000100:00100000:1.0:1788619904.282291:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 312 00000100:00100000:1.0:1788619904.282303:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571292544:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619904.282651:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571292544:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 349us (781us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619904.282660:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 312 00002000:00100000:1.0:1788619904.282668:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df8b3480 x1875503571292544/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:0/0 lens 488/4536 e 0 to 0 dl 1788619915 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619904.282676:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df8b3480 with x1875503571292544 00000100:00100000:1.0:1788619904.286988:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619904.287112:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571292672 00000100:00100000:0.0:1788619904.287135:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571292672 00002000:00100000:0.0:1788619904.287201:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e6478000 x1875503571292672/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:0/0 lens 488/0 e 0 to 0 dl 1788619915 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619904.287214:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e6478000 with x1875503571292672 00000100:00100000:0.0:1788619904.287261:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 313 00000100:00100000:0.0:1788619904.287622:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e6478000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571292672:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619904.288036:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e6478000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571292672:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 764us (1059us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619904.288045:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 313 00002000:00100000:0.0:1788619904.288053:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e6478000 x1875503571292672/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:0/0 lens 488/4536 e 0 to 0 dl 1788619915 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619904.288062:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e6478000 with x1875503571292672 00000100:00100000:0.0:1788619904.295087:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619904.295194:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571292800 00000100:00100000:2.0:1788619904.295207:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571292800 00002000:00100000:2.0:1788619904.295269:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df8b0000 x1875503571292800/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:0/0 lens 488/0 e 0 to 0 dl 1788619915 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619904.295453:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df8b0000 with x1875503571292800 00000100:00100000:2.0:1788619904.295538:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 314 00000100:00100000:2.0:1788619904.295549:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571292800:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619904.295878:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571292800:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 328us (803us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619904.295887:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 314 00002000:00100000:2.0:1788619904.295894:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df8b0000 x1875503571292800/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:0/0 lens 488/4536 e 0 to 0 dl 1788619915 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619904.295903:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df8b0000 with x1875503571292800 00000100:00100000:2.0:1788619904.300519:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619904.300609:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571292928 00000100:00100000:1.0:1788619904.300623:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571292928 00002000:00100000:1.0:1788619904.300689:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b14000 x1875503571292928/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:0/0 lens 488/0 e 0 to 0 dl 1788619915 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619904.300710:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b14000 with x1875503571292928 00000100:00100000:1.0:1788619904.300753:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 315 00000100:00100000:1.0:1788619904.300761:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b14000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571292928:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619904.301107:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b14000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571292928:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 343us (598us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619904.301116:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 315 00002000:00100000:1.0:1788619904.301123:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b14000 x1875503571292928/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:0/0 lens 488/4536 e 0 to 0 dl 1788619915 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619904.301131:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b14000 with x1875503571292928 00000100:00100000:0.0:1788619904.304507:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619904.304684:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571293056 00000100:00100000:2.0:1788619904.304694:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571293056 00002000:00100000:2.0:1788619904.304753:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c7582300 x1875503571293056/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:0/0 lens 488/0 e 0 to 0 dl 1788619915 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619904.304762:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c7582300 with x1875503571293056 00000100:00100000:2.0:1788619904.304801:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 316 00000100:00100000:2.0:1788619904.304811:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7582300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571293056:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:2.0:1788619904.308442:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 352256 -> 352256 00000004:00010000:2.0:1788619904.308455:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619903 -> 1788619904 00000004:00010000:2.0:1788619904.308458:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619901 -> 1788619901 00000004:00010000:2.0:1788619904.308461:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619903 -> 1788619904 00000004:00010000:2.0:1788619904.308464:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1158 -> 774 00000100:00100000:2.0:1788619904.308566:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7582300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2353:x1875503571293056:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 3745us (4065us total) trans 4294967554 rc 0/0 00000100:00100000:2.0:1788619904.308580:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 316 00002000:00100000:2.0:1788619904.308609:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c7582300 x1875503571293056/t4294967554(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:0/0 lens 488/448 e 0 to 0 dl 1788619915 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619904.308638:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c7582300 with x1875503571293056 00000100:00100000:0.0:1788619904.311135:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619904.311269:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571293184 00000100:00100000:2.0:1788619904.311362:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571293184 00000100:00100000:0.0:1788619904.311485:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 645 00000100:00100000:0.0:1788619904.311500:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5771180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2352:x1875503571293184:12345-192.168.201.5@tcp:16:fsx.0 00000100:00100000:0.0:1788619904.361697:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5771180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2352:x1875503571293184:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 50188us (50564us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619904.361719:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 645 00000001:00080000:1.0:1788619904.362195:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967554 is committed 00000100:00100000:0.0:1788619904.367899:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619904.368124:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571293312 00000100:00100000:2.0:1788619904.368139:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571293312 00000100:00100000:2.0:1788619904.368238:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 463 00000100:00100000:2.0:1788619904.368251:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5770700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571293312:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619904.368276:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619904.368301:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007770e595/0x1cd7fe536873394e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1afa4 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619904.368561:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4ba5f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589033088:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619904.368954:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cca80 going to sleep for 11 seconds 00000100:00100000:0.0:1788619904.381626:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4ba5f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589033088:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619904.381691:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007770e595/0x1cd7fe536873394e lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1afa4 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619904.381735:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007770e595/0x1cd7fe536873394e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1afa4 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619904.381755:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505d000, rc 0) 00000100:00100000:0.0:1788619904.381838:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5770700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571293312:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 13589us (13955us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619904.381849:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 463 00000100:00100000:0.0:1788619904.385973:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619904.386117:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571293440 00000100:00100000:1.0:1788619904.386132:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571293440 00000100:00100000:1.0:1788619904.386618:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 464 00000100:00100000:1.0:1788619904.386632:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5773100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571293440:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619904.386657:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619904.386684:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1d61422/0x1cd7fe5368733955 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1afab expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619904.386781:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e6478e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589033216:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619904.387036:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963a80 going to sleep for 11 seconds 00000100:00100000:1.0:1788619904.392049:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e6478e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589033216:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619904.392109:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e1d61422/0x1cd7fe5368733955 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1afab expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619904.392286:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1d61422/0x1cd7fe5368733955 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1afab expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619904.392306:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b51200, rc 0) 00000100:00100000:1.0:1788619904.392498:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5773100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571293440:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5868us (6538us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619904.392508:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 464 00000100:00100000:3.0:1788619904.398084:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619904.398298:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571293568 00000100:00100000:1.0:1788619904.398312:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571293568 00000100:00100000:1.0:1788619904.398523:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 465 00000100:00100000:1.0:1788619904.398534:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa41c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571293568:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619904.398558:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619904.398590:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1d61422/0x1cd7fe536873395c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1afb2 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619904.398676:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e6478e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589033344:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619904.398788:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963a80 going to sleep for 11 seconds 00002000:00010000:1.0:1788619904.404819:0:6697:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size: 1609382 -> 1785469 00000100:00100000:1.0:1788619904.404862:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e6478e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589033344:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619904.404910:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e1d61422/0x1cd7fe536873395c lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1afb2 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619904.404943:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1d61422/0x1cd7fe536873395c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1afb2 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619904.404983:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b51200, rc 0) 00000100:00100000:1.0:1788619904.405085:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa41c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571293568:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6551us (7298us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619904.405095:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 465 00000100:00100000:2.0:1788619904.409559:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619904.409719:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571293696 00000100:00100000:0.0:1788619904.409733:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571293696 00000100:00100000:0.0:1788619904.409809:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 466 00000100:00100000:0.0:1788619904.409819:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571293696:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619904.409840:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619904.409862:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d18e093/0x1cd7fe5368733963 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1afb9 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619904.409948:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5770380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589033472:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619904.410059:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa780 going to sleep for 11 seconds 00000100:00100000:0.0:1788619904.412553:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5770380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589033472:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619904.412671:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009d18e093/0x1cd7fe5368733963 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1afb9 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619904.412728:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d18e093/0x1cd7fe5368733963 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1afb9 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619904.412741:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4bc00, rc 0) 00000100:00100000:0.0:1788619904.412811:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571293696:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2993us (3271us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619904.412819:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 466 00000100:00100000:3.0:1788619904.416850:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619904.417052:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571293824 00000100:00100000:0.0:1788619904.417064:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571293824 00002000:00100000:0.0:1788619904.417136:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1585152->1785468) req@ffff8e78dfa43b80 x1875503571293824/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 504/0 e 0 to 0 dl 1788619915 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619904.417145:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa43b80 with x1875503571293824 ext(1585152->1785468) 00002000:00010000:0.0:1788619904.417150:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa43b80 00000100:00100000:0.0:1788619904.417191:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 246 00000100:00100000:0.0:1788619904.417202:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571293824:12345-192.168.201.5@tcp:4:fsx.0 00000100:00080000:1.0:1788619904.417481:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1788619904.420610:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+8:2352:x1875503571293824:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 3393us (3756us total) trans 4294967361 rc 0/0 00000100:00100000:0.0:1788619904.420628:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 246 00002000:00100000:0.0:1788619904.420645:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1585152->1785468) req@ffff8e78dfa43b80 x1875503571293824/t4294967361(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 504/448 e 0 to 0 dl 1788619915 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619904.420657:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa43b80 with x1875503571293824 ext(1585152->1785468) 00002000:00010000:0.0:1788619904.420665:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa43b80 00000100:00100000:2.0:1788619904.426966:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619904.427188:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571293952 00000100:00100000:0.0:1788619904.427201:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571293952 00000100:00100000:0.0:1788619904.427298:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 247 00000100:00100000:0.0:1788619904.427311:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+8:2353:x1875503571293952:12345-192.168.201.5@tcp:22:fsx.0 00002000:00010000:0.0:1788619904.427908:0:6702:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1785469 -> 1785469 00002000:00010000:0.0:1788619904.427913:0:6702:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619904 -> 1788619904 00002000:00010000:0.0:1788619904.427916:0:6702:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619904 -> 1788619903 00002000:00010000:0.0:1788619904.427919:0:6702:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619904 -> 1788619904 00002000:00010000:0.0:1788619904.427921:0:6702:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 391 -> 391 00000100:00100000:0.0:1788619904.428028:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571293952:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 717us (1335us total) trans 4294967362 rc 0/0 00000100:00100000:0.0:1788619904.428038:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 247 00000100:00100000:3.0:1788619904.431764:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619904.431860:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571294080 00000100:00100000:0.0:1788619904.431873:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571294080 00000100:00100000:0.0:1788619904.432016:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 467 00000100:00100000:0.0:1788619904.432030:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa40e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571294080:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619904.432054:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619904.432080:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d18e093/0x1cd7fe536873396a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1afc0 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619904.432163:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5770380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589033600:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619904.432280:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa780 going to sleep for 11 seconds 00002000:00010000:0.0:1788619904.434948:0:7654:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619903 -> 1788619904 00000100:00100000:0.0:1788619904.434992:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5770380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589033600:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619904.435062:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009d18e093/0x1cd7fe536873396a lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1afc0 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619904.435125:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d18e093/0x1cd7fe536873396a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1afc0 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619904.435140:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4bc00, rc 0) 00000100:00100000:0.0:1788619904.435276:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa40e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571294080:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3247us (3525us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619904.435287:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 467 00000100:00100000:2.0:1788619904.440457:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619904.441038:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571294208 00000100:00100000:1.0:1788619904.441052:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571294208 00000100:00100000:1.0:1788619904.441128:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 468 00000100:00100000:1.0:1788619904.441140:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571294208:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619904.441163:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619904.441185:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1d61422/0x1cd7fe5368733971 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1afc7 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619904.441412:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e6478e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589033728:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619904.441534:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963a80 going to sleep for 11 seconds 00000100:00100000:1.0:1788619904.449807:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e6478e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589033728:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619904.449877:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e1d61422/0x1cd7fe5368733971 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1afc7 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619904.449912:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1d61422/0x1cd7fe5368733971 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1afc7 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619904.449930:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b51200, rc 0) 00000100:00100000:1.0:1788619904.450897:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571294208:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 9748us (10448us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619904.450912:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 468 00000100:00100000:3.0:1788619904.463300:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619904.463509:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571294336 00000100:00100000:1.0:1788619904.463523:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571294336 00000100:00100000:1.0:1788619904.463601:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 646 00000100:00100000:1.0:1788619904.463613:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa42680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571294336:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619904.463640:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619904.463671:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51200/0x1cd7fe5368733978 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1afce expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619904.463720:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c000/0x1cd7fe5368733947 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af9d expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619904.463774:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e6478e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589033856:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619904.463884:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963a80 going to sleep for 11 seconds 00000004:00010000:1.0:1788619904.473057:0:5845:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619901 -> 1788619904 00000100:00100000:1.0:1788619904.473100:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e6478e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589033856:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619904.473130:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 352256 00010000:00010000:1.0:1788619904.473155:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51200/0x1cd7fe5368733978 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1afce expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619904.473181:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619904.473185:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619904.473191:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51200/0x1cd7fe5368733978 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1afce expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619904.473236:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b51200, rc 0) 00000100:00100000:1.0:1788619904.473520:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa42680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571294336:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 9906us (10242us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619904.473531:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 646 00000100:00100000:3.0:1788619904.476846:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619904.477125:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571294464 00000100:00100000:1.0:1788619904.477138:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571294464 00000100:00100000:1.0:1788619904.477214:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 647 00000100:00100000:1.0:1788619904.477226:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571294464:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619904.477251:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619904.477286:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51a00/0x1cd7fe536873397f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1afd5 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619904.477344:0:9417:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c000/0x1cd7fe5368733947 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af9d expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619904.477356:0:9417:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51a00/0x1cd7fe536873397f lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1afd5 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619904.477400:0:9417:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c000/0x1cd7fe5368733947 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af9d expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619904.477416:0:9417:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c000/0x1cd7fe5368733947 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af9d expref: 12 pid: 9417 timeout: 497 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619904.477431:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e6478e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589033984:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619904.477568:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963a80 going to sleep for 11 seconds 00000100:00100000:1.0:1788619904.485688:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e6478e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589033984:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619904.485745:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51a00/0x1cd7fe536873397f lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1afd5 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619904.485772:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619904.485776:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619904.485780:0:9417:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51a00/0x1cd7fe536873397f lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1afd5 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619904.486008:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b51a00, rc 0) 00000100:00100000:1.0:1788619904.486092:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:14908:x1875503571294464:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8866us (9275us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619904.486104:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 647 00000100:00100000:3.0:1788619904.486147:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619904.486239:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571294592 00000100:00100000:1.0:1788619904.486248:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571294592 00010000:00010000:1.0:1788619904.486291:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c000/0x1cd7fe5368733947 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af9d expref: 13 pid: 9417 timeout: 497 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619904.486458:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 210 00000100:00100000:2.0:1788619904.486475:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571294592:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619904.486494:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619904.486531:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c000/0x1cd7fe5368733947 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af9d expref: 13 pid: 9417 timeout: 497 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619904.486545:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c000/0x1cd7fe5368733947 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af9d expref: 13 pid: 9417 timeout: 497 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619904.486575:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c000/0x1cd7fe5368733947 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1af9d expref: 13 pid: 9417 timeout: 497 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619904.486712:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619904.486718:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51a00/0x1cd7fe536873397f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1afd5 expref: 9 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619904.486729:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51a00/0x1cd7fe536873397f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1afd5 expref: 9 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619904.486739:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51a00/0x1cd7fe536873397f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1afd5 expref: 9 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619904.486787:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51a00/0x1cd7fe536873397f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1afd5 expref: 9 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619904.487038:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f5180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589034112:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619904.487921:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc000 going to sleep for 11 seconds 00000100:00100000:2.0:1788619904.495906:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f5180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589034112:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619904.495951:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619904.496041:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2355:x1875503571294592:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 9569us (9891us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619904.496055:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 210 00000100:00100000:1.0:1788619904.499876:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619904.501112:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571294720 00000100:00100000:0.0:1788619904.501131:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571294720 00002000:00100000:0.0:1788619904.501513:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78cefacc50 x1875503571294720/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/0 e 0 to 0 dl 1788619915 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619904.501528:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78cefacc50 with x1875503571294720 00000100:00100000:1.0:1788619904.501617:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 317 00000100:00100000:1.0:1788619904.501634:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cefacc50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571294720:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619904.501973:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cefacc50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571294720:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 342us (2112us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619904.501983:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 317 00002000:00100000:1.0:1788619904.501991:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78cefacc50 x1875503571294720/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/4536 e 0 to 0 dl 1788619915 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619904.502028:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78cefacc50 with x1875503571294720 00000100:00100000:3.0:1788619904.504050:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619904.505460:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571294848 00000100:00100000:0.0:1788619904.505482:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571294848 00002000:00100000:0.0:1788619904.505574:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1609728->1765375) req@ffff8e78c75f5c00 x1875503571294848/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/0 e 0 to 0 dl 1788619915 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619904.505941:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c75f5c00 with x1875503571294848 ext(1609728->1765375) 00002000:00010000:0.0:1788619904.505951:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c75f5c00 00000100:00100000:0.0:1788619904.506013:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 248 00000100:00100000:0.0:1788619904.506027:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571294848:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619904.512322:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619904.512961:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571294976 00000100:00100000:1.0:1788619904.512978:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571294976 00002000:00100000:1.0:1788619904.513061:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e67f5180 x1875503571294976/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/0 e 0 to 0 dl 1788619915 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619904.513075:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e67f5180 with x1875503571294976 00000100:00100000:1.0:1788619904.513168:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 318 00000100:00100000:1.0:1788619904.513181:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571294976:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619904.513510:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571294976:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 330us (1203us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619904.513519:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 318 00002000:00100000:1.0:1788619904.513527:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e67f5180 x1875503571294976/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/4536 e 0 to 0 dl 1788619915 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619904.513535:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e67f5180 with x1875503571294976 00000100:00100000:0.0:1788619904.526299:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571294848:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 20255us (22241us total) trans 0 rc 155648/155648 00000100:00100000:0.0:1788619904.526321:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 248 00002000:00100000:0.0:1788619904.526508:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1609728->1765375) req@ffff8e78c75f5c00 x1875503571294848/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/440 e 0 to 0 dl 1788619915 ref 1 fl Complete:/600/0 rc 155648/155648 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619904.526523:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c75f5c00 with x1875503571294848 ext(1609728->1765375) 00002000:00010000:0.0:1788619904.526532:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c75f5c00 00000100:00100000:3.0:1788619904.527271:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619904.528402:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571295104 00000100:00100000:0.0:1788619904.528422:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571295104 00002000:00100000:0.0:1788619904.528508:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c75f4a80 x1875503571295104/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/0 e 0 to 0 dl 1788619915 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619904.528525:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c75f4a80 with x1875503571295104 00000100:00100000:0.0:1788619904.528574:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 319 00000100:00100000:0.0:1788619904.528982:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571295104:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619904.529793:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571295104:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 1203us (2522us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619904.529806:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 319 00002000:00100000:0.0:1788619904.529817:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c75f4a80 x1875503571295104/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/4536 e 0 to 0 dl 1788619915 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619904.529827:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c75f4a80 with x1875503571295104 00000100:00100000:3.0:1788619904.533783:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619904.533991:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571295232 00000100:00100000:1.0:1788619904.534016:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571295232 00002000:00100000:1.0:1788619904.534085:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c75f4700 x1875503571295232/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/0 e 0 to 0 dl 1788619915 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619904.534100:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c75f4700 with x1875503571295232 00000100:00100000:1.0:1788619904.534160:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 320 00000100:00100000:1.0:1788619904.534172:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571295232:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619904.534541:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571295232:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 370us (773us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619904.534550:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 320 00002000:00100000:1.0:1788619904.534567:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c75f4700 x1875503571295232/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/4536 e 0 to 0 dl 1788619915 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619904.534575:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c75f4700 with x1875503571295232 00000100:00100000:2.0:1788619904.541540:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619904.542116:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571295360 00000100:00100000:1.0:1788619904.542143:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571295360 00002000:00100000:1.0:1788619904.542241:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e67f7b80 x1875503571295360/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/0 e 0 to 0 dl 1788619915 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619904.542265:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e67f7b80 with x1875503571295360 00000100:00100000:1.0:1788619904.542322:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 321 00000100:00100000:1.0:1788619904.542379:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571295360:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619904.542704:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571295360:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 327us (1177us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619904.542715:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 321 00002000:00100000:1.0:1788619904.542723:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e67f7b80 x1875503571295360/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/4536 e 0 to 0 dl 1788619915 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619904.542730:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e67f7b80 with x1875503571295360 00000100:00100000:2.0:1788619904.551800:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619904.552801:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571295488 00000100:00100000:1.0:1788619904.552820:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571295488 00002000:00100000:1.0:1788619904.552914:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e67f6300 x1875503571295488/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/0 e 0 to 0 dl 1788619915 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619904.552929:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e67f6300 with x1875503571295488 00000100:00100000:1.0:1788619904.552984:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 322 00000100:00100000:1.0:1788619904.552996:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571295488:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619904.553611:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571295488:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 613us (1826us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619904.553640:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 322 00002000:00100000:1.0:1788619904.553650:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e67f6300 x1875503571295488/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/4536 e 0 to 0 dl 1788619915 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619904.553658:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e67f6300 with x1875503571295488 00000100:00100000:3.0:1788619904.559506:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619904.559808:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571295616 00000100:00100000:0.0:1788619904.559837:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571295616 00002000:00100000:0.0:1788619904.559915:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c75f6300 x1875503571295616/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/0 e 0 to 0 dl 1788619915 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619904.559928:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c75f6300 with x1875503571295616 00000100:00100000:0.0:1788619904.559978:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 323 00000100:00100000:0.0:1788619904.559991:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571295616:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619904.560408:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571295616:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 417us (916us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619904.560417:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 323 00002000:00100000:0.0:1788619904.560425:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c75f6300 x1875503571295616/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/4536 e 0 to 0 dl 1788619915 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619904.560434:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c75f6300 with x1875503571295616 00000100:00100000:2.0:1788619904.564484:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619904.564708:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571295744 00000100:00100000:1.0:1788619904.564723:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571295744 00002000:00100000:1.0:1788619904.564792:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e67f4700 x1875503571295744/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/0 e 0 to 0 dl 1788619915 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619904.564806:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e67f4700 with x1875503571295744 00000100:00100000:1.0:1788619904.564865:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 324 00000100:00100000:1.0:1788619904.564877:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571295744:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619904.565195:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571295744:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 319us (727us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619904.565204:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 324 00002000:00100000:1.0:1788619904.565212:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e67f4700 x1875503571295744/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/4536 e 0 to 0 dl 1788619915 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619904.565221:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e67f4700 with x1875503571295744 00000100:00100000:1.0:1788619904.570765:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619904.570955:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571295872 00000100:00100000:2.0:1788619904.570987:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571295872 00002000:00100000:2.0:1788619904.571073:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e6478a80 x1875503571295872/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/0 e 0 to 0 dl 1788619915 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619904.571085:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e6478a80 with x1875503571295872 00000100:00100000:2.0:1788619904.571132:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 325 00000100:00100000:2.0:1788619904.571144:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e6478a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571295872:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619904.571711:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e6478a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571295872:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 565us (958us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619904.571721:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 325 00002000:00100000:2.0:1788619904.571730:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e6478a80 x1875503571295872/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/4536 e 0 to 0 dl 1788619915 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619904.571740:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e6478a80 with x1875503571295872 00000100:00100000:1.0:1788619904.576790:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619904.576877:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571296000 00000100:00100000:0.0:1788619904.576892:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571296000 00002000:00100000:0.0:1788619904.576959:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e6479500 x1875503571296000/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/0 e 0 to 0 dl 1788619915 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619904.576973:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e6479500 with x1875503571296000 00000100:00100000:0.0:1788619904.577031:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 326 00000100:00100000:0.0:1788619904.577043:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e6479500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571296000:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619904.577910:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e6479500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571296000:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 863us (1132us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619904.577921:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 326 00002000:00100000:0.0:1788619904.577929:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e6479500 x1875503571296000/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/4536 e 0 to 0 dl 1788619915 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619904.577939:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e6479500 with x1875503571296000 00000100:00100000:1.0:1788619904.583065:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619904.583176:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571296128 00000100:00100000:2.0:1788619904.583189:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571296128 00002000:00100000:2.0:1788619904.583463:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e6478000 x1875503571296128/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/0 e 0 to 0 dl 1788619915 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619904.583476:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e6478000 with x1875503571296128 00000100:00100000:2.0:1788619904.583563:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 327 00000100:00100000:2.0:1788619904.583576:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e6478000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571296128:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619904.584050:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e6478000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571296128:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 474us (996us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619904.584059:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 327 00002000:00100000:2.0:1788619904.584067:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e6478000 x1875503571296128/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/4536 e 0 to 0 dl 1788619915 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619904.584076:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e6478000 with x1875503571296128 00000100:00100000:2.0:1788619904.589103:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619904.589237:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571296256 00000100:00100000:0.0:1788619904.589445:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571296256 00002000:00100000:0.0:1788619904.589528:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fefd4e00 x1875503571296256/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/0 e 0 to 0 dl 1788619915 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619904.589541:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fefd4e00 with x1875503571296256 00000100:00100000:0.0:1788619904.589654:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 328 00000100:00100000:0.0:1788619904.589666:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571296256:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619904.590111:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571296256:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 445us (1021us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619904.590121:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 328 00002000:00100000:0.0:1788619904.590129:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fefd4e00 x1875503571296256/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/4536 e 0 to 0 dl 1788619915 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619904.590138:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fefd4e00 with x1875503571296256 00000100:00100000:1.0:1788619904.595548:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619904.595650:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571296384 00000100:00100000:2.0:1788619904.595664:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571296384 00000100:00100000:2.0:1788619904.595758:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 249 00000100:00100000:2.0:1788619904.595772:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e6478e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571296384:12345-192.168.201.5@tcp:22:fsx.0 00002000:00010000:2.0:1788619904.596122:0:6701:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1785469 -> 1785469 00002000:00010000:2.0:1788619904.596127:0:6701:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619904 -> 1788619904 00002000:00010000:2.0:1788619904.596131:0:6701:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619904 -> 1788619903 00002000:00010000:2.0:1788619904.596133:0:6701:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619904 -> 1788619904 00002000:00010000:2.0:1788619904.596136:0:6701:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 391 -> 391 00000100:00100000:2.0:1788619904.596221:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e6478e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571296384:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 452us (687us total) trans 4294967363 rc 0/0 00000100:00100000:2.0:1788619904.596229:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 249 00000100:00100000:1.0:1788619904.600686:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619904.600743:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619904.600871:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571296512 00000100:00100000:0.0:1788619904.600885:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571296512 00000100:00100000:0.0:1788619904.600967:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 648 00000100:00100000:0.0:1788619904.600979:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e647a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2355:x1875503571296512:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619904.601016:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619904.601045:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4a000/0x1cd7fe5368733986 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1afdc expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619904.601092:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4a000/0x1cd7fe5368733986 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1afdc expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788619904.601107:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 352256 00010000:00010000:0.0:1788619904.601116:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4a000/0x1cd7fe5368733986 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1afdc expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619904.601132:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619904.601135:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619904.601140:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4a000, rc 0) 00000100:00100000:0.0:1788619904.601220:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e647a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571296512:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 243us (476us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619904.601255:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 648 00000100:00100000:3.0:1788619904.602322:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571296640 00000100:00100000:3.0:1788619904.602369:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571296640 00000100:00100000:3.0:1788619904.602429:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 469 00000100:00100000:3.0:1788619904.602440:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e6479c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571296640:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619904.602458:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619904.602476:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe536873398d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1afe3 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619904.602550:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c75f5500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589034240:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619904.602989:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a240 going to sleep for 11 seconds 00002000:00010000:3.0:1788619904.608072:0:6697:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619903 -> 1788619904 00000100:00100000:3.0:1788619904.608116:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c75f5500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589034240:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619904.608167:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe536873398d lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1afe3 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619904.608200:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe536873398d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1afe3 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619904.608216:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c200, rc 0) 00000100:00100000:2.0:1788619904.608983:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e6479c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571296640:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6538us (8308us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619904.609009:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 469 00000100:00100000:2.0:1788619904.618204:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619904.619602:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571296768 00000100:00100000:0.0:1788619904.619622:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571296768 00000100:00100000:1.0:1788619904.620137:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 470 00000100:00100000:1.0:1788619904.620174:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571296768:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619904.620200:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619904.620227:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000262acbad/0x1cd7fe5368733994 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1afea expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619904.620955:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e647a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589034368:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619904.621144:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd9636c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619904.625502:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e647a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589034368:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619904.625563:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000262acbad/0x1cd7fe5368733994 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1afea expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619904.625595:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000262acbad/0x1cd7fe5368733994 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1afea expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619904.625613:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b50200, rc 0) 00000100:00100000:2.0:1788619904.625726:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571296768:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5554us (7537us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619904.625738:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 470 00000100:00100000:2.0:1788619904.634785:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619904.634916:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571296896 00000100:00100000:1.0:1788619904.634930:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571296896 00002000:00100000:1.0:1788619904.635013:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fefd7100 x1875503571296896/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/0 e 0 to 0 dl 1788619915 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619904.635027:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fefd7100 with x1875503571296896 00000100:00100000:1.0:1788619904.635068:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 329 00000100:00100000:1.0:1788619904.635078:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571296896:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619904.635499:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571296896:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 420us (729us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619904.635508:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 329 00002000:00100000:1.0:1788619904.635516:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fefd7100 x1875503571296896/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/440 e 0 to 0 dl 1788619915 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619904.635524:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fefd7100 with x1875503571296896 00000100:00100000:0.0:1788619904.639312:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619904.639940:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571297024 00000100:00100000:1.0:1788619904.639956:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571297024 00002000:00100000:1.0:1788619904.640045:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5770380 x1875503571297024/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/0 e 0 to 0 dl 1788619915 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619904.640059:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5770380 with x1875503571297024 00000100:00100000:1.0:1788619904.640131:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 330 00000100:00100000:1.0:1788619904.640144:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5770380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571297024:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619904.640583:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5770380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571297024:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 439us (1284us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619904.640593:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 330 00002000:00100000:1.0:1788619904.640601:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5770380 x1875503571297024/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/440 e 0 to 0 dl 1788619915 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619904.640610:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5770380 with x1875503571297024 00000100:00100000:2.0:1788619904.644665:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619904.645156:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571297152 00000100:00100000:3.0:1788619904.645170:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571297152 00002000:00100000:3.0:1788619904.645257:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fefd4a80 x1875503571297152/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/0 e 0 to 0 dl 1788619915 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619904.645280:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fefd4a80 with x1875503571297152 00000100:00100000:3.0:1788619904.645383:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 331 00000100:00100000:3.0:1788619904.645416:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571297152:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619904.645799:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571297152:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 383us (1149us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619904.645808:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 331 00002000:00100000:3.0:1788619904.645816:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fefd4a80 x1875503571297152/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/440 e 0 to 0 dl 1788619915 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619904.646247:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fefd4a80 with x1875503571297152 00000100:00100000:3.0:1788619904.664778:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619904.665085:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571297280 00000100:00100000:2.0:1788619904.665100:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571297280 00002000:00100000:2.0:1788619904.665179:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c75f5500 x1875503571297280/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/0 e 0 to 0 dl 1788619915 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619904.665193:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c75f5500 with x1875503571297280 00000100:00100000:2.0:1788619904.665242:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 332 00000100:00100000:2.0:1788619904.665254:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571297280:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619904.665699:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571297280:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 445us (935us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619904.665709:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 332 00002000:00100000:2.0:1788619904.665718:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c75f5500 x1875503571297280/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/440 e 0 to 0 dl 1788619915 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619904.665727:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c75f5500 with x1875503571297280 00000100:00100000:0.0:1788619904.671581:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619904.671741:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571297408 00000100:00100000:1.0:1788619904.671755:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571297408 00002000:00100000:1.0:1788619904.671838:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5773800 x1875503571297408/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/0 e 0 to 0 dl 1788619915 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619904.671852:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5773800 with x1875503571297408 00000100:00100000:1.0:1788619904.671900:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 333 00000100:00100000:1.0:1788619904.671911:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5773800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571297408:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619904.672313:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5773800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571297408:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 400us (743us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619904.672322:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 333 00002000:00100000:1.0:1788619904.672382:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5773800 x1875503571297408/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/440 e 0 to 0 dl 1788619915 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619904.672392:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5773800 with x1875503571297408 00000100:00100000:1.0:1788619904.676786:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619904.676943:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571297536 00000100:00100000:3.0:1788619904.676957:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571297536 00002000:00100000:3.0:1788619904.677038:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e647a300 x1875503571297536/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/0 e 0 to 0 dl 1788619915 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619904.677052:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e647a300 with x1875503571297536 00000100:00100000:3.0:1788619904.677094:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 334 00000100:00100000:3.0:1788619904.677105:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e647a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571297536:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619904.677967:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e647a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571297536:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 856us (1190us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619904.677979:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 334 00002000:00100000:3.0:1788619904.677988:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e647a300 x1875503571297536/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/440 e 0 to 0 dl 1788619915 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619904.678011:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e647a300 with x1875503571297536 00000100:00100000:3.0:1788619904.683391:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619904.683587:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571297664 00000100:00100000:2.0:1788619904.683601:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571297664 00002000:00100000:2.0:1788619904.683674:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c75f6a00 x1875503571297664/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/0 e 0 to 0 dl 1788619915 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619904.683688:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c75f6a00 with x1875503571297664 00000100:00100000:2.0:1788619904.683730:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 335 00000100:00100000:2.0:1788619904.683742:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571297664:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619904.684102:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571297664:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 360us (724us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619904.684109:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 335 00002000:00100000:2.0:1788619904.684116:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c75f6a00 x1875503571297664/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/440 e 0 to 0 dl 1788619915 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619904.684123:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c75f6a00 with x1875503571297664 00000100:00100000:0.0:1788619904.699314:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619904.699549:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571297792 00000100:00100000:1.0:1788619904.699563:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571297792 00002000:00100000:1.0:1788619904.699635:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5772d80 x1875503571297792/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/0 e 0 to 0 dl 1788619915 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619904.699665:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5772d80 with x1875503571297792 00000100:00100000:1.0:1788619904.699712:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 336 00000100:00100000:1.0:1788619904.699724:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5772d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571297792:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619904.700221:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5772d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571297792:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 496us (919us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619904.700231:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 336 00002000:00100000:1.0:1788619904.700239:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5772d80 x1875503571297792/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/440 e 0 to 0 dl 1788619915 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619904.700248:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5772d80 with x1875503571297792 00000100:00100000:1.0:1788619904.703895:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619904.704046:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571297920 00000100:00100000:3.0:1788619904.704060:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571297920 00002000:00100000:3.0:1788619904.704132:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df96a300 x1875503571297920/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/0 e 0 to 0 dl 1788619915 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619904.704168:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df96a300 with x1875503571297920 00000100:00100000:3.0:1788619904.704217:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 337 00000100:00100000:3.0:1788619904.704229:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df96a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571297920:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619904.704971:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df96a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571297920:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 740us (1088us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619904.704982:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 337 00002000:00100000:3.0:1788619904.704991:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df96a300 x1875503571297920/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/440 e 0 to 0 dl 1788619915 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619904.705012:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df96a300 with x1875503571297920 00000100:00100000:2.0:1788619904.713155:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619904.713443:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571298048 00000100:00100000:1.0:1788619904.713459:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571298048 00002000:00100000:1.0:1788619904.713536:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fefd6a00 x1875503571298048/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/0 e 0 to 0 dl 1788619915 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619904.713548:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fefd6a00 with x1875503571298048 00000100:00100000:1.0:1788619904.713591:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 338 00000100:00100000:1.0:1788619904.713602:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571298048:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619904.713970:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571298048:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 368us (832us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619904.713979:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 338 00002000:00100000:1.0:1788619904.713987:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fefd6a00 x1875503571298048/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/440 e 0 to 0 dl 1788619915 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619904.713994:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fefd6a00 with x1875503571298048 00000100:00100000:0.0:1788619904.723031:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619904.723114:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571298176 00000100:00100000:3.0:1788619904.723128:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571298176 00002000:00100000:3.0:1788619904.723190:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5026a00 x1875503571298176/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/0 e 0 to 0 dl 1788619915 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619904.723204:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5026a00 with x1875503571298176 00000100:00100000:3.0:1788619904.723235:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 339 00000100:00100000:3.0:1788619904.723254:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5026a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571298176:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619904.723622:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5026a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571298176:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 376us (602us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619904.723631:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 339 00002000:00100000:3.0:1788619904.723638:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5026a00 x1875503571298176/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/440 e 0 to 0 dl 1788619915 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619904.723646:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5026a00 with x1875503571298176 00000100:00100000:3.0:1788619904.733687:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619904.733853:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571298304 00000100:00100000:2.0:1788619904.733867:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571298304 00002000:00100000:2.0:1788619904.733938:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c75f7800 x1875503571298304/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/0 e 0 to 0 dl 1788619915 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619904.733950:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c75f7800 with x1875503571298304 00000100:00100000:2.0:1788619904.734007:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 340 00000100:00100000:2.0:1788619904.734020:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571298304:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619904.734440:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571298304:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 420us (767us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619904.734450:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 340 00002000:00100000:2.0:1788619904.734459:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c75f7800 x1875503571298304/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/440 e 0 to 0 dl 1788619915 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619904.734468:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c75f7800 with x1875503571298304 00000100:00100000:2.0:1788619904.742044:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619904.742210:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571298432 00000100:00100000:1.0:1788619904.742224:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571298432 00002000:00100000:1.0:1788619904.742451:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fefd6680 x1875503571298432/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/0 e 0 to 0 dl 1788619915 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619904.742463:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fefd6680 with x1875503571298432 00000100:00100000:1.0:1788619904.742501:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 341 00000100:00100000:1.0:1788619904.742513:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571298432:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619904.743492:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571298432:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 974us (1455us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619904.743505:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 341 00002000:00100000:1.0:1788619904.743515:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fefd6680 x1875503571298432/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/440 e 0 to 0 dl 1788619915 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619904.743525:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fefd6680 with x1875503571298432 00000100:00100000:0.0:1788619904.748859:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619904.749571:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571298560 00000100:00100000:2.0:1788619904.749587:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571298560 00000100:00100000:2.0:1788619904.749671:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 471 00000100:00100000:2.0:1788619904.749684:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5027b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571298560:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619904.749727:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619904.749752:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cac82bb5/0x1cd7fe536873399b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1aff1 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619904.750152:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fefd5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589034496:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619904.750676:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619904.754674:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fefd5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589034496:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619904.754733:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000cac82bb5/0x1cd7fe536873399b lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aff1 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619904.754768:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cac82bb5/0x1cd7fe536873399b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aff1 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619904.754786:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505da00, rc 0) 00000100:00100000:2.0:1788619904.754871:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5027b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571298560:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5189us (6042us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619904.754883:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 471 00000100:00100000:3.0:1788619904.758957:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619904.759125:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571298688 00000100:00100000:2.0:1788619904.759138:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571298688 00000100:00100000:2.0:1788619904.759219:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 472 00000100:00100000:2.0:1788619904.759232:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571298688:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619904.759272:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619904.759296:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c22c48a7/0x1cd7fe53687339a2 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1aff8 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619904.759448:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fefd5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589034624:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619904.759563:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619904.763879:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fefd5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589034624:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619904.763951:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c22c48a7/0x1cd7fe53687339a2 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aff8 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619904.763984:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c22c48a7/0x1cd7fe53687339a2 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1aff8 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619904.764009:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505d400, rc 0) 00000100:00100000:2.0:1788619904.764088:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571298688:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4858us (5146us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619904.764099:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 472 00000100:00100000:3.0:1788619904.768625:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619904.768787:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571298816 00000100:00100000:1.0:1788619904.768801:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571298816 00000100:00100000:1.0:1788619904.768877:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 473 00000100:00100000:1.0:1788619904.768888:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe139050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571298816:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619904.768921:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619904.768947:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a7bead88/0x1cd7fe53687339a9 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1afff expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619904.769061:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df969f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589034752:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619904.769160:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963f00 going to sleep for 11 seconds 00000100:00100000:1.0:1788619904.772401:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df969f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589034752:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619904.772467:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a7bead88/0x1cd7fe53687339a9 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1afff expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619904.772533:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a7bead88/0x1cd7fe53687339a9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1afff expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619904.772551:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b51000, rc 0) 00000100:00100000:1.0:1788619904.772636:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe139050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571298816:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3749us (4029us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619904.772647:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 473 00000100:00100000:0.0:1788619904.778171:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619904.778281:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571298944 00000100:00100000:1.0:1788619904.778294:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571298944 00000100:00100000:1.0:1788619904.778416:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 474 00000100:00100000:1.0:1788619904.778429:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5026680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571298944:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619904.778453:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619904.778476:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a7bead88/0x1cd7fe53687339b0 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b006 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619904.778557:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df969f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589034880:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619904.778673:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963f00 going to sleep for 11 seconds 00000100:00100000:1.0:1788619904.789674:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df969f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589034880:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619904.789735:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a7bead88/0x1cd7fe53687339b0 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b006 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619904.789770:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a7bead88/0x1cd7fe53687339b0 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b006 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619904.789818:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b51000, rc 0) 00000100:00100000:1.0:1788619904.790125:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5026680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571298944:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 11696us (11966us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619904.790137:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 474 00000100:00100000:2.0:1788619904.800018:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619904.800288:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571299072 00000100:00100000:0.0:1788619904.800302:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571299072 00000100:00100000:0.0:1788619904.800408:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 475 00000100:00100000:0.0:1788619904.800420:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:14908:x1875503571299072:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619904.800444:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619904.800468:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d5e4548d/0x1cd7fe53687339b7 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b00d expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619904.800511:0:7654:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000c461731d/0x1cd7fe53687337f0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae54 expref: 9 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619904.800522:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=1s v=(1 1) 00010000:00010000:0.0:1788619904.800527:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 1) 00010000:00010000:0.0:1788619904.800534:0:7654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5e4548d/0x1cd7fe53687339b7 lrc: 4/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1576960->1593343] (req 1576960->1593343) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b00d expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619904.800576:0:7654:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000c461731d/0x1cd7fe53687337f0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae54 expref: 9 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619904.800616:0:7654:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000c461731d/0x1cd7fe53687337f0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae54 expref: 9 pid: 7654 timeout: 497 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619904.800632:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5027800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589035008:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619904.800742:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaacc0 going to sleep for 11 seconds 00010000:00010000:3.0:1788619904.806128:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788619904.807460:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5027800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589035008:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619904.807519:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d5e4548d/0x1cd7fe53687339b7 lrc: 3/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1576960->1593343] (req 1576960->1593343) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b00d expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619904.807550:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c4b4ae00, rc 0) 00000100:00100000:2.0:1788619904.807651:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+6:14908:x1875503571299072:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7231us (7655us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619904.807663:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 475 00000100:00100000:3.0:1788619904.809081:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619904.809315:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571299200 00000100:00100000:2.0:1788619904.809435:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571299200 00010000:00010000:2.0:1788619904.809488:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000c461731d/0x1cd7fe53687337f0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae54 expref: 10 pid: 7654 timeout: 497 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619904.809576:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 211 00000100:00100000:2.0:1788619904.809589:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f4e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571299200:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619904.809607:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619904.809639:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000c461731d/0x1cd7fe53687337f0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae54 expref: 10 pid: 7654 timeout: 497 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619904.809655:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000c461731d/0x1cd7fe53687337f0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae54 expref: 10 pid: 7654 timeout: 497 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619904.809692:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c461731d/0x1cd7fe53687337f0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae54 expref: 10 pid: 7654 timeout: 497 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619904.809712:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 1) 00010000:00010000:2.0:1788619904.809717:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 1) 00010000:00010000:2.0:1788619904.809728:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000d5e4548d/0x1cd7fe53687339b7 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1576960->1593343) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b00d expref: 5 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619904.809740:0:5831:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d5e4548d/0x1cd7fe53687339b7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1576960->1593343) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b00d expref: 5 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619904.809789:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000d5e4548d/0x1cd7fe53687339b7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1576960->1593343) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b00d expref: 5 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619904.809821:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fefd4700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5831:1875503589035136:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619904.815777:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619904.819218:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fefd4700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5831:1875503589035136:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619904.819280:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619904.819391:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f4e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571299200:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 9803us (10318us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619904.819405:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 211 00000100:00100000:1.0:1788619904.833556:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619904.833615:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619904.833708:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571299456 00000100:00100000:3.0:1788619904.833724:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571299456 00002000:00100000:3.0:1788619904.833794:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df96b100 x1875503571299456/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:0/0 lens 488/0 e 0 to 0 dl 1788619915 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619904.833807:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df96b100 with x1875503571299456 00000100:00100000:3.0:1788619904.833868:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 342 00000100:00100000:3.0:1788619904.833879:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df96b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571299456:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619904.836353:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571299328 00000100:00100000:0.0:1788619904.836374:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571299328 00002000:00100000:0.0:1788619904.836443:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1785855) req@ffff8e78df969f80 x1875503571299328/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:0/0 lens 488/0 e 0 to 0 dl 1788619915 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619904.836456:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df969f80 with x1875503571299328 ext(1048576->1785855) 00002000:00010000:0.0:1788619904.836463:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df969f80 00000100:00100000:0.0:1788619904.836509:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 250 00000100:00100000:0.0:1788619904.836521:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df969f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571299328:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619904.868691:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df96b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571299456:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 34796us (35058us total) trans 0 rc 352256/352256 00000100:00100000:3.0:1788619904.868709:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 342 00002000:00100000:3.0:1788619904.868725:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df96b100 x1875503571299456/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:0/0 lens 488/440 e 0 to 0 dl 1788619915 ref 1 fl Complete:/600/0 rc 352256/352256 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619904.868738:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df96b100 with x1875503571299456 00000100:00100000:0.0:1788619904.890430:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df969f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571299328:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 53888us (56868us total) trans 0 rc 737280/737280 00000100:00100000:0.0:1788619904.890449:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 250 00002000:00100000:0.0:1788619904.890467:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1785855) req@ffff8e78df969f80 x1875503571299328/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:0/0 lens 488/440 e 0 to 0 dl 1788619915 ref 1 fl Complete:/600/0 rc 737280/737280 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619904.890480:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df969f80 with x1875503571299328 ext(1048576->1785855) 00002000:00010000:0.0:1788619904.890489:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df969f80 00000100:00100000:0.0:1788619904.900909:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619904.901129:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571299584 00000100:00100000:1.0:1788619904.901143:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571299584 00002000:00100000:1.0:1788619904.901214:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5024a80 x1875503571299584/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 504/0 e 0 to 0 dl 1788619915 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619904.901227:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5024a80 with x1875503571299584 00000100:00100000:1.0:1788619904.901273:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 343 00000100:00100000:1.0:1788619904.901284:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5024a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571299584:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619904.916747:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5024a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571299584:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 15446us (15835us total) trans 0 rc 303104/303104 00000100:00100000:1.0:1788619904.916767:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 343 00002000:00100000:1.0:1788619904.916784:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5024a80 x1875503571299584/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 504/440 e 0 to 0 dl 1788619915 ref 1 fl Complete:/600/0 rc 303104/303104 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619904.916797:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5024a80 with x1875503571299584 00000100:00100000:0.0:1788619904.936746:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619904.936957:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571299712 00000100:00100000:2.0:1788619904.936969:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571299712 00000100:00100000:2.0:1788619904.937061:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 476 00000100:00100000:2.0:1788619904.937075:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5025180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571299712:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619904.937100:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619904.937125:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c22c48a7/0x1cd7fe53687339be lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b014 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619904.937174:0:6698:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000d5e4548d/0x1cd7fe53687339b7 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1576960->1593343) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b00d expref: 5 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619904.937185:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 1) 00010000:00010000:2.0:1788619904.937189:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 1) 00010000:00010000:2.0:1788619904.937195:0:6698:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c22c48a7/0x1cd7fe53687339be lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1646592->1691647] (req 1646592->1691647) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b014 expref: 10 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619904.937239:0:6698:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000d5e4548d/0x1cd7fe53687339b7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1576960->1593343) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b00d expref: 5 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619904.941793:0:6698:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000d5e4548d/0x1cd7fe53687339b7 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1576960->1593343) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b00d expref: 5 pid: 7654 timeout: 497 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619904.941845:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fefd4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589035264:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619904.942028:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619904.944526:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fefd4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589035264:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619904.944581:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c22c48a7/0x1cd7fe53687339be lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1646592->1691647] (req 1646592->1691647) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b014 expref: 10 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619904.944611:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505d400, rc 0) 00000100:00100000:2.0:1788619904.944729:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5025180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:14908:x1875503571299712:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7657us (7995us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619904.944738:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 476 00000100:00100000:3.0:1788619904.956439:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619904.956725:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571299840 00000100:00100000:1.0:1788619904.956741:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571299840 00010000:00010000:1.0:1788619904.956796:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000d5e4548d/0x1cd7fe53687339b7 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1576960->1593343) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b00d expref: 6 pid: 7654 timeout: 497 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619904.956884:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 212 00000100:00100000:1.0:1788619904.956895:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f4000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571299840:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619904.956915:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619904.956944:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d5e4548d/0x1cd7fe53687339b7 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1576960->1593343) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b00d expref: 6 pid: 7654 timeout: 497 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619904.956959:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d5e4548d/0x1cd7fe53687339b7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1576960->1593343) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b00d expref: 6 pid: 7654 timeout: 497 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619904.957006:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d5e4548d/0x1cd7fe53687339b7 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1576960->1593343) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b00d expref: 6 pid: 7654 timeout: 497 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619904.957027:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 1) 00010000:00010000:1.0:1788619904.957032:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 1) 00010000:00010000:1.0:1788619904.957042:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000c22c48a7/0x1cd7fe53687339be lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1646592->1691647) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b014 expref: 9 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619904.957052:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c22c48a7/0x1cd7fe53687339be lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1646592->1691647) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b014 expref: 9 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619904.957101:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000c22c48a7/0x1cd7fe53687339be lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1646592->1691647) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b014 expref: 9 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619904.957118:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df968700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589035392:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619904.957607:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963180 going to sleep for 11 seconds 00000100:00100000:1.0:1788619904.969738:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df968700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589035392:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619904.969783:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619904.969959:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f4000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571299840:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 13063us (13535us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619904.969973:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 212 00000100:00100000:3.0:1788619904.977168:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619904.977234:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619904.977406:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571300096 00000100:00100000:1.0:1788619904.977423:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571300096 00002000:00100000:1.0:1788619904.977498:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e67f1c00 x1875503571300096/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 504/0 e 0 to 0 dl 1788619915 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619904.977631:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e67f1c00 with x1875503571300096 00000100:00100000:1.0:1788619904.977698:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 344 00000100:00100000:1.0:1788619904.977711:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571300096:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619904.978681:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571299968 00000100:00100000:0.0:1788619904.978694:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571299968 00002000:00100000:0.0:1788619904.978744:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1785855) req@ffff8e78e67f1880 x1875503571299968/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/0 e 0 to 0 dl 1788619915 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619904.978757:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78e67f1880 with x1875503571299968 ext(1048576->1785855) 00002000:00010000:0.0:1788619904.978763:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78e67f1880 00000100:00100000:0.0:1788619904.978807:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 251 00000100:00100000:0.0:1788619904.978817:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571299968:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619904.992667:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571300096:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 14926us (15399us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619904.992686:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 344 00002000:00100000:2.0:1788619904.992705:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e67f1c00 x1875503571300096/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 504/440 e 0 to 0 dl 1788619915 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619904.992717:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e67f1c00 with x1875503571300096 00000100:00100000:0.0:1788619905.035075:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571299968:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 56243us (57906us total) trans 0 rc 737280/737280 00000100:00100000:0.0:1788619905.035094:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 251 00002000:00100000:0.0:1788619905.035111:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1785855) req@ffff8e78e67f1880 x1875503571299968/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:0/0 lens 488/440 e 0 to 0 dl 1788619915 ref 1 fl Complete:/600/0 rc 737280/737280 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619905.035122:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78e67f1880 with x1875503571299968 ext(1048576->1785855) 00002000:00010000:0.0:1788619905.035129:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78e67f1880 00000100:00100000:3.0:1788619905.049232:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619905.051054:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571300224 00000100:00100000:1.0:1788619905.051074:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571300224 00000100:00100000:1.0:1788619905.051168:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 477 00000100:00100000:1.0:1788619905.051183:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571300224:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619905.051213:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619905.051243:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000199d2a35/0x1cd7fe53687339c5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b01b expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619905.051358:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df968700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589035520:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619905.051488:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963180 going to sleep for 11 seconds 00000100:00100000:1.0:1788619905.055208:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df968700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589035520:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619905.055262:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000199d2a35/0x1cd7fe53687339c5 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b01b expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619905.055310:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000199d2a35/0x1cd7fe53687339c5 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b01b expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619905.055367:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5023800, rc 0) 00000100:00100000:1.0:1788619905.055438:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571300224:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4259us (6451us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619905.055448:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 477 00000100:00100000:3.0:1788619905.059909:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619905.063127:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571300352 00000100:00100000:1.0:1788619905.063150:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571300352 00000100:00100000:0.0:1788619905.065443:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 478 00000100:00100000:0.0:1788619905.065476:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571300352:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619905.065504:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619905.065543:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004402d61a/0x1cd7fe53687339cc lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b022 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619905.065633:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5024380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589035648:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619905.065766:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa300 going to sleep for 11 seconds 00000100:00100000:0.0:1788619905.072457:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5024380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589035648:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619905.072595:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004402d61a/0x1cd7fe53687339cc lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b022 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.072626:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004402d61a/0x1cd7fe53687339cc lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b022 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.072642:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c0031600, rc 0) 00000100:00100000:0.0:1788619905.072724:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571300352:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7264us (12829us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619905.072735:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 478 00000100:00100000:3.0:1788619905.077248:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619905.078710:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571300480 00000100:00100000:2.0:1788619905.078729:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571300480 00000100:00100000:2.0:1788619905.078860:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 479 00000100:00100000:2.0:1788619905.078873:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571300480:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619905.078919:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619905.078945:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c461731d/0x1cd7fe53687339d3 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b029 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619905.079048:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fefd4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589035776:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619905.079173:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619905.090465:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fefd4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589035776:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619905.090537:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c461731d/0x1cd7fe53687339d3 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b029 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.090576:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c461731d/0x1cd7fe53687339d3 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b029 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.090593:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505cc00, rc 0) 00000100:00100000:2.0:1788619905.090681:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571300480:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 11809us (13450us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619905.090692:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 479 00000100:00100000:3.0:1788619905.100971:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619905.101106:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571300608 00000100:00100000:1.0:1788619905.101119:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571300608 00000100:00100000:1.0:1788619905.101194:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 480 00000100:00100000:1.0:1788619905.101206:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571300608:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619905.101230:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619905.101253:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000199d2a35/0x1cd7fe53687339da lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b030 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619905.101408:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df968700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589035904:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619905.101523:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963180 going to sleep for 11 seconds 00000100:00100000:1.0:1788619905.106394:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df968700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589035904:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619905.106450:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000199d2a35/0x1cd7fe53687339da lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b030 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619905.106480:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000199d2a35/0x1cd7fe53687339da lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b030 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619905.106516:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5023800, rc 0) 00000100:00100000:1.0:1788619905.106591:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571300608:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5386us (5785us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619905.106602:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 480 00000100:00100000:3.0:1788619905.111665:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619905.111793:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571300736 00000100:00100000:1.0:1788619905.111806:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571300736 00000100:00100000:1.0:1788619905.111899:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 481 00000100:00100000:1.0:1788619905.111912:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571300736:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619905.111937:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619905.111964:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2924e8d/0x1cd7fe53687339e1 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b037 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619905.112062:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df968700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589036032:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619905.112173:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963180 going to sleep for 11 seconds 00000100:00100000:1.0:1788619905.115098:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df968700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589036032:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619905.115157:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a2924e8d/0x1cd7fe53687339e1 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b037 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619905.115190:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2924e8d/0x1cd7fe53687339e1 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b037 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619905.115207:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5023c00, rc 0) 00000100:00100000:1.0:1788619905.115295:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571300736:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3385us (3646us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619905.115381:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 481 00000100:00100000:2.0:1788619905.122717:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df968700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589036160:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1788619905.122822:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619905.123050:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589036160 00000100:00100000:1.0:1788619905.123061:0:6699:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589036160 00000100:00100000:1.0:1788619905.123189:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 83 00000100:00100000:1.0:1788619905.123201:0:6699:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589036160:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1788619905.123230:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 80019456 pending 1703936 free 3859808256 avail 3857711104 00000100:00100000:1.0:1788619905.123313:0:6699:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589036160:12345-0@lo:13:osp-pre-0-0.0 Request processed in 115us (491us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619905.123320:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 83 00000100:00100000:3.0:1788619905.123453:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df968700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589036160:0@lo:13:osp-pre-0-0.0 00000100:00080000:0.0:1788619905.123606:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1788619905.126756:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619905.127013:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571300864 00000100:00100000:0.0:1788619905.127027:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571300864 00000100:00100000:0.0:1788619905.127105:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 482 00000100:00100000:0.0:1788619905.127125:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571300864:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619905.127149:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619905.127173:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004402d61a/0x1cd7fe53687339e8 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b03e expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619905.127269:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5024380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589036288:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619905.134483:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa300 going to sleep for 11 seconds 00000100:00100000:0.0:1788619905.144123:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5024380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589036288:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619905.144185:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004402d61a/0x1cd7fe53687339e8 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b03e expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.144245:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004402d61a/0x1cd7fe53687339e8 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b03e expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.144276:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c0031600, rc 0) 00000100:00100000:0.0:1788619905.144419:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571300864:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 17303us (17677us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619905.144430:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 482 00000100:00100000:3.0:1788619905.153815:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619905.153964:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571300992 00000100:00100000:1.0:1788619905.153978:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571300992 00000100:00100000:1.0:1788619905.154073:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 649 00000100:00100000:1.0:1788619905.154086:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571300992:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619905.154117:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619905.154121:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:1.0:1788619905.154185:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4a000/0x1cd7fe5368733986 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1afdc expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619905.154214:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619905.154222:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788619905.154243:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5023c00/0x1cd7fe53687339ef lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b045 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619905.154290:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51a00/0x1cd7fe536873397f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1afd5 expref: 9 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619905.154394:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5023c00/0x1cd7fe53687339ef lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b045 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619905.154443:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51a00/0x1cd7fe536873397f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1afd5 expref: 9 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619905.154459:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51a00/0x1cd7fe536873397f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1afd5 expref: 9 pid: 9417 timeout: 497 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619905.154476:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df968000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589036416:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619905.154603:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963180 going to sleep for 11 seconds 00000100:00100000:1.0:1788619905.159294:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df968000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589036416:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619905.159393:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5023c00/0x1cd7fe53687339ef lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b045 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619905.159414:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619905.159417:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619905.159421:0:5845:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5023c00/0x1cd7fe53687339ef lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b045 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619905.159434:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5023c00, rc 0) 00000100:00100000:1.0:1788619905.159510:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571300992:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5426us (5710us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619905.159521:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 649 00000100:00100000:3.0:1788619905.170693:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619905.170800:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571301120 00000100:00100000:0.0:1788619905.170820:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571301120 00010000:00010000:0.0:1788619905.170871:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51a00/0x1cd7fe536873397f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1afd5 expref: 10 pid: 9417 timeout: 497 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619905.170912:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 213 00000100:00100000:0.0:1788619905.170924:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c0ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571301120:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619905.170946:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619905.170980:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51a00/0x1cd7fe536873397f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1afd5 expref: 10 pid: 9417 timeout: 497 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.171025:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51a00/0x1cd7fe536873397f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1afd5 expref: 10 pid: 9417 timeout: 497 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.171056:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b51a00/0x1cd7fe536873397f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1afd5 expref: 10 pid: 9417 timeout: 497 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.171083:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619905.171112:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5023c00/0x1cd7fe53687339ef lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b045 expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.171126:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5023c00/0x1cd7fe53687339ef lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b045 expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.171137:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5023c00/0x1cd7fe53687339ef lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b045 expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.171189:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5023c00/0x1cd7fe53687339ef lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b045 expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619905.171206:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5024380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589036544:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619905.171955:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa300 going to sleep for 11 seconds 00000100:00100000:0.0:1788619905.175147:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5024380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589036544:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619905.175194:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619905.175273:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c0ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571301120:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 4350us (4591us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619905.175287:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 213 00000100:00100000:3.0:1788619905.177091:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619905.177164:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571301248 00000100:00100000:1.0:1788619905.177190:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571301248 00000100:00100000:1.0:1788619905.177271:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 345 00000100:00100000:1.0:1788619905.177280:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c0a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571301248:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:1.0:1788619905.177699:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 352256 -> 352256 00000004:00010000:1.0:1788619905.177704:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619904 -> 1788619904 00000004:00010000:1.0:1788619905.177707:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619904 -> 1788619904 00000004:00010000:1.0:1788619905.177709:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619904 -> 1788619904 00000004:00010000:1.0:1788619905.177711:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 774 -> 774 00000100:00100000:1.0:1788619905.177798:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c0a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571301248:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 515us (893us total) trans 4294967555 rc 0/0 00000100:00100000:1.0:1788619905.177806:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 345 00000100:00100000:3.0:1788619905.181217:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619905.181281:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571301376 00000100:00100000:2.0:1788619905.181296:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571301376 00000100:00100000:2.0:1788619905.181454:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 483 00000100:00100000:2.0:1788619905.181467:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c0b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571301376:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619905.181491:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619905.181684:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c461731d/0x1cd7fe53687339f6 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b04c expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619905.181747:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fefd4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589036672:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619905.182364:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619905.185186:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fefd4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589036672:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619905.185245:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c461731d/0x1cd7fe53687339f6 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b04c expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.185277:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c461731d/0x1cd7fe53687339f6 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b04c expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.185296:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505cc00, rc 0) 00000100:00100000:2.0:1788619905.185458:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c0b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571301376:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3993us (4252us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619905.185469:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 483 00000100:00100000:3.0:1788619905.191250:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619905.191548:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571301504 00000100:00100000:1.0:1788619905.191559:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571301504 00000100:00100000:1.0:1788619905.191623:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 484 00000100:00100000:1.0:1788619905.191635:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c09500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571301504:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619905.191659:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619905.191683:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000199d2a35/0x1cd7fe53687339fd lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b053 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619905.191761:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df968000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589036800:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619905.191867:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963180 going to sleep for 11 seconds 00000100:00100000:1.0:1788619905.193778:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df968000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589036800:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619905.193822:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000199d2a35/0x1cd7fe53687339fd lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b053 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619905.194140:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000199d2a35/0x1cd7fe53687339fd lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b053 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619905.194158:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5023800, rc 0) 00000100:00100000:1.0:1788619905.194235:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c09500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571301504:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2601us (2997us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619905.194246:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 484 00000100:00100000:0.0:1788619905.200216:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619905.200313:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571301632 00000100:00100000:1.0:1788619905.200353:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571301632 00000100:00100000:1.0:1788619905.200421:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 650 00000100:00100000:1.0:1788619905.200432:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5024380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571301632:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619905.200462:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619905.200489:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5023800/0x1cd7fe5368733a04 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b05a expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619905.200538:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5023c00/0x1cd7fe53687339ef lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b045 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619905.200591:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df968000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589036928:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619905.200812:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963180 going to sleep for 11 seconds 00000100:00100000:1.0:1788619905.205162:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df968000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589036928:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619905.205199:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 352256 00010000:00010000:1.0:1788619905.205224:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5023800/0x1cd7fe5368733a04 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b05a expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619905.208472:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619905.208487:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619905.208505:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5023800/0x1cd7fe5368733a04 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b05a expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619905.208534:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5023800, rc 0) 00000100:00100000:1.0:1788619905.208720:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5024380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571301632:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8287us (8518us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619905.208731:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 650 00000100:00100000:3.0:1788619905.212663:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619905.215673:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571301760 00000100:00100000:0.0:1788619905.215696:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571301760 00000100:00100000:0.0:1788619905.215788:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 651 00000100:00100000:0.0:1788619905.215800:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c0b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571301760:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619905.215828:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619905.215856:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c0030400/0x1cd7fe5368733a0b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b061 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.215932:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5023c00/0x1cd7fe53687339ef lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b045 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619905.215987:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5025f80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589037056:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619905.216312:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa300 going to sleep for 11 seconds 00000100:00100000:2.0:1788619905.234066:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5025f80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589037056:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619905.234118:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 352256 00010000:00010000:2.0:1788619905.234151:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c0030400/0x1cd7fe5368733a0b lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b061 expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.234180:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619905.234184:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619905.234191:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c0030400/0x1cd7fe5368733a0b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b061 expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.234247:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c0030400, rc 0) 00000100:00100000:2.0:1788619905.235809:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c0b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571301760:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 19996us (23149us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619905.236375:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 651 00000100:00100000:2.0:1788619905.253755:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619905.254244:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571302016 00000100:00100000:1.0:1788619905.254269:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571302016 00000100:00100000:1.0:1788619905.254400:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 652 00000100:00100000:1.0:1788619905.254412:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571302016:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:1.0:1788619905.254493:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619905.254527:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4a200/0x1cd7fe5368733828 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae7e expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619905.254553:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619905.254559:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788619905.255429:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5023800/0x1cd7fe5368733a12 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619905.255453:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4b600/0x1cd7fe53687337fe lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae5b expref: 8 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619905.255462:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5023800/0x1cd7fe5368733a12 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619905.255540:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4b600/0x1cd7fe53687337fe lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae5b expref: 8 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619905.255560:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4b600/0x1cd7fe53687337fe lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae5b expref: 8 pid: 5845 timeout: 497 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619905.255579:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df968000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589037184:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619905.255713:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963180 going to sleep for 11 seconds 00000100:00100000:3.0:1788619905.260650:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619905.264620:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df968000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589037184:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619905.264665:0:5845:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5023800/0x1cd7fe5368733a12 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619905.273830:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571302144 00000100:00100000:2.0:1788619905.273856:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571302144 00010000:00010000:2.0:1788619905.273939:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4b600/0x1cd7fe53687337fe lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae5b expref: 9 pid: 5845 timeout: 497 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619905.274112:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 214 00000100:00100000:2.0:1788619905.274124:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c09f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571302144:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619905.274144:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619905.274153:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4b600/0x1cd7fe53687337fe lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae5b expref: 9 pid: 5845 timeout: 497 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.274193:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4b600/0x1cd7fe53687337fe lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae5b expref: 9 pid: 5845 timeout: 497 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.274223:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c4b4b600/0x1cd7fe53687337fe lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ae5b expref: 9 pid: 5845 timeout: 497 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.274247:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619905.280051:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5023800/0x1cd7fe5368733a12 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.280120:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5023800/0x1cd7fe5368733a12 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.280129:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5023800/0x1cd7fe5368733a12 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.280196:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619905.280314:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c09f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+8:2354:x1875503571302144:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 6184us (19672us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619905.280344:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 214 00010000:00010000:0.0:1788619905.281443:0:5845:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5023800/0x1cd7fe5368733a12 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.281461:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5023800/0x1cd7fe5368733a12 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788619905.281632:0:5845:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967556 00000004:00010000:0.0:1788619905.281696:0:5845:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 352256 -> 352256 00000004:00010000:0.0:1788619905.281700:0:5845:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619904 -> 1788619905 00000004:00010000:0.0:1788619905.281703:0:5845:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619904 -> 1788619904 00000004:00010000:0.0:1788619905.281705:0:5845:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619904 -> 1788619905 00000004:00010000:0.0:1788619905.281708:0:5845:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 774 -> 774 00000004:00010000:0.0:1788619905.281716:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000000cd76450 reply state 00000000802ffbeb transno 4294967557 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5023800/0x1cd7fe5368733a12 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619905.281726:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788619905.281872:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:14908:x1875503571302016:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 27459us (28127us total) trans 4294967557 rc 0/0 00000100:00100000:0.0:1788619905.281883:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 652 00000020:00080000:0.0:1788619905.281889:0:5845:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1788619905.286262:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619905.286431:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571302272 00000100:00100000:3.0:1788619905.286448:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571302272 00000100:00100000:3.0:1788619905.286548:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 485 00000100:00100000:3.0:1788619905.286563:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cd3c9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:14908:x1875503571302272:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619905.286594:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619905.286623:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062c34eb6/0x1cd7fe5368733a19 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b068 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619905.286670:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000c22c48a7/0x1cd7fe53687339be lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1646592->1691647) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b014 expref: 9 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619905.286682:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 1) 00010000:00010000:3.0:1788619905.286688:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(3 1) 00010000:00010000:3.0:1788619905.286695:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062c34eb6/0x1cd7fe5368733a19 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b068 expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619905.286749:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000c22c48a7/0x1cd7fe53687339be lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1646592->1691647) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b014 expref: 9 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619905.286795:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000c22c48a7/0x1cd7fe53687339be lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1646592->1691647) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b014 expref: 9 pid: 6698 timeout: 497 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619905.286829:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78cd3cad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589037312:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619905.286942:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a240 going to sleep for 11 seconds 00010000:00010000:0.0:1788619905.289185:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5023800/0x1cd7fe5368733a12 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.289236:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5023800/0x1cd7fe5368733a12 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.289246:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5023800/0x1cd7fe5368733a12 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.289265:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c5023800 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788619905.289272:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5023800/0x1cd7fe5368733a12 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.289283:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5023800/0x1cd7fe5368733a12 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.289309:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619905.289313:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788619905.289319:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5023800/0x1cd7fe5368733a12 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.289358:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5023800/0x1cd7fe5368733a12 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619905.293956:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619905.294500:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571302400 00000100:00100000:0.0:1788619905.294525:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571302400 00002000:00100000:0.0:1788619905.294611:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1646592->1691647) req@ffff8e78df968000 x1875503571302400/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:1/0 lens 488/0 e 0 to 0 dl 1788619916 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619905.294626:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df968000 with x1875503571302400 ext(1646592->1691647) 00010000:00010000:0.0:1788619905.294640:0:6703:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000c22c48a7/0x1cd7fe53687339be lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1646592->1691647) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b014 expref: 10 pid: 6698 timeout: 497 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.294690:0:6703:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000c22c48a7/0x1cd7fe53687339be lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1646592->1691647) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b014 expref: 10 pid: 6698 timeout: 497 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788619905.294700:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff8e78df968000 00000100:00100000:0.0:1788619905.294746:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 5 00000100:00100000:0.0:1788619905.294758:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df968000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571302400:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:3.0:1788619905.296159:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78cd3cad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589037312:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619905.296205:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000062c34eb6/0x1cd7fe5368733a19 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b068 expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619905.296229:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c800, rc 0) 00000100:00100000:3.0:1788619905.296316:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cd3c9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:14908:x1875503571302272:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 9756us (10084us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619905.296360:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 485 00000100:00100000:0.0:1788619905.303032:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df968000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571302400:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 8260us (9073us total) trans 4294967364 rc 0/0 00000100:00100000:0.0:1788619905.303052:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 5 00002000:00100000:0.0:1788619905.303069:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1646592->1691647) req@ffff8e78df968000 x1875503571302400/t4294967364(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:1/0 lens 488/448 e 0 to 0 dl 1788619916 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619905.303081:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df968000 with x1875503571302400 ext(1646592->1691647) 00010000:00010000:0.0:1788619905.303094:0:6703:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000c22c48a7/0x1cd7fe53687339be lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1646592->1691647) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b014 expref: 12 pid: 6698 timeout: 497 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.303118:0:6703:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000c22c48a7/0x1cd7fe53687339be lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1646592->1691647) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b014 expref: 12 pid: 6698 timeout: 497 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788619905.303128:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff8e78df968000 00000100:00100000:1.0:1788619905.310493:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619905.310610:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571302528 00000100:00100000:2.0:1788619905.310628:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571302528 00010000:00010000:2.0:1788619905.310691:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000c22c48a7/0x1cd7fe53687339be lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1646592->1691647) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b014 expref: 12 pid: 6698 timeout: 497 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619905.310752:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 215 00000100:00100000:2.0:1788619905.310764:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8023850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571302528:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619905.310787:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619905.310827:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000c22c48a7/0x1cd7fe53687339be lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1646592->1691647) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b014 expref: 12 pid: 6698 timeout: 497 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.310844:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000c22c48a7/0x1cd7fe53687339be lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1646592->1691647) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b014 expref: 12 pid: 6698 timeout: 497 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.310875:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c22c48a7/0x1cd7fe53687339be lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1646592->1691647) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b014 expref: 12 pid: 6698 timeout: 497 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.310923:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(3 1) 00010000:00010000:2.0:1788619905.310928:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(3 1) 00010000:00010000:2.0:1788619905.310940:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000062c34eb6/0x1cd7fe5368733a19 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b068 expref: 5 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.310951:0:5831:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062c34eb6/0x1cd7fe5368733a19 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b068 expref: 5 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.311009:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000062c34eb6/0x1cd7fe5368733a19 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b068 expref: 5 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619905.311026:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fefd4000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5831:1875503589037440:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619905.311140:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619905.314624:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fefd4000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5831:1875503589037440:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619905.314664:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619905.314747:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8023850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571302528:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3983us (4269us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619905.314761:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 215 00000100:00100000:3.0:1788619905.325767:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619905.326229:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571302656 00000100:00100000:0.0:1788619905.326245:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571302656 00002000:00010000:0.0:1788619905.326308:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1618652->18446744073709551615) 00002000:00100000:0.0:1788619905.326314:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78cd3cad80 with x1875503571302656 ext(1618652->18446744073709551615) 00002000:00010000:0.0:1788619905.326408:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78cd3cad80. 00000100:00100000:0.0:1788619905.326510:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 252 00000100:00100000:0.0:1788619905.326523:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cd3cad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571302656:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:0.0:1788619905.327307:0:6702:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1785469 -> 1618652 00002000:00010000:0.0:1788619905.327315:0:6702:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619904 -> 1788619905 00002000:00010000:0.0:1788619905.327317:0:6702:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619904 -> 1788619904 00002000:00010000:0.0:1788619905.327320:0:6702:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619904 -> 1788619905 00002000:00010000:0.0:1788619905.327322:0:6702:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 391 -> 391 00000100:00100000:0.0:1788619905.327446:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cd3cad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571302656:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 921us (1691us total) trans 4294967365 rc 0/0 00000100:00100000:0.0:1788619905.327457:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 252 00002000:00010000:0.0:1788619905.327462:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1618652->18446744073709551615) 00002000:00100000:0.0:1788619905.327466:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78cd3cad80 with x1875503571302656 ext(1618652->18446744073709551615) 00002000:00010000:0.0:1788619905.327473:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78cd3cad80. 00000100:00100000:1.0:1788619905.331317:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619905.331473:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571302784 00000100:00100000:2.0:1788619905.331487:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571302784 00000100:00100000:2.0:1788619905.331563:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 653 00000100:00100000:2.0:1788619905.331576:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df968380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:14908:x1875503571302784:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619905.331611:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619905.331657:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d400/0x1cd7fe5368733a20 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b06f expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.331771:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505ca00/0x1cd7fe5368733a27 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.331789:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505ca00/0x1cd7fe5368733a27 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.331801:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505ca00/0x1cd7fe5368733a27 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788619905.331832:0:9417:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1618652 00000004:00200000:2.0:1788619905.331837:0:9417:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788619905.331861:0:9417:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505ca00/0x1cd7fe5368733a27 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.331891:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d400/0x1cd7fe5368733a20 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b06f expref: 17 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.331910:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505ca00/0x1cd7fe5368733a27 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b06f expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.331918:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619905.331921:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788619905.331924:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505ca00, rc 0) 00000100:00100000:2.0:1788619905.332137:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df968380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:14908:x1875503571302784:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 440us (712us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619905.332145:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 653 00000100:00100000:0.0:1788619905.337217:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619905.337364:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571302912 00000100:00100000:2.0:1788619905.337396:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571302912 00000100:00100000:2.0:1788619905.337465:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 654 00000100:00100000:2.0:1788619905.337477:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5024700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+8:14908:x1875503571302912:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619905.337504:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619905.337531:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d400/0x1cd7fe5368733a2e lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b076 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.337643:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c600/0x1cd7fe5368733a35 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.337664:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c600/0x1cd7fe5368733a35 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.337678:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c600/0x1cd7fe5368733a35 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788619905.337710:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1618652 00000004:00200000:2.0:1788619905.337715:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788619905.337737:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c600/0x1cd7fe5368733a35 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.337768:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d400/0x1cd7fe5368733a2e lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b076 expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.337787:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c600/0x1cd7fe5368733a35 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b076 expref: 9 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.337797:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619905.337801:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788619905.337806:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505c600, rc 0) 00000100:00100000:2.0:1788619905.337896:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5024700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571302912:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 422us (694us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619905.337904:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 654 00000100:00100000:3.0:1788619905.342111:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619905.342274:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571303040 00000100:00100000:0.0:1788619905.342287:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571303040 00000100:00100000:0.0:1788619905.342408:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 655 00000100:00100000:0.0:1788619905.342422:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cd3cb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571303040:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619905.342456:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619905.342487:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c0031c00/0x1cd7fe5368733a3c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b07d expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.342545:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5023c00/0x1cd7fe53687339ef lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b045 expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619905.342628:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5027100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589037568:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619905.342742:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa780 going to sleep for 11 seconds 00000100:00100000:1.0:1788619905.352407:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619905.352583:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5027100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589037568:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619905.352618:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 352256 00010000:00010000:2.0:1788619905.352644:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c0031c00/0x1cd7fe5368733a3c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b07d expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.352668:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619905.352670:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619905.352675:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c0031c00/0x1cd7fe5368733a3c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b07d expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.352690:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c0031c00, rc 0) 00000100:00100000:2.0:1788619905.352758:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cd3cb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571303040:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 10341us (10668us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619905.352766:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 655 00000100:00100000:0.0:1788619905.353652:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571303168 00000100:00100000:0.0:1788619905.353666:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571303168 00000100:00100000:0.0:1788619905.353786:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 486 00000100:00100000:0.0:1788619905.353799:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df969880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571303168:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619905.353818:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619905.353841:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000aaf7f27/0x1cd7fe5368733a43 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b084 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619905.353920:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5027100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589037696:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619905.354037:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa180 going to sleep for 11 seconds 00000100:00100000:0.0:1788619905.372450:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5027100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589037696:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619905.372511:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000aaf7f27/0x1cd7fe5368733a43 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b084 expref: 12 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.372544:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000aaf7f27/0x1cd7fe5368733a43 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b084 expref: 12 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.372562:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c0030800, rc 0) 00000100:00100000:0.0:1788619905.372642:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df969880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571303168:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 18846us (20248us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619905.372654:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 486 00000100:00100000:3.0:1788619905.376402:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619905.376463:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619905.376580:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571303296 00000100:00100000:2.0:1788619905.376593:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571303296 00000100:00100000:2.0:1788619905.376685:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 656 00000100:00100000:2.0:1788619905.376699:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cd3c9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571303296:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619905.376733:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619905.376764:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d400/0x1cd7fe5368733a4a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b08b expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.376815:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5023c00/0x1cd7fe53687339ef lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b045 expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619905.376899:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fefd4000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589037824:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619905.377046:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccc00 going to sleep for 11 seconds 00000100:00100000:0.0:1788619905.377587:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571303424 00000100:00100000:0.0:1788619905.377597:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571303424 00000100:00100000:0.0:1788619905.377647:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 487 00000100:00100000:0.0:1788619905.377656:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cd3c9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571303424:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619905.377668:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619905.377683:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000aaf7f27/0x1cd7fe5368733a51 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b092 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619905.377735:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5027100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589037952:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619905.377800:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa180 going to sleep for 11 seconds 00000100:00100000:2.0:1788619905.379495:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fefd4000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589037824:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619905.379534:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 352256 00010000:00010000:2.0:1788619905.379562:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d400/0x1cd7fe5368733a4a lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b08b expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.379590:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619905.379593:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619905.379600:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d400/0x1cd7fe5368733a4a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b08b expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.379618:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505d400, rc 0) 00000100:00100000:2.0:1788619905.379693:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cd3c9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571303296:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2996us (3311us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619905.379702:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 656 00000100:00100000:0.0:1788619905.380430:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5027100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589037952:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619905.380466:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000aaf7f27/0x1cd7fe5368733a51 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b092 expref: 12 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.380489:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000aaf7f27/0x1cd7fe5368733a51 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b092 expref: 12 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.380501:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c0030800, rc 0) 00000100:00100000:0.0:1788619905.380550:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cd3c9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571303424:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2895us (4086us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619905.380558:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 487 00000100:00100000:1.0:1788619905.402493:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619905.402594:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619905.402688:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571303680 00000100:00100000:2.0:1788619905.402703:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571303680 00000100:00100000:2.0:1788619905.402784:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 488 00000100:00100000:2.0:1788619905.402797:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df96ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571303680:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619905.402821:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619905.402845:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c22c48a7/0x1cd7fe5368733a58 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b0a0 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619905.402928:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fefd4000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589038080:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619905.403082:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccc00 going to sleep for 11 seconds 00000100:00100000:0.0:1788619905.404469:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571303552 00000100:00100000:0.0:1788619905.404486:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571303552 00000100:00100000:0.0:1788619905.404610:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 657 00000100:00100000:0.0:1788619905.404622:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df968e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571303552:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619905.404661:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619905.404686:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c0030800/0x1cd7fe5368733a5f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b099 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.404733:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5023c00/0x1cd7fe53687339ef lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b045 expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619905.404781:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5027100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589038208:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619905.404877:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa180 going to sleep for 11 seconds 00000100:00100000:1.0:1788619905.418034:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5027100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589038208:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619905.418075:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 352256 00010000:00010000:1.0:1788619905.418101:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c0030800/0x1cd7fe5368733a5f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b099 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619905.418129:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619905.418132:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619905.418138:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c0030800/0x1cd7fe5368733a5f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b099 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619905.418175:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c0030800, rc 0) 00000100:00100000:1.0:1788619905.418255:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df968e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571303552:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 13635us (15932us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619905.418272:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 657 00000100:00100000:0.0:1788619905.418698:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fefd4000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589038080:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619905.418733:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c22c48a7/0x1cd7fe5368733a58 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0a0 expref: 12 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.418752:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c22c48a7/0x1cd7fe5368733a58 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0a0 expref: 12 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.418764:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505d400, rc 0) 00000100:00100000:0.0:1788619905.418812:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df96ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571303680:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 16018us (16217us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619905.418819:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 488 00000100:00100000:3.0:1788619905.425466:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619905.425672:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571303808 00000100:00100000:2.0:1788619905.425686:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571303808 00000100:00100000:2.0:1788619905.425759:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 658 00000100:00100000:2.0:1788619905.425772:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cd3ca680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571303808:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619905.425801:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619905.425829:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d600/0x1cd7fe5368733a66 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b0a7 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.425877:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5023c00/0x1cd7fe53687339ef lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b045 expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619905.425934:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fefd4000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589038336:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619905.426068:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccf00 going to sleep for 11 seconds 00000100:00100000:3.0:1788619905.439434:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619905.444134:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571303936 00000100:00100000:1.0:1788619905.444158:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571303936 00000100:00100000:1.0:1788619905.444264:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 489 00000100:00100000:1.0:1788619905.444277:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cd3ca300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571303936:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619905.444305:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619905.444355:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb562d34/0x1cd7fe5368733a6d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b0ae expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619905.444439:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df96bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589038464:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619905.444558:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963180 going to sleep for 11 seconds 00000100:00100000:2.0:1788619905.445840:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fefd4000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589038336:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619905.445874:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 352256 00010000:00010000:2.0:1788619905.445898:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d600/0x1cd7fe5368733a66 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0a7 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.445924:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619905.445928:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619905.445935:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d600/0x1cd7fe5368733a66 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0a7 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.445954:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505d600, rc 0) 00000100:00100000:2.0:1788619905.446044:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cd3ca680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571303808:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 20274us (20594us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619905.446080:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 658 00000100:00100000:1.0:1788619905.462302:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df96bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589038464:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619905.462431:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bb562d34/0x1cd7fe5368733a6d lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0ae expref: 12 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619905.462464:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb562d34/0x1cd7fe5368733a6d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0ae expref: 12 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619905.462482:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5022c00, rc 0) 00000100:00100000:1.0:1788619905.462578:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cd3ca300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571303936:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 18302us (23160us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619905.462605:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 489 00000100:00100000:3.0:1788619905.467286:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619905.467695:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571304192 00000100:00100000:2.0:1788619905.467710:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571304192 00000100:00100000:2.0:1788619905.467801:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 659 00000100:00100000:2.0:1788619905.467824:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cd3caa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:14908:x1875503571304192:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:2.0:1788619905.467901:0:9417:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619905.467940:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505ca00/0x1cd7fe5368733a27 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b06f expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.467963:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619905.467969:0:9417:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788619905.468264:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c800/0x1cd7fe5368733a74 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.468282:0:9417:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c600/0x1cd7fe5368733a35 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b076 expref: 8 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.468291:0:9417:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c800/0x1cd7fe5368733a74 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.468357:0:9417:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c600/0x1cd7fe5368733a35 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b076 expref: 8 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.468374:0:9417:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c600/0x1cd7fe5368733a35 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b076 expref: 8 pid: 5844 timeout: 498 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619905.468390:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fefd4000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589038592:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619905.468683:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccf00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619905.476712:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619905.476818:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571304320 00000100:00100000:0.0:1788619905.476828:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571304320 00000100:00100000:1.0:1788619905.476846:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fefd4000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589038592:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619905.476875:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c600/0x1cd7fe5368733a35 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b076 expref: 9 pid: 5844 timeout: 498 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619905.476880:0:9417:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c800/0x1cd7fe5368733a74 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619905.476929:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 216 00000100:00100000:0.0:1788619905.476951:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571304320:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619905.476971:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619905.476979:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c600/0x1cd7fe5368733a35 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b076 expref: 9 pid: 5844 timeout: 498 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.477028:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c600/0x1cd7fe5368733a35 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b076 expref: 9 pid: 5844 timeout: 498 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.477058:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c600/0x1cd7fe5368733a35 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b076 expref: 9 pid: 5844 timeout: 498 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.477077:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619905.477082:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c800/0x1cd7fe5368733a74 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.477095:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c800/0x1cd7fe5368733a74 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.477104:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c800/0x1cd7fe5368733a74 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.477137:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788619905.477157:0:9417:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c800/0x1cd7fe5368733a74 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619905.477164:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c800/0x1cd7fe5368733a74 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619905.477205:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+8:2354:x1875503571304320:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 256us (508us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619905.477214:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 216 00000001:00080000:1.0:1788619905.477380:0:9417:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967558 00000004:00010000:1.0:1788619905.477457:0:9417:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 352256 -> 352256 00000004:00010000:1.0:1788619905.477461:0:9417:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619905 -> 1788619905 00000004:00010000:1.0:1788619905.477464:0:9417:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619904 -> 1788619904 00000004:00010000:1.0:1788619905.477466:0:9417:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619905 -> 1788619905 00000004:00010000:1.0:1788619905.477469:0:9417:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 774 -> 774 00000004:00010000:1.0:1788619905.477478:0:9417:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000f1368186 reply state 00000000b9999fd3 transno 4294967559 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c800/0x1cd7fe5368733a74 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619905.477489:0:9417:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788619905.477588:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cd3caa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+18:14908:x1875503571304192:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 9767us (10322us total) trans 4294967559 rc 0/0 00000100:00100000:1.0:1788619905.477596:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 659 00000020:00080000:1.0:1788619905.477600:0:9417:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1788619905.483993:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00010000:00010000:2.0:1788619905.484162:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c800/0x1cd7fe5368733a74 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.484179:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c800/0x1cd7fe5368733a74 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.484187:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c800/0x1cd7fe5368733a74 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.484193:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c505c800 already unused, calling callback (ffffffffc22064f0) 00000100:00100000:1.0:1788619905.484195:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571304448 00010000:00010000:2.0:1788619905.484199:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c800/0x1cd7fe5368733a74 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.484209:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c800/0x1cd7fe5368733a74 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619905.484209:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571304448 00010000:00010000:2.0:1788619905.484230:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619905.484234:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788619905.484239:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c800/0x1cd7fe5368733a74 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.484245:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c800/0x1cd7fe5368733a74 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788619905.484266:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1076053->18446744073709551615) 00002000:00100000:1.0:1788619905.484295:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5025f80 with x1875503571304448 ext(1076053->18446744073709551615) 00002000:00010000:1.0:1788619905.484309:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5025f80. 00000100:00100000:1.0:1788619905.484390:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 253 00000100:00100000:1.0:1788619905.484402:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5025f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571304448:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:0.0:1788619905.486834:0:6703:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1618652 -> 1076053 00002000:00010000:0.0:1788619905.486847:0:6703:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619905 -> 1788619905 00002000:00010000:0.0:1788619905.486850:0:6703:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619904 -> 1788619904 00002000:00010000:0.0:1788619905.486863:0:6703:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619905 -> 1788619905 00002000:00010000:0.0:1788619905.486866:0:6703:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 391 -> 391 00000100:00100000:0.0:1788619905.487228:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5025f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571304448:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 2567us (3002us total) trans 4294967366 rc 0/0 00000100:00100000:0.0:1788619905.487244:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 253 00002000:00010000:0.0:1788619905.487251:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1076053->18446744073709551615) 00002000:00100000:0.0:1788619905.487255:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5025f80 with x1875503571304448 ext(1076053->18446744073709551615) 00002000:00010000:0.0:1788619905.487264:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5025f80. 00000100:00100000:1.0:1788619905.498408:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619905.498870:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571304576 00000100:00100000:0.0:1788619905.498883:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571304576 00000100:00100000:0.0:1788619905.498957:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 660 00000100:00100000:0.0:1788619905.498968:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df96bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+17:14908:x1875503571304576:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619905.499022:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619905.499049:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c0031200/0x1cd7fe5368733a7b lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b0b5 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.499153:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c0030400/0x1cd7fe5368733a82 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.499170:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c0030400/0x1cd7fe5368733a82 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.499183:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c0030400/0x1cd7fe5368733a82 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619905.499212:0:5846:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1076053 00000004:00200000:0.0:1788619905.499218:0:5846:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619905.499241:0:5846:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c0030400/0x1cd7fe5368733a82 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.499295:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c0031200/0x1cd7fe5368733a7b lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0b5 expref: 19 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.499316:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c0030400/0x1cd7fe5368733a82 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0b5 expref: 18 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.499393:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619905.499396:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619905.499401:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c0030400, rc 0) 00000100:00100000:0.0:1788619905.499479:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df96bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+18:14908:x1875503571304576:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 513us (1087us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619905.499486:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 660 00000100:00100000:3.0:1788619905.507608:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619905.507695:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571304704 00000100:00100000:2.0:1788619905.507709:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571304704 00000100:00100000:2.0:1788619905.507787:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 661 00000100:00100000:2.0:1788619905.507799:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd93ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+8:14908:x1875503571304704:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619905.507824:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619905.507867:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505ca00/0x1cd7fe5368733a89 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b0bc expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.507995:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505de00/0x1cd7fe5368733a90 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.508015:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505de00/0x1cd7fe5368733a90 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.508031:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505de00/0x1cd7fe5368733a90 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788619905.508068:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1076053 00000004:00200000:2.0:1788619905.508073:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788619905.508097:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505de00/0x1cd7fe5368733a90 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.508128:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505ca00/0x1cd7fe5368733a89 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0bc expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.508150:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505de00/0x1cd7fe5368733a90 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0bc expref: 9 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.508160:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619905.508162:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788619905.508166:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505de00, rc 0) 00000100:00100000:2.0:1788619905.508242:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd93ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571304704:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 444us (651us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619905.508248:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 661 00000100:00100000:2.0:1788619905.514659:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619905.514735:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619905.514834:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571304832 00000100:00100000:1.0:1788619905.514848:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571304832 00000100:00100000:1.0:1788619905.514926:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 662 00000100:00100000:1.0:1788619905.514938:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571304832:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619905.514965:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619905.515006:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5022e00/0x1cd7fe5368733a97 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b0c3 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619905.515056:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5023c00/0x1cd7fe53687339ef lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b045 expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619905.515113:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df968700 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589038720:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619905.515227:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963300 going to sleep for 11 seconds 00000100:00100000:0.0:1788619905.515313:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571304960 00000100:00100000:0.0:1788619905.515318:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571304960 00000100:00100000:0.0:1788619905.515396:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 490 00000100:00100000:0.0:1788619905.515404:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571304960:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619905.515413:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619905.515427:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b9881d70/0x1cd7fe5368733a9e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b0ca expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619905.515471:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5027800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589038848:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619905.515509:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaab40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619905.527033:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5027800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589038848:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619905.527095:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b9881d70/0x1cd7fe5368733a9e lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0ca expref: 12 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.527158:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b9881d70/0x1cd7fe5368733a9e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0ca expref: 12 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.527176:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c0031200, rc 0) 00000100:00100000:0.0:1788619905.527267:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571304960:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 11862us (12530us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619905.527277:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 490 00000100:00100000:2.0:1788619905.527806:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df968700 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589038720:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619905.527829:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 352256 00010000:00010000:2.0:1788619905.527846:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5022e00/0x1cd7fe5368733a97 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0c3 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.527863:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619905.527867:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619905.527872:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5022e00/0x1cd7fe5368733a97 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0c3 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.527884:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5022e00, rc 0) 00000100:00100000:2.0:1788619905.527931:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571304832:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 12996us (13287us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619905.527939:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 662 00000100:00100000:1.0:1788619905.531872:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619905.531957:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619905.532616:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571305216 00000100:00100000:0.0:1788619905.532629:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571305216 00000100:00100000:0.0:1788619905.532688:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 491 00000100:00100000:0.0:1788619905.532697:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5770700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571305216:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619905.532720:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619905.532742:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b9881d70/0x1cd7fe5368733aa5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b0d8 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619905.532819:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5027800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589038976:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619905.532935:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaab40 going to sleep for 11 seconds 00000100:00100000:3.0:1788619905.534166:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571305088 00000100:00100000:3.0:1788619905.534180:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571305088 00000100:00100000:2.0:1788619905.534422:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 663 00000100:00100000:2.0:1788619905.534432:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df968700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571305088:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619905.534455:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619905.534478:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505ce00/0x1cd7fe5368733aac lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b0d1 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.534534:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5023c00/0x1cd7fe53687339ef lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b045 expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619905.534585:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c75f4e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589039104:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619905.534679:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc600 going to sleep for 11 seconds 00000100:00100000:0.0:1788619905.536598:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5027800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589038976:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619905.536646:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b9881d70/0x1cd7fe5368733aa5 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0d8 expref: 12 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.536677:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b9881d70/0x1cd7fe5368733aa5 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0d8 expref: 12 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.536717:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c0031200, rc 0) 00000100:00100000:0.0:1788619905.536780:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5770700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571305216:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4083us (4821us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619905.536790:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 491 00000100:00100000:2.0:1788619905.537451:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c75f4e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589039104:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619905.537478:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 352256 00010000:00010000:2.0:1788619905.537493:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505ce00/0x1cd7fe5368733aac lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0d1 expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.537521:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619905.537525:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619905.537530:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505ce00/0x1cd7fe5368733aac lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0d1 expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.537543:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505ce00, rc 0) 00000100:00100000:2.0:1788619905.537586:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df968700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571305088:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3155us (5728us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619905.537592:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 663 00000100:00100000:2.0:1788619905.544055:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619905.544264:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571305344 00000100:00100000:1.0:1788619905.544279:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571305344 00000100:00100000:1.0:1788619905.544418:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 346 00000100:00100000:1.0:1788619905.544431:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2352:x1875503571305344:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:1.0:1788619905.544790:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 352256 -> 352256 00000004:00010000:1.0:1788619905.544795:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619905 -> 1788619904 00000004:00010000:1.0:1788619905.544814:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619904 -> 1788619904 00000004:00010000:1.0:1788619905.544816:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619905 -> 1788619904 00000004:00010000:1.0:1788619905.544819:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 774 -> 774 00000100:00100000:1.0:1788619905.544918:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2352:x1875503571305344:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 489us (875us total) trans 4294967560 rc 0/0 00000100:00100000:1.0:1788619905.544926:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 346 00000100:00100000:1.0:1788619905.549160:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619905.549215:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619905.549371:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571305600 00000100:00100000:0.0:1788619905.549384:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571305600 00000100:00100000:0.0:1788619905.549448:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 492 00000100:00100000:0.0:1788619905.549459:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571305600:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619905.549483:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619905.549504:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b9881d70/0x1cd7fe5368733ab3 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b0e6 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619905.549591:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5027800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589039232:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619905.549696:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaab40 going to sleep for 11 seconds 00000100:00100000:3.0:1788619905.550261:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571305472 00000100:00100000:3.0:1788619905.550287:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571305472 00000100:00100000:3.0:1788619905.550379:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 664 00000100:00100000:3.0:1788619905.550389:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5773100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571305472:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619905.550404:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619905.550423:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368733aba lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b0df expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619905.550461:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5023c00/0x1cd7fe53687339ef lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b045 expref: 18 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619905.550495:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fd93a300 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589039360:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619905.550781:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a240 going to sleep for 11 seconds 00000004:00010000:2.0:1788619905.561221:0:11397:0:(mdt_lvb.c:183:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime: 1788619904 -> 1788619905 00000004:00010000:2.0:1788619905.561232:0:11397:0:(mdt_lvb.c:195:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime: 1788619904 -> 1788619905 00000100:00100000:2.0:1788619905.561257:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fd93a300 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589039360:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619905.561282:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 352256 00010000:00010000:2.0:1788619905.561301:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368733aba lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0df expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.561321:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619905.561364:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619905.561371:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368733aba lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0df expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.561385:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683ce00, rc 0) 00000100:00100000:2.0:1788619905.561526:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5773100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571305472:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 11137us (12381us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619905.561535:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 664 00000100:00100000:0.0:1788619905.561659:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5027800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589039232:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619905.561680:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b9881d70/0x1cd7fe5368733ab3 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0e6 expref: 12 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.561696:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b9881d70/0x1cd7fe5368733ab3 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0e6 expref: 12 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.561704:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c0031200, rc 0) 00000100:00100000:0.0:1788619905.561725:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571305600:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 12270us (12511us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619905.561751:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 492 00000100:00100000:2.0:1788619905.576467:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619905.576545:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619905.576676:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571305856 00000100:00100000:3.0:1788619905.576688:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571305856 00000100:00100000:3.0:1788619905.576747:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 493 00000100:00100000:3.0:1788619905.576758:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c0ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2352:x1875503571305856:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619905.576780:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619905.576801:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003b84bbc9/0x1cd7fe5368733ac1 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b0f4 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619905.576882:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fd939f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589039488:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619905.577033:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a6c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619905.577154:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571305728 00000100:00100000:1.0:1788619905.577159:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571305728 00000100:00100000:1.0:1788619905.577209:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 665 00000100:00100000:1.0:1788619905.577218:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571305728:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619905.577229:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619905.577247:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5022800/0x1cd7fe5368733ac8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b0ed expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619905.577282:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5023c00/0x1cd7fe53687339ef lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b045 expref: 18 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619905.577313:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e647aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589039616:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619905.577382:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963840 going to sleep for 11 seconds 00000100:00100000:2.0:1788619905.582734:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e647aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589039616:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619905.582774:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 352256 00010000:00010000:2.0:1788619905.582800:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5022800/0x1cd7fe5368733ac8 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0ed expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.582841:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619905.582845:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619905.582851:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5022800/0x1cd7fe5368733ac8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0ed expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.582869:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5022800, rc 0) 00000100:00100000:2.0:1788619905.582945:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571305728:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5727us (6493us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619905.582955:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 665 00000100:00100000:3.0:1788619905.583862:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fd939f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589039488:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619905.583898:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003b84bbc9/0x1cd7fe5368733ac1 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0f4 expref: 12 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619905.583949:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b84bbc9/0x1cd7fe5368733ac1 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0f4 expref: 12 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619905.583963:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683de00, rc 0) 00000100:00100000:3.0:1788619905.584026:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c0ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2352:x1875503571305856:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7269us (7480us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619905.584034:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 493 00000100:00100000:1.0:1788619905.596005:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619905.596122:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571305984 00000100:00100000:2.0:1788619905.596136:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571305984 00000100:00100000:2.0:1788619905.596451:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 666 00000100:00100000:2.0:1788619905.596467:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e647bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571305984:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619905.596499:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619905.596535:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d000/0x1cd7fe5368733acf lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b0fb expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.596609:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5023c00/0x1cd7fe53687339ef lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b045 expref: 18 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.596621:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d000/0x1cd7fe5368733acf lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0fb expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.596675:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5023c00/0x1cd7fe53687339ef lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b045 expref: 18 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.596690:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5023c00/0x1cd7fe53687339ef lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b045 expref: 18 pid: 5845 timeout: 498 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619905.596706:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c09f80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589039744:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619905.596822:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc600 going to sleep for 11 seconds 00000100:00100000:2.0:1788619905.600613:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619905.600724:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571306112 00000100:00100000:1.0:1788619905.600737:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571306112 00010000:00010000:1.0:1788619905.600790:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5023c00/0x1cd7fe53687339ef lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b045 expref: 19 pid: 5845 timeout: 498 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619905.600866:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 217 00000100:00100000:1.0:1788619905.600878:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2354:x1875503571306112:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619905.600900:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619905.600938:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5023c00/0x1cd7fe53687339ef lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b045 expref: 19 pid: 5845 timeout: 498 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619905.600955:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5023c00/0x1cd7fe53687339ef lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b045 expref: 19 pid: 5845 timeout: 498 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619905.600999:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619905.601006:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d000/0x1cd7fe5368733acf lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0fb expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619905.601022:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d000/0x1cd7fe5368733acf lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0fb expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619905.601033:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d000/0x1cd7fe5368733acf lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0fb expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619905.601087:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d000/0x1cd7fe5368733acf lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0fb expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619905.601105:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e647b480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589039872:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619905.601211:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd9633c0 going to sleep for 11 seconds 00010000:00010000:3.0:1788619905.602268:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5023c00/0x1cd7fe53687339ef lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b045 expref: 19 pid: 5845 timeout: 498 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619905.602294:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c09f80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589039744:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619905.602387:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d000/0x1cd7fe5368733acf lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0fb expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619905.602402:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619905.602405:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619905.602409:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505d000, rc 0) 00000100:00100000:3.0:1788619905.602477:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e647bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:14908:x1875503571305984:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6015us (6499us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619905.602486:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 666 00000100:00100000:2.0:1788619905.621659:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619905.621804:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e647b480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589039872:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619905.621840:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619905.621925:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2354:x1875503571306112:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 21049us (21327us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619905.622015:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 217 00000100:00100000:0.0:1788619905.622122:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571306240 00000100:00100000:0.0:1788619905.622132:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571306240 00000100:00100000:0.0:1788619905.622194:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 667 00000100:00100000:0.0:1788619905.622206:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2355:x1875503571306240:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619905.622236:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619905.622263:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c0031600/0x1cd7fe5368733ad6 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b102 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.622314:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d000/0x1cd7fe5368733acf lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0fb expref: 9 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619905.622397:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5027800 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589040000:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619905.622464:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaab40 going to sleep for 11 seconds 00000004:00010000:2.0:1788619905.626612:0:11397:0:(mdt_lvb.c:177:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size: 352256 -> 782531 00000100:00100000:2.0:1788619905.626657:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5027800 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589040000:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619905.626687:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 782531 00010000:00010000:2.0:1788619905.626714:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c0031600/0x1cd7fe5368733ad6 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b102 expref: 19 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.626741:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619905.626744:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619905.626751:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c0031600/0x1cd7fe5368733ad6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b102 expref: 19 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.626796:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c0031600, rc 0) 00000100:00100000:2.0:1788619905.626876:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2355:x1875503571306240:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4671us (5229us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619905.626886:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 667 00000100:00100000:2.0:1788619905.639675:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619905.640370:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571306368 00000100:00100000:1.0:1788619905.640404:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571306368 00000100:00100000:1.0:1788619905.640494:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 668 00000100:00100000:1.0:1788619905.640509:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2353:x1875503571306368:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619905.640538:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619905.640565:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5023600/0x1cd7fe5368733add lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b109 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619905.640613:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d000/0x1cd7fe5368733acf lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0fb expref: 9 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619905.640667:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e6479180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589040128:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619905.640801:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd9636c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619905.647457:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e6479180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589040128:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619905.647499:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 782531 00010000:00010000:1.0:1788619905.647526:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5023600/0x1cd7fe5368733add lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b109 expref: 19 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619905.647555:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619905.647558:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619905.647565:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5023600/0x1cd7fe5368733add lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b109 expref: 19 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619905.647587:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5023600, rc 0) 00000100:00100000:1.0:1788619905.647679:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2353:x1875503571306368:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7174us (8018us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619905.647691:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 668 00000100:00100000:3.0:1788619905.654485:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619905.654912:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571306496 00000100:00100000:2.0:1788619905.654927:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571306496 00000100:00100000:2.0:1788619905.655031:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 669 00000100:00100000:2.0:1788619905.655044:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd939f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2353:x1875503571306496:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619905.655076:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619905.655104:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d200/0x1cd7fe5368733ae4 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b110 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.655155:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d000/0x1cd7fe5368733acf lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0fb expref: 9 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619905.655215:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c4a4f100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589040256:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619905.655794:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc600 going to sleep for 11 seconds 00000100:00100000:3.0:1788619905.660046:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c4a4f100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589040256:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619905.660088:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 782531 00010000:00010000:3.0:1788619905.660115:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d200/0x1cd7fe5368733ae4 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b110 expref: 19 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619905.660143:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619905.660147:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619905.660168:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d200/0x1cd7fe5368733ae4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b110 expref: 19 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619905.660189:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505d200, rc 0) 00000100:00100000:3.0:1788619905.660284:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd939f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2353:x1875503571306496:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5242us (5813us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619905.660293:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 669 00000100:00100000:0.0:1788619905.665232:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619905.665443:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571306624 00000100:00100000:2.0:1788619905.665458:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571306624 00000100:00100000:2.0:1788619905.665534:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 670 00000100:00100000:2.0:1788619905.665544:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5027b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2352:x1875503571306624:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619905.665570:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619905.665596:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505dc00/0x1cd7fe5368733aeb lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b117 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.665673:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d000/0x1cd7fe5368733acf lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0fb expref: 9 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619905.665731:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c4a4c700 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589040384:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619905.666067:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc840 going to sleep for 11 seconds 00000100:00100000:2.0:1788619905.668605:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c4a4c700 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589040384:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619905.668643:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 782531 00010000:00010000:2.0:1788619905.668671:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505dc00/0x1cd7fe5368733aeb lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b117 expref: 19 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.668698:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619905.668701:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619905.668709:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505dc00/0x1cd7fe5368733aeb lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b117 expref: 19 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.668728:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505dc00, rc 0) 00000100:00100000:2.0:1788619905.668818:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5027b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2352:x1875503571306624:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3274us (3602us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619905.668828:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 670 00000100:00100000:1.0:1788619905.676567:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619905.676819:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571306752 00000100:00100000:3.0:1788619905.676834:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571306752 00002000:00100000:3.0:1788619905.676914:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e6479180 x1875503571306752/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:1/0 lens 488/0 e 0 to 0 dl 1788619916 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619905.676927:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e6479180 with x1875503571306752 00000100:00100000:3.0:1788619905.676970:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 347 00000100:00100000:3.0:1788619905.677023:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e6479180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571306752:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619905.690471:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e6479180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571306752:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 13432us (13901us total) trans 0 rc 352256/352256 00000100:00100000:3.0:1788619905.690510:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 347 00002000:00100000:3.0:1788619905.690526:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e6479180 x1875503571306752/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:1/0 lens 488/440 e 0 to 0 dl 1788619916 ref 1 fl Complete:/600/0 rc 352256/352256 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619905.690539:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e6479180 with x1875503571306752 00000100:00100000:0.0:1788619905.695230:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619905.695356:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571306880 00000100:00100000:1.0:1788619905.695370:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571306880 00000100:00100000:1.0:1788619905.695465:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 671 00000100:00100000:1.0:1788619905.695479:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5027800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2353:x1875503571306880:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619905.695506:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619905.695533:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5022a00/0x1cd7fe5368733af2 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b11e expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619905.695584:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d000/0x1cd7fe5368733acf lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0fb expref: 9 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619905.695636:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e6479c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589040512:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619905.695751:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd9636c0 going to sleep for 11 seconds 00000004:00010000:1.0:1788619905.698677:0:9417:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619904 -> 1788619905 00000100:00100000:1.0:1788619905.698726:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e6479c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589040512:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619905.698758:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 782531 00010000:00010000:1.0:1788619905.698783:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5022a00/0x1cd7fe5368733af2 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b11e expref: 19 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619905.698820:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619905.698824:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619905.698830:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5022a00/0x1cd7fe5368733af2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b11e expref: 19 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619905.698851:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5022a00, rc 0) 00000100:00100000:1.0:1788619905.698930:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5027800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2353:x1875503571306880:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3454us (3716us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619905.698963:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 671 00000100:00100000:2.0:1788619905.702508:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619905.702688:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571307008 00000100:00100000:3.0:1788619905.702700:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571307008 00000100:00100000:3.0:1788619905.702777:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 672 00000100:00100000:3.0:1788619905.702788:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2355:x1875503571307008:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619905.702822:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619905.702850:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368733af9 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b125 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619905.702901:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d000/0x1cd7fe5368733acf lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0fb expref: 9 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619905.703052:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fd93a300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589040640:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619905.703215:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619905.707998:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fd93a300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589040640:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619905.708040:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 782531 00010000:00010000:3.0:1788619905.708068:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368733af9 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b125 expref: 19 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619905.708094:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619905.708098:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619905.708104:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368733af9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b125 expref: 19 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619905.708151:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c600, rc 0) 00000100:00100000:3.0:1788619905.708237:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2355:x1875503571307008:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5449us (5746us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619905.711929:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 672 00000100:00100000:2.0:1788619905.712241:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619905.712559:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571307136 00000100:00100000:3.0:1788619905.712574:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571307136 00000100:00100000:3.0:1788619905.712647:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 673 00000100:00100000:3.0:1788619905.712663:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2353:x1875503571307136:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619905.712701:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619905.712736:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368733b00 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b12c expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619905.712788:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d000/0x1cd7fe5368733acf lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0fb expref: 9 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619905.712856:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fd93a300 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589040768:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619905.712990:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619905.716100:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fd93a300 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589040768:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619905.716186:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 782531 00010000:00010000:3.0:1788619905.716212:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368733b00 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b12c expref: 19 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619905.716238:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619905.716241:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619905.716256:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368733b00 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b12c expref: 19 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619905.716292:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d600, rc 0) 00000100:00100000:3.0:1788619905.716407:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2353:x1875503571307136:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3747us (4170us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619905.716418:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 673 00000100:00100000:1.0:1788619905.719953:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619905.720085:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571307264 00000100:00100000:2.0:1788619905.720098:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571307264 00000100:00100000:2.0:1788619905.720181:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 674 00000100:00100000:2.0:1788619905.720194:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e6479c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+18:14908:x1875503571307264:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619905.720256:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619905.720287:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c200/0x1cd7fe5368733b07 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b133 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.720362:0:9417:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d000/0x1cd7fe5368733acf lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0fb expref: 9 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.720376:0:9417:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c200/0x1cd7fe5368733b07 lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b133 expref: 19 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.720423:0:9417:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d000/0x1cd7fe5368733acf lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0fb expref: 9 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619905.720440:0:9417:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d000/0x1cd7fe5368733acf lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0fb expref: 9 pid: 5844 timeout: 498 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619905.720456:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c4a4ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589040896:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619905.720565:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc840 going to sleep for 11 seconds 00000100:00100000:1.0:1788619905.724782:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619905.724917:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571307392 00000100:00100000:0.0:1788619905.724932:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571307392 00002000:00100000:0.0:1788619905.725018:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e647b480 x1875503571307392/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:1/0 lens 4784/0 e 0 to 0 dl 1788619916 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619905.725032:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e647b480 with x1875503571307392 00010000:00010000:0.0:1788619905.725044:0:5857:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d000/0x1cd7fe5368733acf lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0fb expref: 10 pid: 5844 timeout: 498 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.725061:0:5857:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d000/0x1cd7fe5368733acf lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0fb expref: 10 pid: 5844 timeout: 498 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788619905.725095:0:5857:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e78e647b480 00000100:00100000:0.0:1788619905.725143:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 17 00000100:00100000:0.0:1788619905.725154:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e647b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571307392:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:0.0:1788619905.725716:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 782531 -> 782531 00000004:00010000:0.0:1788619905.725722:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619905 -> 1788619905 00000004:00010000:0.0:1788619905.725725:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619905 -> 1788619904 00000004:00010000:0.0:1788619905.725727:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619905 -> 1788619905 00000004:00010000:0.0:1788619905.725730:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 774 -> 774 00000100:00100000:0.0:1788619905.726463:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e647b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571307392:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 1302us (1688us total) trans 4294967561 rc 0/0 00000100:00100000:0.0:1788619905.726478:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 17 00002000:00100000:0.0:1788619905.726492:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e647b480 x1875503571307392/t4294967561(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:1/0 lens 4784/448 e 0 to 0 dl 1788619916 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619905.726503:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e647b480 with x1875503571307392 00010000:00010000:0.0:1788619905.726513:0:5857:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d000/0x1cd7fe5368733acf lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0fb expref: 12 pid: 5844 timeout: 498 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.726530:0:5857:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d000/0x1cd7fe5368733acf lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0fb expref: 12 pid: 5844 timeout: 498 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788619905.726539:0:5857:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e78e647b480 00000100:00100000:3.0:1788619905.726695:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c4a4ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589040896:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619905.726737:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c200/0x1cd7fe5368733b07 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b133 expref: 19 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619905.726755:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619905.726759:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619905.726764:0:9417:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c200/0x1cd7fe5368733b07 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b133 expref: 19 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619905.726777:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c505c200, rc 0) 00000100:00100000:3.0:1788619905.726894:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e6479c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+19:14908:x1875503571307264:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6703us (6956us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619905.726903:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 674 00000100:00100000:1.0:1788619905.733576:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619905.733875:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571307520 00000100:00100000:0.0:1788619905.733890:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571307520 00010000:00010000:0.0:1788619905.733948:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d000/0x1cd7fe5368733acf lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0fb expref: 12 pid: 5844 timeout: 498 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619905.734089:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 218 00000100:00100000:0.0:1788619905.734100:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e647aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2353:x1875503571307520:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619905.734120:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619905.734153:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d000/0x1cd7fe5368733acf lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0fb expref: 12 pid: 5844 timeout: 498 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.734168:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d000/0x1cd7fe5368733acf lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0fb expref: 12 pid: 5844 timeout: 498 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.734194:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505d000/0x1cd7fe5368733acf lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0fb expref: 12 pid: 5844 timeout: 498 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.734216:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619905.734224:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c200/0x1cd7fe5368733b07 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b133 expref: 18 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.734239:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c200/0x1cd7fe5368733b07 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b133 expref: 18 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.735119:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c200/0x1cd7fe5368733b07 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b133 expref: 18 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.737090:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c200/0x1cd7fe5368733b07 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b133 expref: 18 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619905.737146:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5027100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589041024:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619905.737289:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:1.0:1788619905.745989:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619905.746033:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619905.746158:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5027100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589041024:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619905.746192:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619905.746273:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e647aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571307520:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 12173us (12713us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619905.746286:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 218 00000100:00100000:2.0:1788619905.746460:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571307776 00000100:00100000:2.0:1788619905.746470:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571307776 00002000:00100000:2.0:1788619905.746533:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1785856->1957887) req@ffff8e77c74a9f80 x1875503571307776/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:1/0 lens 488/0 e 0 to 0 dl 1788619916 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619905.746545:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c74a9f80 with x1875503571307776 ext(1785856->1957887) 00002000:00010000:2.0:1788619905.746553:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c74a9f80 00000100:00100000:2.0:1788619905.746593:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 254 00000100:00100000:2.0:1788619905.746605:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74a9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571307776:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619905.747775:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571307648 00000100:00100000:3.0:1788619905.747791:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571307648 00002000:00100000:3.0:1788619905.747854:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c74a9500 x1875503571307648/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:1/0 lens 488/0 e 0 to 0 dl 1788619916 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619905.747867:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c74a9500 with x1875503571307648 00000100:00100000:3.0:1788619905.747897:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 348 00000100:00100000:3.0:1788619905.747908:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74a9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2355:x1875503571307648:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619905.748566:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74a9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2355:x1875503571307648:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 657us (2598us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619905.748576:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 348 00002000:00100000:3.0:1788619905.748584:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c74a9500 x1875503571307648/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:1/0 lens 488/4536 e 0 to 0 dl 1788619916 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619905.748593:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c74a9500 with x1875503571307648 00000100:00100000:2.0:1788619905.759745:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74a9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571307776:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 13123us (13691us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619905.759763:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 254 00002000:00100000:2.0:1788619905.759787:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1785856->1957887) req@ffff8e77c74a9f80 x1875503571307776/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:1/0 lens 488/440 e 0 to 0 dl 1788619916 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619905.759799:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c74a9f80 with x1875503571307776 ext(1785856->1957887) 00002000:00010000:2.0:1788619905.759886:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c74a9f80 00000100:00100000:2.0:1788619905.768729:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619905.768922:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571307904 00000100:00100000:0.0:1788619905.768935:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571307904 00002000:00100000:0.0:1788619905.769013:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c4a4d880 x1875503571307904/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:1/0 lens 488/0 e 0 to 0 dl 1788619916 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619905.769027:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c4a4d880 with x1875503571307904 00000100:00100000:0.0:1788619905.769113:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 349 00000100:00100000:0.0:1788619905.769124:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2355:x1875503571307904:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619905.769546:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2355:x1875503571307904:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 421us (1050us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619905.769555:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 349 00002000:00100000:0.0:1788619905.769562:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c4a4d880 x1875503571307904/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:1/0 lens 488/4536 e 0 to 0 dl 1788619916 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619905.769571:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c4a4d880 with x1875503571307904 00000100:00100000:3.0:1788619905.773645:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619905.774928:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571308032 00000100:00100000:1.0:1788619905.774948:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571308032 00002000:00100000:1.0:1788619905.775041:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fd93a300 x1875503571308032/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:1/0 lens 488/0 e 0 to 0 dl 1788619916 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619905.775055:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fd93a300 with x1875503571308032 00000100:00100000:1.0:1788619905.775090:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 350 00000100:00100000:1.0:1788619905.775101:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd93a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2354:x1875503571308032:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619905.775549:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd93a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2354:x1875503571308032:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 447us (1922us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619905.775558:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 350 00002000:00100000:1.0:1788619905.775566:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fd93a300 x1875503571308032/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:1/0 lens 488/4536 e 0 to 0 dl 1788619916 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619905.775575:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fd93a300 with x1875503571308032 00000100:00100000:3.0:1788619905.779032:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619905.779195:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571308160 00000100:00100000:1.0:1788619905.779208:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571308160 00002000:00100000:1.0:1788619905.779280:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5c09f80 x1875503571308160/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:1/0 lens 488/0 e 0 to 0 dl 1788619916 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619905.779294:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5c09f80 with x1875503571308160 00000100:00100000:1.0:1788619905.779373:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 351 00000100:00100000:1.0:1788619905.779403:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c09f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2355:x1875503571308160:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619905.779756:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c09f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2355:x1875503571308160:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 354us (744us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619905.779764:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 351 00002000:00100000:1.0:1788619905.779771:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5c09f80 x1875503571308160/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:1/0 lens 488/4536 e 0 to 0 dl 1788619916 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619905.779779:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5c09f80 with x1875503571308160 00000100:00100000:3.0:1788619905.784221:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619905.784642:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571308288 00000100:00100000:1.0:1788619905.784657:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571308288 00002000:00100000:1.0:1788619905.784730:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6532300 x1875503571308288/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:1/0 lens 488/0 e 0 to 0 dl 1788619916 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619905.784743:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6532300 with x1875503571308288 00000100:00100000:1.0:1788619905.784796:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 352 00000100:00100000:1.0:1788619905.784820:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6532300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2354:x1875503571308288:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619905.785433:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6532300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2354:x1875503571308288:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 623us (1222us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619905.785444:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 352 00002000:00100000:1.0:1788619905.785453:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6532300 x1875503571308288/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:1/0 lens 488/4536 e 0 to 0 dl 1788619916 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619905.785463:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6532300 with x1875503571308288 00000100:00100000:3.0:1788619905.789712:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619905.790454:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571308416 00000100:00100000:2.0:1788619905.790471:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571308416 00002000:00100000:2.0:1788619905.790549:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6533100 x1875503571308416/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:1/0 lens 488/0 e 0 to 0 dl 1788619916 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619905.790563:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6533100 with x1875503571308416 00000100:00100000:2.0:1788619905.790651:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 353 00000100:00100000:2.0:1788619905.790662:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6533100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2353:x1875503571308416:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619905.791049:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6533100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2353:x1875503571308416:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 387us (1353us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619905.791058:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 353 00002000:00100000:2.0:1788619905.791066:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6533100 x1875503571308416/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:1/0 lens 488/4536 e 0 to 0 dl 1788619916 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619905.791074:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6533100 with x1875503571308416 00000100:00100000:3.0:1788619905.795156:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619905.795515:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571308544 00000100:00100000:1.0:1788619905.795529:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571308544 00002000:00100000:1.0:1788619905.795598:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6530380 x1875503571308544/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:1/0 lens 488/0 e 0 to 0 dl 1788619916 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619905.795612:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6530380 with x1875503571308544 00000100:00100000:1.0:1788619905.795687:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 354 00000100:00100000:1.0:1788619905.795699:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6530380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2355:x1875503571308544:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619905.796097:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6530380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2355:x1875503571308544:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 399us (957us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619905.796106:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 354 00002000:00100000:1.0:1788619905.796113:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6530380 x1875503571308544/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:1/0 lens 488/4536 e 0 to 0 dl 1788619916 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619905.796122:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6530380 with x1875503571308544 00000100:00100000:3.0:1788619905.799994:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619905.800237:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571308672 00000100:00100000:1.0:1788619905.800259:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571308672 00002000:00100000:1.0:1788619905.800365:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6532a00 x1875503571308672/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:1/0 lens 488/0 e 0 to 0 dl 1788619916 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619905.800380:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6532a00 with x1875503571308672 00000100:00100000:1.0:1788619905.800417:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 355 00000100:00100000:1.0:1788619905.800427:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6532a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2352:x1875503571308672:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619905.800773:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6532a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2352:x1875503571308672:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 346us (806us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619905.800782:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 355 00002000:00100000:1.0:1788619905.800790:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6532a00 x1875503571308672/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:1/0 lens 488/4536 e 0 to 0 dl 1788619916 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619905.800798:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6532a00 with x1875503571308672 00000100:00100000:1.0:1788619905.805551:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619905.805676:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571308800 00000100:00100000:2.0:1788619905.805690:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571308800 00002000:00100000:2.0:1788619905.805761:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c74a9880 x1875503571308800/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:1/0 lens 488/0 e 0 to 0 dl 1788619916 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619905.805799:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c74a9880 with x1875503571308800 00000100:00100000:2.0:1788619905.805860:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 356 00000100:00100000:2.0:1788619905.805872:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74a9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2355:x1875503571308800:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619905.806611:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74a9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2355:x1875503571308800:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 736us (1071us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619905.806623:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 356 00002000:00100000:2.0:1788619905.806632:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c74a9880 x1875503571308800/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:1/0 lens 488/4536 e 0 to 0 dl 1788619916 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619905.806641:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c74a9880 with x1875503571308800 00000100:00100000:2.0:1788619905.810763:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619905.810928:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571308928 00000100:00100000:3.0:1788619905.810941:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571308928 00002000:00100000:3.0:1788619905.811025:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c4a4d500 x1875503571308928/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:1/0 lens 488/0 e 0 to 0 dl 1788619916 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619905.811037:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c4a4d500 with x1875503571308928 00000100:00100000:3.0:1788619905.811446:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 357 00000100:00100000:3.0:1788619905.811459:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2352:x1875503571308928:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619905.812187:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2352:x1875503571308928:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 726us (1435us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619905.812199:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 357 00002000:00100000:3.0:1788619905.812209:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c4a4d500 x1875503571308928/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:1/0 lens 488/4536 e 0 to 0 dl 1788619916 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619905.812218:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c4a4d500 with x1875503571308928 00000100:00100000:2.0:1788619905.817049:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619905.817147:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571309056 00000100:00100000:3.0:1788619905.817160:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571309056 00002000:00100000:3.0:1788619905.817235:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c4a4f800 x1875503571309056/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:1/0 lens 488/0 e 0 to 0 dl 1788619916 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619905.817260:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c4a4f800 with x1875503571309056 00000100:00100000:3.0:1788619905.817306:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 358 00000100:00100000:3.0:1788619905.817317:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2355:x1875503571309056:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619905.817734:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2355:x1875503571309056:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 416us (699us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619905.817743:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 358 00002000:00100000:3.0:1788619905.817751:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c4a4f800 x1875503571309056/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:1/0 lens 488/4536 e 0 to 0 dl 1788619916 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619905.817759:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c4a4f800 with x1875503571309056 00000100:00100000:2.0:1788619905.822197:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619905.822706:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571309184 00000100:00100000:3.0:1788619905.822721:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571309184 00002000:00100000:3.0:1788619905.822794:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c4a4c000 x1875503571309184/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:1/0 lens 488/0 e 0 to 0 dl 1788619916 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619905.822820:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c4a4c000 with x1875503571309184 00000100:00100000:3.0:1788619905.822871:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 359 00000100:00100000:3.0:1788619905.822884:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2354:x1875503571309184:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619905.823648:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2354:x1875503571309184:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 760us (1459us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619905.823659:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 359 00002000:00100000:3.0:1788619905.823669:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c4a4c000 x1875503571309184/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:1/0 lens 488/4536 e 0 to 0 dl 1788619916 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619905.823702:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c4a4c000 with x1875503571309184 00010000:00010000:3.0:1788619905.828630:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788619905.829776:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619905.829897:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571309312 00000100:00100000:1.0:1788619905.829911:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571309312 00002000:00100000:1.0:1788619905.829971:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c4a4ea00 x1875503571309312/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:1/0 lens 488/0 e 0 to 0 dl 1788619916 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619905.830013:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c4a4ea00 with x1875503571309312 00000100:00100000:1.0:1788619905.830104:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 360 00000100:00100000:1.0:1788619905.830116:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2354:x1875503571309312:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619905.830509:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2354:x1875503571309312:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 393us (740us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619905.830517:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 360 00002000:00100000:1.0:1788619905.830525:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c4a4ea00 x1875503571309312/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:1/0 lens 488/4536 e 0 to 0 dl 1788619916 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619905.830533:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c4a4ea00 with x1875503571309312 00000100:00100000:0.0:1788619905.833479:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619905.833585:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571309440 00000100:00100000:3.0:1788619905.833597:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571309440 00002000:00100000:3.0:1788619905.833662:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5027100 x1875503571309440/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:1/0 lens 488/0 e 0 to 0 dl 1788619916 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619905.833674:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5027100 with x1875503571309440 00000100:00100000:3.0:1788619905.833719:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 361 00000100:00100000:3.0:1788619905.833730:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5027100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2354:x1875503571309440:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619905.834288:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5027100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2354:x1875503571309440:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 556us (819us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619905.834298:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 361 00002000:00100000:3.0:1788619905.834307:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5027100 x1875503571309440/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:1/0 lens 488/4536 e 0 to 0 dl 1788619916 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619905.834316:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5027100 with x1875503571309440 00000100:00100000:2.0:1788619905.847681:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619905.847789:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619905.848019:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571309696 00000100:00100000:3.0:1788619905.848033:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571309696 00002000:00100000:3.0:1788619905.848101:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78dfa41500 x1875503571309696/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:1/0 lens 504/0 e 0 to 0 dl 1788619916 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619905.848114:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78dfa41500 with x1875503571309696 00000100:00100000:3.0:1788619905.848165:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 362 00000100:00100000:3.0:1788619905.848176:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa41500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2355:x1875503571309696:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619905.852473:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571309568 00000100:00100000:0.0:1788619905.852495:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571309568 00002000:00100000:0.0:1788619905.852591:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1785855) req@ffff8e78c4a4f100 x1875503571309568/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:1/0 lens 488/0 e 0 to 0 dl 1788619916 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619905.852604:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c4a4f100 with x1875503571309568 ext(1048576->1785855) 00002000:00010000:0.0:1788619905.852612:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c4a4f100 00000100:00100000:0.0:1788619905.852727:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 255 00000100:00100000:0.0:1788619905.852739:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571309568:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619905.862457:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa41500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2355:x1875503571309696:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 14265us (14650us total) trans 0 rc 729088/729088 00000100:00100000:3.0:1788619905.862502:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 362 00002000:00100000:3.0:1788619905.862520:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78dfa41500 x1875503571309696/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:1/0 lens 504/440 e 0 to 0 dl 1788619916 ref 1 fl Complete:/600/0 rc 729088/729088 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619905.862532:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78dfa41500 with x1875503571309696 00000100:00100000:0.0:1788619905.866162:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571309568:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 13409us (18480us total) trans 0 rc 28672/28672 00000100:00100000:0.0:1788619905.866179:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 255 00002000:00100000:0.0:1788619905.866196:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1785855) req@ffff8e78c4a4f100 x1875503571309568/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:1/0 lens 488/440 e 0 to 0 dl 1788619916 ref 1 fl Complete:/600/0 rc 28672/28672 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619905.866207:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c4a4f100 with x1875503571309568 ext(1048576->1785855) 00002000:00010000:0.0:1788619905.866215:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c4a4f100 00000100:00100000:2.0:1788619905.873505:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619905.873552:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619905.873681:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571309952 00000100:00100000:1.0:1788619905.873695:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571309952 00000100:00100000:1.0:1788619905.873760:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 494 00000100:00100000:1.0:1788619905.873771:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa42d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571309952:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619905.873797:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619905.873829:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d036776/0x1cd7fe5368733b0e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b141 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619905.873905:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c74a8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589041152:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619905.874027:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd9639c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619905.875887:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571309824 00000100:00100000:3.0:1788619905.875904:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571309824 00000100:00100000:3.0:1788619905.875990:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 675 00000100:00100000:3.0:1788619905.876002:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa43800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571309824:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619905.876033:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619905.876059:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe5368733b15 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b13a expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619905.876104:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe5368733b15 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b13a expref: 12 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788619905.876119:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 782531 00010000:00010000:3.0:1788619905.876128:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe5368733b15 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b13a expref: 12 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619905.876143:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619905.876146:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619905.876152:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683da00, rc 0) 00000100:00100000:3.0:1788619905.876221:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa43800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571309824:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 221us (2732us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619905.876228:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 675 00002000:00010000:0.0:1788619905.876770:0:6696:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size: 1076053 -> 1973002 00002000:00010000:0.0:1788619905.876776:0:6696:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619904 -> 1788619905 00000100:00100000:0.0:1788619905.876802:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c74a8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589041152:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619905.876844:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005d036776/0x1cd7fe5368733b0e lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b141 expref: 12 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.876868:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d036776/0x1cd7fe5368733b0e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b141 expref: 12 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.876883:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5023400, rc 0) 00000100:00100000:0.0:1788619905.876924:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa42d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571309952:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3154us (3370us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619905.876951:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 494 00000100:00100000:2.0:1788619905.882561:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619905.882696:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571310080 00000100:00100000:0.0:1788619905.882710:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571310080 00000100:00100000:0.0:1788619905.882788:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 495 00000100:00100000:0.0:1788619905.882800:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa42300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571310080:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619905.882833:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619905.882858:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002aaba2be/0x1cd7fe5368733b1c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b148 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619905.882940:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed60e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589041280:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619905.883061:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa600 going to sleep for 11 seconds 00000100:00100000:0.0:1788619905.887197:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed60e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589041280:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619905.887271:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002aaba2be/0x1cd7fe5368733b1c lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b148 expref: 12 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.887309:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002aaba2be/0x1cd7fe5368733b1c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b148 expref: 12 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.887369:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78ccca6200, rc 0) 00000100:00100000:0.0:1788619905.887602:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa42300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571310080:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4802us (5056us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619905.887613:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 495 00000100:00100000:2.0:1788619905.893966:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619905.894079:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571310208 00000100:00100000:0.0:1788619905.894093:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571310208 00000100:00100000:0.0:1788619905.894181:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 496 00000100:00100000:0.0:1788619905.894192:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa43100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2352:x1875503571310208:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619905.894217:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619905.894241:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002aaba2be/0x1cd7fe5368733b23 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b14f expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619905.894320:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed60e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589041408:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619905.894436:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa600 going to sleep for 11 seconds 00000100:00100000:0.0:1788619905.897318:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed60e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589041408:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619905.897513:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002aaba2be/0x1cd7fe5368733b23 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b14f expref: 12 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.897546:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002aaba2be/0x1cd7fe5368733b23 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b14f expref: 12 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.897565:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78ccca6200, rc 0) 00000100:00100000:1.0:1788619905.897722:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa43100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2352:x1875503571310208:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3528us (3768us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619905.897734:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 496 00000100:00100000:2.0:1788619905.901446:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619905.901578:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571310336 00000100:00100000:3.0:1788619905.901612:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571310336 00000100:00100000:3.0:1788619905.901678:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 497 00000100:00100000:3.0:1788619905.901690:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa40700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571310336:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619905.901716:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619905.901742:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe5368733b2a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b156 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619905.901782:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000062c34eb6/0x1cd7fe5368733a19 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b068 expref: 7 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619905.901792:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=1s v=(3 1) 00010000:00010000:3.0:1788619905.901797:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(4 1) 00010000:00010000:3.0:1788619905.901816:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe5368733b2a lrc: 4/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1921024->1925119] (req 1921024->1925119) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b156 expref: 12 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619905.901862:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000062c34eb6/0x1cd7fe5368733a19 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b068 expref: 7 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619905.901880:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000062c34eb6/0x1cd7fe5368733a19 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b068 expref: 7 pid: 6697 timeout: 498 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619905.901897:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6531c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589041536:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619905.902022:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a240 going to sleep for 11 seconds 00000100:00100000:3.0:1788619905.905585:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6531c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589041536:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619905.905645:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe5368733b2a lrc: 3/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1921024->1925119] (req 1921024->1925119) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b156 expref: 12 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619905.905701:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c200, rc 0) 00000100:00100000:3.0:1788619905.905766:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa40700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:14908:x1875503571310336:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4079us (4334us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619905.905776:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 497 00000100:00100000:2.0:1788619905.908448:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619905.908546:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571310464 00000100:00100000:0.0:1788619905.908560:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571310464 00002000:00100000:0.0:1788619905.908635:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1957888->1973001) req@ffff8e78dfa40000 x1875503571310464/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:1/0 lens 15608/0 e 0 to 0 dl 1788619916 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619905.908648:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa40000 with x1875503571310464 ext(1957888->1973001) 00010000:00010000:0.0:1788619905.908661:0:6702:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 0000000062c34eb6/0x1cd7fe5368733a19 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b068 expref: 8 pid: 6697 timeout: 498 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.908684:0:6702:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 0000000062c34eb6/0x1cd7fe5368733a19 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b068 expref: 8 pid: 6697 timeout: 498 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788619905.908708:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff8e78dfa40000 00000100:00100000:0.0:1788619905.908750:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 6 00000100:00100000:0.0:1788619905.908761:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa40000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571310464:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:0.0:1788619905.909823:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa40000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+10:2354:x1875503571310464:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 1056us (1379us total) trans 4294967367 rc 0/0 00000100:00100000:0.0:1788619905.909834:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 6 00002000:00100000:0.0:1788619905.909843:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1957888->1973001) req@ffff8e78dfa40000 x1875503571310464/t4294967367(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:1/0 lens 15608/448 e 0 to 0 dl 1788619916 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619905.909852:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa40000 with x1875503571310464 ext(1957888->1973001) 00010000:00010000:0.0:1788619905.909860:0:6702:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 0000000062c34eb6/0x1cd7fe5368733a19 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b068 expref: 10 pid: 6697 timeout: 498 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.909873:0:6702:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 0000000062c34eb6/0x1cd7fe5368733a19 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b068 expref: 10 pid: 6697 timeout: 498 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788619905.909878:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff8e78dfa40000 00000100:00100000:2.0:1788619905.914429:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619905.914521:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571310592 00000100:00100000:0.0:1788619905.914532:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571310592 00010000:00010000:0.0:1788619905.914585:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000062c34eb6/0x1cd7fe5368733a19 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b068 expref: 10 pid: 6697 timeout: 498 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619905.914635:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 219 00000100:00100000:0.0:1788619905.914646:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa40380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+10:2354:x1875503571310592:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619905.914696:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619905.914726:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000062c34eb6/0x1cd7fe5368733a19 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b068 expref: 10 pid: 6697 timeout: 498 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.914742:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000062c34eb6/0x1cd7fe5368733a19 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b068 expref: 10 pid: 6697 timeout: 498 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.914770:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062c34eb6/0x1cd7fe5368733a19 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b068 expref: 10 pid: 6697 timeout: 498 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.914792:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(4 1) 00010000:00010000:0.0:1788619905.914797:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(4 1) 00010000:00010000:0.0:1788619905.914964:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe5368733b2a lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1921024->1925119) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b156 expref: 11 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.914989:0:13330:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe5368733b2a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1921024->1925119) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b156 expref: 11 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.915043:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe5368733b2a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1921024->1925119) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b156 expref: 11 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619905.915057:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed60e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589041664:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619905.915178:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa600 going to sleep for 11 seconds 00000100:00100000:2.0:1788619905.921002:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619905.921056:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619905.921202:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571310848 00000100:00100000:3.0:1788619905.921215:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571310848 00002000:00100000:3.0:1788619905.921287:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78dfa43480 x1875503571310848/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:1/0 lens 488/0 e 0 to 0 dl 1788619916 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619905.921297:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78dfa43480 with x1875503571310848 00000100:00100000:3.0:1788619905.921358:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 363 00000100:00100000:3.0:1788619905.921367:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa43480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571310848:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619905.921863:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed60e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589041664:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619905.921888:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619905.921943:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa40380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571310592:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 7299us (7527us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619905.921951:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 219 00000100:00100000:1.0:1788619905.922173:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571310720 00000100:00100000:1.0:1788619905.922179:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571310720 00002000:00100000:1.0:1788619905.922221:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1974271) req@ffff8e78dfa42a00 x1875503571310720/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:1/0 lens 488/0 e 0 to 0 dl 1788619916 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619905.922234:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa42a00 with x1875503571310720 ext(1048576->1974271) 00002000:00010000:1.0:1788619905.922248:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa42a00 00000100:00100000:1.0:1788619905.922270:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 256 00000100:00100000:1.0:1788619905.922277:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571310720:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619905.934185:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa43480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571310848:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 12526us (12837us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619905.934204:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 363 00002000:00100000:3.0:1788619905.934221:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78dfa43480 x1875503571310848/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:1/0 lens 488/440 e 0 to 0 dl 1788619916 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619905.934232:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78dfa43480 with x1875503571310848 00000100:00100000:1.0:1788619905.937707:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571310720:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 15412us (16714us total) trans 0 rc 925696/925696 00000100:00100000:1.0:1788619905.937725:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 256 00002000:00100000:1.0:1788619905.937741:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1974271) req@ffff8e78dfa42a00 x1875503571310720/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:1/0 lens 488/440 e 0 to 0 dl 1788619916 ref 1 fl Complete:/600/0 rc 925696/925696 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619905.937752:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa42a00 with x1875503571310720 ext(1048576->1974271) 00002000:00010000:1.0:1788619905.937760:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa42a00 00000100:00100000:3.0:1788619905.942725:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619905.942907:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571310976 00000100:00100000:1.0:1788619905.942923:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571310976 00000100:00100000:1.0:1788619905.943067:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 498 00000100:00100000:1.0:1788619905.943080:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6531c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571310976:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619905.943104:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619905.943129:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d036776/0x1cd7fe5368733b31 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b15d expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619905.943162:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(4 1) 00010000:00010000:1.0:1788619905.943167:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(4 1) 00010000:00010000:1.0:1788619905.943179:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d036776/0x1cd7fe5368733b31 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b15d expref: 10 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619905.943227:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005d036776/0x1cd7fe5368733b31 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b15d expref: 10 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619905.943255:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5023400, rc 0) 00000100:00100000:1.0:1788619905.943389:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6531c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+10:2354:x1875503571310976:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 312us (680us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619905.943396:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 498 00000100:00100000:3.0:1788619905.947837:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619905.948321:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571311104 00000100:00100000:2.0:1788619905.948370:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571311104 00000100:00100000:0.0:1788619905.948693:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 499 00000100:00100000:0.0:1788619905.948708:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6532680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:14908:x1875503571311104:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619905.948732:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619905.948755:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002aaba2be/0x1cd7fe5368733b38 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b164 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.948810:0:6698:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe5368733b2a lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1921024->1925119) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b156 expref: 13 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.948823:0:6698:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000005d036776/0x1cd7fe5368733b31 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b15d expref: 9 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.948831:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(4 1) 00010000:00010000:0.0:1788619905.948837:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(4 1) 00010000:00010000:0.0:1788619905.948842:0:6698:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002aaba2be/0x1cd7fe5368733b38 lrc: 5/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [1462272->1466367] (req 1462272->1466367) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b164 expref: 13 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.948889:0:6698:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000005d036776/0x1cd7fe5368733b31 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b15d expref: 9 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.948906:0:6698:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000005d036776/0x1cd7fe5368733b31 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b15d expref: 9 pid: 6696 timeout: 498 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619905.948939:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed60e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589041792:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619905.949076:0:6698:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe5368733b2a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1921024->1925119) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b156 expref: 13 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.949087:0:6698:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe5368733b2a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1921024->1925119) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b156 expref: 13 pid: 6697 timeout: 498 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619905.949098:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed61180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589041920:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619905.949124:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa600 going to sleep for 11 seconds 00000100:00100000:0.0:1788619905.951969:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed60e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589041792:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619905.952002:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed61180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589041920:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619905.952052:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002aaba2be/0x1cd7fe5368733b38 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [1462272->1466367] (req 1462272->1466367) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b164 expref: 13 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.952083:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78ccca6200, rc 0) 00000100:00100000:0.0:1788619905.952159:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6532680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+13:14908:x1875503571311104:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3454us (4337us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619905.952171:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 499 00000100:00100000:2.0:1788619905.958893:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619905.958941:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619905.959021:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571311232 00000100:00100000:0.0:1788619905.959035:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571311232 00010000:00010000:0.0:1788619905.959090:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000005d036776/0x1cd7fe5368733b31 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b15d expref: 10 pid: 6696 timeout: 498 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619905.959160:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571311360 00000100:00100000:0.0:1788619905.959164:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571311360 00010000:00010000:0.0:1788619905.959182:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe5368733b2a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1921024->1925119) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b156 expref: 13 pid: 6697 timeout: 498 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619905.959211:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 220 00000100:00100000:0.0:1788619905.959223:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa41c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+10:2354:x1875503571311232:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619905.959252:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619905.959283:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000005d036776/0x1cd7fe5368733b31 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b15d expref: 10 pid: 6696 timeout: 498 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.959300:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000005d036776/0x1cd7fe5368733b31 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b15d expref: 10 pid: 6696 timeout: 498 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.959358:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d036776/0x1cd7fe5368733b31 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b15d expref: 10 pid: 6696 timeout: 498 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.959380:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619905.959439:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa41c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571311232:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 219us (561us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619905.959447:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 220 00000100:00100000:0.0:1788619905.959466:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 221 00000100:00100000:0.0:1788619905.959472:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa40e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+13:2355:x1875503571311360:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619905.959494:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619905.959503:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe5368733b2a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1921024->1925119) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b156 expref: 13 pid: 6697 timeout: 498 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.959514:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe5368733b2a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1921024->1925119) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b156 expref: 13 pid: 6697 timeout: 498 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.959530:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe5368733b2a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1921024->1925119) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b156 expref: 13 pid: 6697 timeout: 498 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.959541:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(4 1) 00010000:00010000:0.0:1788619905.959546:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(4 1) 00010000:00010000:0.0:1788619905.959556:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000002aaba2be/0x1cd7fe5368733b38 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1462272->1466367) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b164 expref: 12 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.959566:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002aaba2be/0x1cd7fe5368733b38 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1462272->1466367) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b164 expref: 12 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619905.959611:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000002aaba2be/0x1cd7fe5368733b38 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1462272->1466367) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b164 expref: 12 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619905.959628:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed61180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589042048:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619905.959693:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa600 going to sleep for 11 seconds 00000100:00100000:0.0:1788619905.962612:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed61180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589042048:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619905.962648:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619905.962744:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa40e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571311360:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3269us (3800us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619905.962758:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 221 00000100:00100000:2.0:1788619905.979477:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619905.979590:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571311488 00000100:00100000:0.0:1788619905.979604:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571311488 00002000:00100000:0.0:1788619905.979676:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1462272->1466367) req@ffff8e78dfa41180 x1875503571311488/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:1/0 lens 488/0 e 0 to 0 dl 1788619916 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619905.979690:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa41180 with x1875503571311488 ext(1462272->1466367) 00002000:00010000:0.0:1788619905.979698:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa41180 00000100:00100000:0.0:1788619905.979740:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 257 00000100:00100000:0.0:1788619905.979752:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa41180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2353:x1875503571311488:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619905.980119:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa41180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2353:x1875503571311488:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 366us (655us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619905.980128:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 257 00002000:00100000:0.0:1788619905.980136:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1462272->1466367) req@ffff8e78dfa41180 x1875503571311488/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:1/0 lens 488/4536 e 0 to 0 dl 1788619916 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619905.980145:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa41180 with x1875503571311488 ext(1462272->1466367) 00002000:00010000:0.0:1788619905.980149:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa41180 00000100:00100000:2.0:1788619905.990906:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619905.991041:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571311616 00000100:00100000:1.0:1788619905.991055:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571311616 00002000:00100000:1.0:1788619905.991125:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1466368->1470463) req@ffff8e78dfa40a80 x1875503571311616/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:1/0 lens 488/0 e 0 to 0 dl 1788619916 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619905.991138:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa40a80 with x1875503571311616 ext(1466368->1470463) 00002000:00010000:1.0:1788619905.991147:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa40a80 00000100:00100000:1.0:1788619905.991193:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 258 00000100:00100000:1.0:1788619905.991204:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa40a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571311616:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619905.991597:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa40a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571311616:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 392us (705us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619905.991605:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 258 00002000:00100000:1.0:1788619905.991613:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1466368->1470463) req@ffff8e78dfa40a80 x1875503571311616/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:1/0 lens 488/4536 e 0 to 0 dl 1788619916 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619905.991622:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa40a80 with x1875503571311616 ext(1466368->1470463) 00002000:00010000:1.0:1788619905.991627:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa40a80 00000100:00100000:1.0:1788619905.996694:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619905.996913:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571311744 00000100:00100000:2.0:1788619905.996928:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571311744 00002000:00100000:2.0:1788619905.997138:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1470464->1474559) req@ffff8e77c74a8380 x1875503571311744/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:1/0 lens 488/0 e 0 to 0 dl 1788619916 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619905.997152:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c74a8380 with x1875503571311744 ext(1470464->1474559) 00002000:00010000:2.0:1788619905.997159:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c74a8380 00000100:00100000:2.0:1788619905.997201:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 259 00000100:00100000:2.0:1788619905.997214:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74a8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571311744:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619905.997829:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74a8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571311744:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 616us (1148us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619905.997840:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 259 00002000:00100000:2.0:1788619905.997847:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1470464->1474559) req@ffff8e77c74a8380 x1875503571311744/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:1/0 lens 488/4536 e 0 to 0 dl 1788619916 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619905.997857:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c74a8380 with x1875503571311744 ext(1470464->1474559) 00002000:00010000:2.0:1788619905.997862:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c74a8380 00000100:00100000:2.0:1788619906.003215:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619906.003371:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571311872 00000100:00100000:0.0:1788619906.003385:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571311872 00002000:00100000:0.0:1788619906.003460:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1474560->1478655) req@ffff8e78dfa41f80 x1875503571311872/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:2/0 lens 488/0 e 0 to 0 dl 1788619917 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619906.003474:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa41f80 with x1875503571311872 ext(1474560->1478655) 00002000:00010000:0.0:1788619906.003480:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa41f80 00000100:00100000:0.0:1788619906.003519:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 260 00000100:00100000:0.0:1788619906.003545:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa41f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571311872:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619906.003908:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa41f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571311872:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 364us (705us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619906.003917:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 260 00002000:00100000:0.0:1788619906.003924:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1474560->1478655) req@ffff8e78dfa41f80 x1875503571311872/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:2/0 lens 488/4536 e 0 to 0 dl 1788619917 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619906.003934:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa41f80 with x1875503571311872 ext(1474560->1478655) 00002000:00010000:0.0:1788619906.003938:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa41f80 00000100:00080000:1.0:1788619906.004194:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1788619906.010831:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619906.011002:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571312000 00000100:00100000:2.0:1788619906.011024:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571312000 00002000:00100000:2.0:1788619906.011093:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1478656->1482751) req@ffff8e77c6533b80 x1875503571312000/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:2/0 lens 488/0 e 0 to 0 dl 1788619917 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619906.011106:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6533b80 with x1875503571312000 ext(1478656->1482751) 00002000:00010000:2.0:1788619906.011136:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6533b80 00000100:00100000:2.0:1788619906.011178:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 261 00000100:00100000:2.0:1788619906.011190:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6533b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571312000:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619906.011779:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6533b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571312000:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 589us (958us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619906.011789:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 261 00002000:00100000:2.0:1788619906.011808:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1478656->1482751) req@ffff8e77c6533b80 x1875503571312000/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:2/0 lens 488/4536 e 0 to 0 dl 1788619917 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619906.011820:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6533b80 with x1875503571312000 ext(1478656->1482751) 00002000:00010000:2.0:1788619906.011825:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6533b80 00000100:00100000:3.0:1788619906.019576:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619906.020050:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571312128 00000100:00100000:2.0:1788619906.020065:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571312128 00002000:00100000:2.0:1788619906.020134:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1482752->1486847) req@ffff8e77c6530a80 x1875503571312128/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:2/0 lens 488/0 e 0 to 0 dl 1788619917 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619906.020147:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6530a80 with x1875503571312128 ext(1482752->1486847) 00002000:00010000:2.0:1788619906.020154:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6530a80 00000100:00100000:2.0:1788619906.020186:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 262 00000100:00100000:2.0:1788619906.020198:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6530a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571312128:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619906.020592:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6530a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571312128:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 394us (1030us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619906.020602:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 262 00002000:00100000:2.0:1788619906.020609:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1482752->1486847) req@ffff8e77c6530a80 x1875503571312128/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:2/0 lens 488/4536 e 0 to 0 dl 1788619917 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619906.020618:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6530a80 with x1875503571312128 ext(1482752->1486847) 00002000:00010000:2.0:1788619906.020643:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6530a80 00000100:00100000:0.0:1788619906.031667:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619906.032077:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571312256 00000100:00100000:2.0:1788619906.032093:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571312256 00002000:00100000:2.0:1788619906.032166:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1486848->1490943) req@ffff8e78fed61180 x1875503571312256/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:2/0 lens 488/0 e 0 to 0 dl 1788619917 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619906.032181:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fed61180 with x1875503571312256 ext(1486848->1490943) 00002000:00010000:2.0:1788619906.032188:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fed61180 00000100:00100000:2.0:1788619906.032233:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 263 00000100:00100000:2.0:1788619906.032252:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed61180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2352:x1875503571312256:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619906.032661:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed61180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2352:x1875503571312256:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 415us (1011us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619906.032670:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 263 00002000:00100000:2.0:1788619906.032678:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1486848->1490943) req@ffff8e78fed61180 x1875503571312256/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:2/0 lens 488/4536 e 0 to 0 dl 1788619917 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619906.032687:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fed61180 with x1875503571312256 ext(1486848->1490943) 00002000:00010000:2.0:1788619906.032691:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fed61180 00000100:00100000:3.0:1788619906.037227:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619906.037417:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571312384 00000100:00100000:0.0:1788619906.037431:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571312384 00002000:00100000:0.0:1788619906.037494:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1490944->1495039) req@ffff8e77c6530000 x1875503571312384/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:2/0 lens 488/0 e 0 to 0 dl 1788619917 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619906.037508:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6530000 with x1875503571312384 ext(1490944->1495039) 00002000:00010000:0.0:1788619906.037515:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6530000 00000100:00100000:0.0:1788619906.037556:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 264 00000100:00100000:0.0:1788619906.037566:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6530000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571312384:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619906.037894:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6530000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571312384:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 328us (680us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619906.037903:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 264 00002000:00100000:0.0:1788619906.037910:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1490944->1495039) req@ffff8e77c6530000 x1875503571312384/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:2/0 lens 488/4536 e 0 to 0 dl 1788619917 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619906.037919:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6530000 with x1875503571312384 ext(1490944->1495039) 00002000:00010000:0.0:1788619906.037922:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6530000 00000100:00100000:3.0:1788619906.042296:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619906.042531:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571312512 00000100:00100000:1.0:1788619906.042545:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571312512 00002000:00100000:1.0:1788619906.042621:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1495040->1499135) req@ffff8e77c6531500 x1875503571312512/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:2/0 lens 488/0 e 0 to 0 dl 1788619917 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619906.042636:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6531500 with x1875503571312512 ext(1495040->1499135) 00002000:00010000:1.0:1788619906.042643:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6531500 00000100:00100000:1.0:1788619906.042674:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 265 00000100:00100000:1.0:1788619906.042685:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6531500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2353:x1875503571312512:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619906.043039:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6531500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2353:x1875503571312512:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 354us (758us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619906.043048:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 265 00002000:00100000:1.0:1788619906.043056:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1495040->1499135) req@ffff8e77c6531500 x1875503571312512/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:2/0 lens 488/4536 e 0 to 0 dl 1788619917 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619906.043064:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6531500 with x1875503571312512 ext(1495040->1499135) 00002000:00010000:1.0:1788619906.043068:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6531500 00000100:00100000:1.0:1788619906.047734:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619906.047868:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571312640 00000100:00100000:2.0:1788619906.047884:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571312640 00002000:00100000:2.0:1788619906.047959:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1462272->1499135) req@ffff8e77c6b13100 x1875503571312640/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:2/0 lens 488/0 e 0 to 0 dl 1788619917 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619906.047993:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6b13100 with x1875503571312640 ext(1462272->1499135) 00002000:00010000:2.0:1788619906.048002:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6b13100 00000100:00100000:2.0:1788619906.048045:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 266 00000100:00100000:2.0:1788619906.048059:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b13100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571312640:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:2.0:1788619906.053510:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b13100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+14:2355:x1875503571312640:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 5437us (5777us total) trans 4294967368 rc 0/0 00000100:00100000:2.0:1788619906.053530:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 266 00002000:00100000:2.0:1788619906.053548:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1462272->1499135) req@ffff8e77c6b13100 x1875503571312640/t4294967368(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:2/0 lens 488/448 e 0 to 0 dl 1788619917 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619906.053561:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6b13100 with x1875503571312640 ext(1462272->1499135) 00002000:00010000:2.0:1788619906.053570:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6b13100 00000100:00100000:1.0:1788619906.057618:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619906.057734:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571312768 00000100:00100000:0.0:1788619906.057749:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571312768 00000100:00100000:0.0:1788619906.057821:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 500 00000100:00100000:0.0:1788619906.057833:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b10a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+14:2353:x1875503571312768:12345-192.168.201.5@tcp:16:fsx.0 00000001:00080000:2.0:1788619906.126836:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967367 is committed 00000001:00080000:2.0:1788619906.126888:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967364 is committed 00000001:00080000:2.0:1788619906.126907:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967368 is committed 00000100:00100000:0.0:1788619906.127511:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b10a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571312768:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 69669us (69900us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619906.127528:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 500 00000100:00100000:3.0:1788619906.138279:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619906.138805:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571312896 00000100:00100000:2.0:1788619906.138820:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571312896 00000100:00100000:2.0:1788619906.138932:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 676 00000100:00100000:2.0:1788619906.138946:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6531880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+19:14908:x1875503571312896:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619906.138991:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619906.138996:0:11397:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1788619906.139063:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505c200/0x1cd7fe5368733b07 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b133 expref: 19 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.139088:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619906.139096:0:11397:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788619906.139112:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7200/0x1cd7fe5368733b3f lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b16b expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.139145:0:11397:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe5368733b15 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b13a expref: 11 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.139171:0:11397:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7200/0x1cd7fe5368733b3f lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b16b expref: 19 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.139223:0:11397:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe5368733b15 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b13a expref: 11 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.139251:0:11397:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe5368733b15 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b13a expref: 11 pid: 5844 timeout: 498 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619906.139269:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c8597100 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589042176:192.168.201.5@tcp:104: 00000100:00080000:1.0:1788619906.139401:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1788619906.139888:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc480 going to sleep for 11 seconds 00000100:00100000:2.0:1788619906.155018:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c8597100 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589042176:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619906.155075:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7200/0x1cd7fe5368733b3f lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b16b expref: 19 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.155100:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619906.155103:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619906.155108:0:11397:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7200/0x1cd7fe5368733b3f lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b16b expref: 19 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.155121:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7200, rc 0) 00000100:00100000:2.0:1788619906.155547:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6531880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+19:14908:x1875503571312896:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 16602us (17280us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619906.155559:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 676 00000100:00100000:3.0:1788619906.156561:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619906.157247:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571313024 00000100:00100000:2.0:1788619906.157274:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571313024 00000100:00080000:0.0:1788619906.157456:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:2.0:1788619906.157550:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe5368733b15 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b13a expref: 12 pid: 5844 timeout: 498 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619906.157693:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 222 00000100:00100000:2.0:1788619906.157705:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8021050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571313024:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619906.157723:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619906.157757:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe5368733b15 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b13a expref: 12 pid: 5844 timeout: 498 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.158145:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe5368733b15 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b13a expref: 12 pid: 5844 timeout: 498 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.158175:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe5368733b15 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b13a expref: 12 pid: 5844 timeout: 498 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.158194:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619906.158200:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7200/0x1cd7fe5368733b3f lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b16b expref: 18 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.158212:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7200/0x1cd7fe5368733b3f lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b16b expref: 18 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.158222:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7200/0x1cd7fe5368733b3f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b16b expref: 18 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.159472:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7200/0x1cd7fe5368733b3f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b16b expref: 18 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619906.159532:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c8597100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589042304:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619906.159660:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc480 going to sleep for 11 seconds 00000100:00100000:3.0:1788619906.164943:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619906.165109:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571313152 00000100:00100000:0.0:1788619906.165122:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571313152 00000100:00100000:0.0:1788619906.165215:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 364 00000100:00100000:0.0:1788619906.165228:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6531f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2354:x1875503571313152:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:0.0:1788619906.165631:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 782531 -> 782531 00000004:00010000:0.0:1788619906.165637:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619905 -> 1788619905 00000004:00010000:0.0:1788619906.165640:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619904 -> 1788619904 00000004:00010000:0.0:1788619906.165642:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619905 -> 1788619905 00000004:00010000:0.0:1788619906.165645:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 774 -> 774 00000100:00100000:0.0:1788619906.165738:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6531f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2354:x1875503571313152:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 512us (811us total) trans 4294967562 rc 0/0 00000100:00100000:0.0:1788619906.165748:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 364 00000100:00100000:2.0:1788619906.165992:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c8597100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589042304:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619906.166021:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619906.166050:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8021050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571313024:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 8349us (9497us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619906.166060:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 222 00000100:00100000:3.0:1788619906.170049:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619906.175035:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571313280 00000100:00100000:2.0:1788619906.175058:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571313280 00000100:00100000:2.0:1788619906.175154:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 677 00000100:00100000:2.0:1788619906.175168:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5019500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571313280:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619906.175199:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619906.175226:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe5368733b46 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b172 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.175288:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7200/0x1cd7fe5368733b3f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b16b expref: 19 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619906.175373:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c8597100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589042432:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619906.175498:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc180 going to sleep for 11 seconds 00000004:00010000:2.0:1788619906.179064:0:9417:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619904 -> 1788619905 00000004:00010000:2.0:1788619906.179078:0:9417:0:(mdt_lvb.c:195:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime: 1788619905 -> 1788619906 00000100:00100000:2.0:1788619906.179113:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c8597100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589042432:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619906.179147:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 782531 00010000:00010000:2.0:1788619906.179191:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe5368733b46 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b172 expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.179220:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619906.179224:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619906.179231:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe5368733b46 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b172 expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.179544:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6000, rc 0) 00000100:00100000:2.0:1788619906.179848:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5019500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571313280:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4681us (10046us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619906.179858:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 677 00000100:00100000:0.0:1788619906.192081:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619906.192215:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571313408 00000100:00100000:1.0:1788619906.192229:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571313408 00000100:00100000:1.0:1788619906.192569:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 678 00000100:00100000:1.0:1788619906.192583:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed60e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2352:x1875503571313408:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619906.192610:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619906.192641:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5022800/0x1cd7fe5368733b4d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b179 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619906.192725:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7200/0x1cd7fe5368733b3f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b16b expref: 19 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619906.192787:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b13800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589042560:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619906.192992:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd9639c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619906.198767:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b13800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589042560:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619906.198820:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 782531 00010000:00010000:1.0:1788619906.198847:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5022800/0x1cd7fe5368733b4d lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b179 expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619906.198875:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619906.198878:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619906.198884:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5022800/0x1cd7fe5368733b4d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b179 expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619906.198903:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5022800, rc 0) 00000100:00100000:1.0:1788619906.199005:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed60e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2352:x1875503571313408:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6423us (6938us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619906.199015:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 678 00000100:00100000:2.0:1788619906.204959:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619906.205140:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571313536 00000100:00100000:3.0:1788619906.205153:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571313536 00000100:00100000:3.0:1788619906.205234:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 679 00000100:00100000:3.0:1788619906.205255:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8597100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+11:2352:x1875503571313536:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619906.205284:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619906.205308:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368733b54 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b180 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619906.205409:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7200/0x1cd7fe5368733b3f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b16b expref: 19 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619906.205469:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589042688:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619906.205590:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cd25f540 going to sleep for 11 seconds 00000100:00100000:3.0:1788619906.208849:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589042688:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619906.208883:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 782531 00010000:00010000:3.0:1788619906.208907:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368733b54 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b180 expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619906.208932:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619906.208935:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619906.208940:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368733b54 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b180 expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619906.208958:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d000, rc 0) 00000100:00100000:3.0:1788619906.209070:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8597100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+11:2352:x1875503571313536:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3817us (4126us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619906.209080:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 679 00000100:00100000:2.0:1788619906.213393:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619906.213632:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571313664 00000100:00100000:3.0:1788619906.213647:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571313664 00000100:00100000:3.0:1788619906.213715:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 680 00000100:00100000:3.0:1788619906.213727:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8597800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571313664:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619906.213755:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619906.213780:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368733b5b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b187 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619906.213842:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7200/0x1cd7fe5368733b3f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b16b expref: 19 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619906.213899:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589042816:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619906.214225:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cd25f540 going to sleep for 11 seconds 00000100:00100000:3.0:1788619906.217384:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589042816:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619906.217423:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 782531 00010000:00010000:3.0:1788619906.217450:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368733b5b lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b187 expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619906.217477:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619906.217480:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619906.217488:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368733b5b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b187 expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619906.217508:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d600, rc 0) 00000100:00100000:3.0:1788619906.217613:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8597800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571313664:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3888us (4238us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619906.217625:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 680 00000100:00100000:0.0:1788619906.223474:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619906.223694:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571313792 00000100:00100000:3.0:1788619906.223708:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571313792 00000100:00100000:3.0:1788619906.223815:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 501 00000100:00100000:3.0:1788619906.223829:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed62a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571313792:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619906.223853:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619906.223877:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a803caae/0x1cd7fe5368733b62 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b18e expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619906.223986:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589042944:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619906.224103:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cd25f540 going to sleep for 11 seconds 00000100:00080000:0.0:1788619906.226882:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:3.0:1788619906.227745:0:6697:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime: 1788619905 -> 1788619906 00000100:00100000:3.0:1788619906.227784:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589042944:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619906.227844:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a803caae/0x1cd7fe5368733b62 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b18e expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619906.227878:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a803caae/0x1cd7fe5368733b62 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b18e expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619906.227895:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d000, rc 0) 00000100:00100000:3.0:1788619906.228189:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed62a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571313792:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4361us (4731us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619906.228200:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 501 00000100:00100000:2.0:1788619906.233578:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619906.233703:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571313920 00000100:00100000:0.0:1788619906.233718:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571313920 00000100:00100000:0.0:1788619906.233806:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 502 00000100:00100000:0.0:1788619906.233820:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8594a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571313920:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619906.233866:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619906.233890:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7278240/0x1cd7fe5368733b69 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b195 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619906.234000:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed62d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589043072:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619906.234119:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa780 going to sleep for 11 seconds 00000100:00100000:1.0:1788619906.239488:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed62d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589043072:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619906.239558:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d7278240/0x1cd7fe5368733b69 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b195 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619906.239593:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7278240/0x1cd7fe5368733b69 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b195 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619906.240446:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78ccca7400, rc 0) 00000100:00100000:1.0:1788619906.240601:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8594a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571313920:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6781us (7036us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619906.240613:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 502 00000100:00100000:2.0:1788619906.244786:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619906.245066:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571314048 00000100:00100000:3.0:1788619906.245080:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571314048 00002000:00100000:3.0:1788619906.245152:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c8594380 x1875503571314048/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:2/0 lens 488/0 e 0 to 0 dl 1788619917 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619906.245165:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c8594380 with x1875503571314048 00000100:00100000:3.0:1788619906.245365:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 365 00000100:00100000:3.0:1788619906.245377:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8594380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2355:x1875503571314048:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619906.245735:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8594380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2355:x1875503571314048:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 359us (963us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619906.245744:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 365 00002000:00100000:3.0:1788619906.245753:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c8594380 x1875503571314048/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:2/0 lens 488/440 e 0 to 0 dl 1788619917 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619906.245760:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c8594380 with x1875503571314048 00000100:00100000:2.0:1788619906.253900:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619906.254218:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571314176 00000100:00100000:0.0:1788619906.254231:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571314176 00000100:00100000:0.0:1788619906.254655:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 503 00000100:00100000:0.0:1788619906.254669:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8595880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571314176:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619906.254694:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619906.254719:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001eabdd3d/0x1cd7fe5368733b70 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b19c expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619906.254816:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed61f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589043200:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619906.254990:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:0.0:1788619906.258307:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed61f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589043200:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619906.258625:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001eabdd3d/0x1cd7fe5368733b70 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b19c expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.258660:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001eabdd3d/0x1cd7fe5368733b70 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b19c expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.258678:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78ccca7e00, rc 0) 00000100:00100000:0.0:1788619906.258755:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8595880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571314176:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4089us (4871us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619906.258786:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 503 00000100:00100000:3.0:1788619906.264233:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619906.264675:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571314304 00000100:00100000:2.0:1788619906.264690:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571314304 00000100:00100000:2.0:1788619906.264772:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 504 00000100:00100000:2.0:1788619906.264785:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571314304:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619906.264819:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619906.264844:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003a2d9d44/0x1cd7fe5368733b77 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b1a3 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619906.264930:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c8595500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589043328:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619906.265218:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc180 going to sleep for 11 seconds 00000100:00100000:1.0:1788619906.270845:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c8595500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589043328:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619906.270897:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003a2d9d44/0x1cd7fe5368733b77 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b1a3 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619906.270929:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a2d9d44/0x1cd7fe5368733b77 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b1a3 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619906.270946:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7600, rc 0) 00000100:00100000:1.0:1788619906.271040:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571314304:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6257us (6824us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619906.271051:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 504 00000100:00100000:1.0:1788619906.275194:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619906.275376:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571314432 00000100:00100000:2.0:1788619906.275391:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571314432 00002000:00100000:2.0:1788619906.275466:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b13800 x1875503571314432/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:2/0 lens 488/0 e 0 to 0 dl 1788619917 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619906.275478:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b13800 with x1875503571314432 00000100:00100000:2.0:1788619906.275526:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 366 00000100:00100000:2.0:1788619906.275538:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b13800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2355:x1875503571314432:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619906.275902:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b13800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2355:x1875503571314432:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 363us (723us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619906.275912:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 366 00002000:00100000:2.0:1788619906.275920:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b13800 x1875503571314432/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:2/0 lens 488/440 e 0 to 0 dl 1788619917 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619906.275928:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b13800 with x1875503571314432 00000100:00100000:3.0:1788619906.283778:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619906.284421:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571314560 00000100:00100000:2.0:1788619906.284436:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571314560 00000100:00100000:2.0:1788619906.284513:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 505 00000100:00100000:2.0:1788619906.284527:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571314560:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619906.284549:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619906.284574:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000092f9790d/0x1cd7fe5368733b7e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b1aa expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619906.284656:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c8594000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589043456:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619906.284771:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccb40 going to sleep for 11 seconds 00000100:00100000:2.0:1788619906.288691:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c8594000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589043456:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619906.288751:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000092f9790d/0x1cd7fe5368733b7e lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b1aa expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.288821:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000092f9790d/0x1cd7fe5368733b7e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b1aa expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.288840:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7800, rc 0) 00000100:00100000:2.0:1788619906.288924:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571314560:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4399us (5161us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619906.288933:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 505 00000100:00100000:1.0:1788619906.294759:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619906.294939:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571314688 00000100:00100000:0.0:1788619906.294953:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571314688 00000100:00100000:0.0:1788619906.295044:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 506 00000100:00100000:0.0:1788619906.295058:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b12300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571314688:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619906.295082:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619906.295107:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001eabdd3d/0x1cd7fe5368733b85 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b1b1 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619906.295188:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed61f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589043584:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619906.295716:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:0.0:1788619906.299604:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed61f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589043584:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619906.299669:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001eabdd3d/0x1cd7fe5368733b85 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b1b1 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.299702:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001eabdd3d/0x1cd7fe5368733b85 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b1b1 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.299720:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78ccca7e00, rc 0) 00000100:00100000:0.0:1788619906.299812:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b12300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571314688:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4755us (5066us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619906.299823:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 506 00000100:00100000:2.0:1788619906.304785:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619906.304882:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571314816 00000100:00100000:3.0:1788619906.304896:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571314816 00002000:00100000:3.0:1788619906.304983:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c8594000 x1875503571314816/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:2/0 lens 488/0 e 0 to 0 dl 1788619917 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619906.304997:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c8594000 with x1875503571314816 00000100:00100000:3.0:1788619906.305042:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 367 00000100:00100000:3.0:1788619906.305054:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8594000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2354:x1875503571314816:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619906.305771:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8594000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2354:x1875503571314816:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 715us (1000us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619906.305782:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 367 00002000:00100000:3.0:1788619906.305811:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c8594000 x1875503571314816/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:2/0 lens 488/440 e 0 to 0 dl 1788619917 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619906.305819:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c8594000 with x1875503571314816 00000100:00100000:1.0:1788619906.310672:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619906.310865:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571314944 00000100:00100000:0.0:1788619906.310877:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571314944 00000100:00100000:0.0:1788619906.311089:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 507 00000100:00100000:0.0:1788619906.311102:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b13b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571314944:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619906.311125:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619906.311147:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001eabdd3d/0x1cd7fe5368733b8c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b1b8 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619906.311228:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed61f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589043712:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619906.311491:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:0.0:1788619906.314567:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed61f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589043712:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619906.314624:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001eabdd3d/0x1cd7fe5368733b8c lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b1b8 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.314657:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001eabdd3d/0x1cd7fe5368733b8c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b1b8 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.314676:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78ccca7e00, rc 0) 00000100:00100000:0.0:1788619906.314756:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b13b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571314944:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3656us (4100us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619906.314767:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 507 00000100:00100000:1.0:1788619906.320290:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619906.320573:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571315072 00000100:00100000:2.0:1788619906.320587:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571315072 00000100:00100000:2.0:1788619906.320660:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 508 00000100:00100000:2.0:1788619906.320673:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b11c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571315072:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619906.320698:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619906.320722:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fb640cb5/0x1cd7fe5368733b93 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b1bf expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619906.320808:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c8595500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589043840:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619906.320920:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccb40 going to sleep for 11 seconds 00000100:00100000:2.0:1788619906.324897:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c8595500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589043840:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619906.325161:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000fb640cb5/0x1cd7fe5368733b93 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b1bf expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.325196:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fb640cb5/0x1cd7fe5368733b93 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b1bf expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.325232:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6e00, rc 0) 00000100:00100000:2.0:1788619906.325675:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b11c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571315072:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5001us (5397us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619906.325686:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 508 00000100:00100000:2.0:1788619906.330713:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619906.330893:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571315200 00000100:00100000:0.0:1788619906.331065:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571315200 00002000:00100000:0.0:1788619906.331143:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c8595500 x1875503571315200/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:2/0 lens 488/0 e 0 to 0 dl 1788619917 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619906.331156:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c8595500 with x1875503571315200 00000100:00100000:0.0:1788619906.331230:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 368 00000100:00100000:0.0:1788619906.331274:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8595500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2355:x1875503571315200:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619906.331674:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8595500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2355:x1875503571315200:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 400us (976us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619906.331682:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 368 00002000:00100000:0.0:1788619906.331690:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c8595500 x1875503571315200/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:2/0 lens 488/440 e 0 to 0 dl 1788619917 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619906.331698:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c8595500 with x1875503571315200 00000100:00100000:1.0:1788619906.336598:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619906.336694:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571315328 00000100:00100000:2.0:1788619906.336708:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571315328 00000100:00100000:2.0:1788619906.336803:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 509 00000100:00100000:2.0:1788619906.336816:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b11500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571315328:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619906.336843:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619906.336867:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fb640cb5/0x1cd7fe5368733b9a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b1c6 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619906.336951:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fefd7800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589043968:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619906.337311:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccb40 going to sleep for 11 seconds 00000100:00100000:2.0:1788619906.340705:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fefd7800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589043968:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619906.340760:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000fb640cb5/0x1cd7fe5368733b9a lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b1c6 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.340802:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fb640cb5/0x1cd7fe5368733b9a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b1c6 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.340819:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6e00, rc 0) 00000100:00100000:2.0:1788619906.341663:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b11500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571315328:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4843us (5074us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619906.341675:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 509 00000100:00100000:3.0:1788619906.347165:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619906.347275:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571315456 00000100:00100000:0.0:1788619906.347289:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571315456 00000100:00100000:0.0:1788619906.347548:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 510 00000100:00100000:0.0:1788619906.347563:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5019180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571315456:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619906.347587:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619906.347613:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001eabdd3d/0x1cd7fe5368733ba1 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b1cd expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619906.347701:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed61f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589044096:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619906.347829:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:0.0:1788619906.352692:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed61f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589044096:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619906.352748:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001eabdd3d/0x1cd7fe5368733ba1 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b1cd expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.352781:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001eabdd3d/0x1cd7fe5368733ba1 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b1cd expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.352835:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78ccca7e00, rc 0) 00000100:00100000:0.0:1788619906.352916:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5019180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571315456:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5356us (5766us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619906.352926:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 510 00000100:00100000:1.0:1788619906.359644:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619906.359746:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571315584 00000100:00100000:3.0:1788619906.359760:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571315584 00002000:00100000:3.0:1788619906.359844:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b11180 x1875503571315584/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:2/0 lens 488/0 e 0 to 0 dl 1788619917 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619906.359857:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b11180 with x1875503571315584 00000100:00100000:3.0:1788619906.359907:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 369 00000100:00100000:3.0:1788619906.359919:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b11180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2353:x1875503571315584:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619906.360391:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b11180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2353:x1875503571315584:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 472us (762us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619906.360400:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 369 00002000:00100000:3.0:1788619906.360408:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b11180 x1875503571315584/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:2/0 lens 488/440 e 0 to 0 dl 1788619917 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619906.360417:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b11180 with x1875503571315584 00000100:00100000:3.0:1788619906.367605:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619906.367825:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571315712 00000100:00100000:1.0:1788619906.367838:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571315712 00000100:00100000:1.0:1788619906.367978:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 511 00000100:00100000:1.0:1788619906.368079:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571315712:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619906.368106:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619906.368131:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000199d2a35/0x1cd7fe5368733ba8 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b1d4 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619906.368212:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b10380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589044224:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619906.369005:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd9639c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619906.375661:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b10380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589044224:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619906.375721:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000199d2a35/0x1cd7fe5368733ba8 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b1d4 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.375756:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000199d2a35/0x1cd7fe5368733ba8 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b1d4 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.375774:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5023800, rc 0) 00000100:00100000:0.0:1788619906.375949:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571315712:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7938us (8360us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619906.375961:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 511 00000100:00100000:1.0:1788619906.384637:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619906.384812:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571315840 00000100:00100000:2.0:1788619906.384829:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571315840 00000100:00100000:2.0:1788619906.385032:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 512 00000100:00100000:2.0:1788619906.385048:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b10380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571315840:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619906.386296:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619906.387211:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fb640cb5/0x1cd7fe5368733baf lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b1db expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619906.387316:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fefd7800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589044352:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619906.387530:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccb40 going to sleep for 11 seconds 00000100:00100000:2.0:1788619906.393991:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fefd7800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589044352:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619906.394213:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000fb640cb5/0x1cd7fe5368733baf lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b1db expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.394249:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fb640cb5/0x1cd7fe5368733baf lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b1db expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.394268:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6e00, rc 0) 00000100:00100000:2.0:1788619906.394605:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b10380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571315840:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 9559us (9988us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619906.394616:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 512 00000100:00100000:1.0:1788619906.402450:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619906.402550:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571315968 00000100:00100000:0.0:1788619906.402573:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571315968 00002000:00100000:0.0:1788619906.402647:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c71f1f80 x1875503571315968/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:2/0 lens 488/0 e 0 to 0 dl 1788619917 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619906.402662:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c71f1f80 with x1875503571315968 00000100:00100000:0.0:1788619906.402710:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 370 00000100:00100000:0.0:1788619906.402724:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2352:x1875503571315968:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619906.403078:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2352:x1875503571315968:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 356us (642us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619906.403087:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 370 00002000:00100000:0.0:1788619906.403095:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c71f1f80 x1875503571315968/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:2/0 lens 488/440 e 0 to 0 dl 1788619917 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619906.403103:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c71f1f80 with x1875503571315968 00000100:00100000:1.0:1788619906.416436:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619906.416708:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571316096 00000100:00100000:2.0:1788619906.416740:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571316096 00000100:00100000:2.0:1788619906.416825:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 513 00000100:00100000:2.0:1788619906.416839:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571316096:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619906.416864:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619906.416888:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fb640cb5/0x1cd7fe5368733bb6 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b1e2 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619906.416998:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fefd7800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589044480:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619906.417118:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccb40 going to sleep for 11 seconds 00000100:00100000:2.0:1788619906.421687:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fefd7800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589044480:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619906.421751:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000fb640cb5/0x1cd7fe5368733bb6 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b1e2 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.421793:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fb640cb5/0x1cd7fe5368733bb6 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b1e2 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.421811:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6e00, rc 0) 00000100:00100000:2.0:1788619906.421912:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571316096:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5075us (5491us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619906.421924:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 513 00000100:00100000:1.0:1788619906.432877:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619906.433054:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571316224 00000100:00100000:0.0:1788619906.433070:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571316224 00000100:00100000:0.0:1788619906.433150:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 514 00000100:00100000:0.0:1788619906.433163:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571316224:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619906.433189:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619906.433213:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca5ff342/0x1cd7fe5368733bbd lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b1e9 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619906.433320:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed61f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589044608:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619906.433483:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:0.0:1788619906.441318:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed61f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589044608:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619906.441434:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ca5ff342/0x1cd7fe5368733bbd lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b1e9 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.441471:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca5ff342/0x1cd7fe5368733bbd lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b1e9 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.441492:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78ccca7000, rc 0) 00000100:00100000:0.0:1788619906.441591:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571316224:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8431us (8737us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619906.441603:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 514 00000100:00100000:1.0:1788619906.449487:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619906.449663:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571316352 00000100:00100000:2.0:1788619906.449678:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571316352 00002000:00100000:2.0:1788619906.449749:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c71f2d80 x1875503571316352/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:2/0 lens 488/0 e 0 to 0 dl 1788619917 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619906.449763:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c71f2d80 with x1875503571316352 00000100:00100000:2.0:1788619906.449875:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 371 00000100:00100000:2.0:1788619906.449888:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2354:x1875503571316352:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619906.450288:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2354:x1875503571316352:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 401us (817us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619906.450298:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 371 00002000:00100000:2.0:1788619906.450307:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c71f2d80 x1875503571316352/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:2/0 lens 488/440 e 0 to 0 dl 1788619917 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619906.450363:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c71f2d80 with x1875503571316352 00000100:00100000:1.0:1788619906.454469:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619906.454584:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571316480 00000100:00100000:0.0:1788619906.454598:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571316480 00000100:00100000:0.0:1788619906.454680:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 515 00000100:00100000:0.0:1788619906.454693:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571316480:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619906.454716:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619906.454740:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002d38f023/0x1cd7fe5368733bc4 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b1f0 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619906.454823:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed61f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589044736:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619906.454942:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:0.0:1788619906.458563:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed61f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589044736:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619906.458629:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002d38f023/0x1cd7fe5368733bc4 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b1f0 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.458663:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d38f023/0x1cd7fe5368733bc4 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b1f0 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.458681:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78ccca6e00, rc 0) 00000100:00100000:0.0:1788619906.458767:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571316480:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4075us (4313us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619906.458777:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 515 00000100:00100000:3.0:1788619906.462644:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619906.466094:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571316608 00000100:00100000:2.0:1788619906.466120:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571316608 00000100:00100000:2.0:1788619906.466225:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 516 00000100:00100000:2.0:1788619906.466248:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa15500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571316608:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619906.466272:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619906.466295:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fb640cb5/0x1cd7fe5368733bcb lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b1f7 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619906.466417:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fefd7800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589044864:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619906.466528:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccb40 going to sleep for 11 seconds 00000100:00100000:2.0:1788619906.470706:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fefd7800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589044864:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619906.470765:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000fb640cb5/0x1cd7fe5368733bcb lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b1f7 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.470809:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fb640cb5/0x1cd7fe5368733bcb lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b1f7 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.470852:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6e00, rc 0) 00000100:00100000:2.0:1788619906.470947:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa15500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571316608:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4710us (8316us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619906.470959:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 516 00000100:00100000:1.0:1788619906.478028:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619906.478124:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571316736 00000100:00100000:3.0:1788619906.478139:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571316736 00002000:00100000:3.0:1788619906.478217:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c71f0a80 x1875503571316736/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:2/0 lens 488/0 e 0 to 0 dl 1788619917 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619906.478242:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c71f0a80 with x1875503571316736 00000100:00100000:3.0:1788619906.478289:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 372 00000100:00100000:3.0:1788619906.478302:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2355:x1875503571316736:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619906.478745:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2355:x1875503571316736:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 444us (733us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619906.478755:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 372 00002000:00100000:3.0:1788619906.478762:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c71f0a80 x1875503571316736/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:2/0 lens 488/440 e 0 to 0 dl 1788619917 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619906.478770:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c71f0a80 with x1875503571316736 00000100:00100000:3.0:1788619906.484496:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619906.484729:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571316864 00000100:00100000:1.0:1788619906.484742:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571316864 00000100:00100000:1.0:1788619906.484825:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 517 00000100:00100000:1.0:1788619906.484838:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa16a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571316864:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619906.484874:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619906.484901:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006db26947/0x1cd7fe5368733bd2 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b1fe expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619906.484991:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c71f3100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589044992:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619906.485110:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963180 going to sleep for 11 seconds 00000100:00100000:2.0:1788619906.488919:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c71f3100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589044992:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619906.488991:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000006db26947/0x1cd7fe5368733bd2 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b1fe expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.489026:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006db26947/0x1cd7fe5368733bd2 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b1fe expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.489043:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5023200, rc 0) 00000100:00100000:2.0:1788619906.489126:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa16a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571316864:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4290us (4647us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619906.489137:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 517 00000100:00100000:2.0:1788619906.493686:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619906.493777:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571316992 00000100:00100000:0.0:1788619906.493807:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571316992 00000100:00100000:0.0:1788619906.493882:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 518 00000100:00100000:0.0:1788619906.493894:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571316992:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619906.494053:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619906.494081:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002d38f023/0x1cd7fe5368733bd9 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b205 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619906.494177:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed61f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589045120:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619906.494309:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:0.0:1788619906.498585:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed61f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589045120:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619906.498650:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002d38f023/0x1cd7fe5368733bd9 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b205 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.498685:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d38f023/0x1cd7fe5368733bd9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b205 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.498702:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78ccca6e00, rc 0) 00000100:00100000:0.0:1788619906.498835:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571316992:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4941us (5163us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619906.498846:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 518 00000100:00100000:1.0:1788619906.502892:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619906.503298:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571317120 00000100:00100000:2.0:1788619906.503312:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571317120 00002000:00100000:2.0:1788619906.503464:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c71f2a00 x1875503571317120/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:2/0 lens 488/0 e 0 to 0 dl 1788619917 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619906.503478:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c71f2a00 with x1875503571317120 00000100:00100000:2.0:1788619906.503574:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 373 00000100:00100000:2.0:1788619906.503588:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2354:x1875503571317120:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619906.504410:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2354:x1875503571317120:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 818us (1527us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619906.504422:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 373 00002000:00100000:2.0:1788619906.504431:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c71f2a00 x1875503571317120/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:2/0 lens 488/440 e 0 to 0 dl 1788619917 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619906.504442:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c71f2a00 with x1875503571317120 00000100:00100000:1.0:1788619906.510663:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619906.510780:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571317248 00000100:00100000:0.0:1788619906.510873:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571317248 00000100:00100000:0.0:1788619906.511265:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 519 00000100:00100000:0.0:1788619906.511280:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571317248:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619906.511307:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619906.511491:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca5ff342/0x1cd7fe5368733be0 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b20c expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619906.511588:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed61f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589045248:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619906.511773:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:2.0:1788619906.515516:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed61f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589045248:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619906.515576:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ca5ff342/0x1cd7fe5368733be0 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b20c expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.515611:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca5ff342/0x1cd7fe5368733be0 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b20c expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.515631:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78ccca7000, rc 0) 00000100:00100000:2.0:1788619906.515715:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571317248:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4438us (5070us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619906.515726:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 519 00000100:00100000:1.0:1788619906.520425:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619906.520605:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571317376 00000100:00100000:2.0:1788619906.520618:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571317376 00000100:00100000:2.0:1788619906.520693:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 520 00000100:00100000:2.0:1788619906.520704:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe13ac50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571317376:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619906.520728:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619906.520752:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001dc9de0/0x1cd7fe5368733be7 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b213 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619906.520843:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c4a4ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589045376:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619906.520950:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccb40 going to sleep for 11 seconds 00000100:00100000:2.0:1788619906.524232:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c4a4ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589045376:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619906.524298:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000001dc9de0/0x1cd7fe5368733be7 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b213 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.524376:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001dc9de0/0x1cd7fe5368733be7 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b213 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.524394:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6a00, rc 0) 00000100:00100000:2.0:1788619906.524470:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe13ac50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571317376:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3767us (4061us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619906.524481:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 520 00000100:00100000:1.0:1788619906.527993:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619906.528851:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571317504 00000100:00100000:0.0:1788619906.528867:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571317504 00002000:00100000:1.0:1788619906.529078:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c71f1500 x1875503571317504/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:2/0 lens 488/0 e 0 to 0 dl 1788619917 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619906.529091:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c71f1500 with x1875503571317504 00000100:00100000:1.0:1788619906.529187:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 374 00000100:00100000:1.0:1788619906.529199:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2355:x1875503571317504:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619906.529643:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2355:x1875503571317504:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 443us (1676us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619906.529652:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 374 00002000:00100000:1.0:1788619906.529660:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c71f1500 x1875503571317504/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:2/0 lens 488/440 e 0 to 0 dl 1788619917 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619906.529669:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c71f1500 with x1875503571317504 00000100:00100000:0.0:1788619906.533939:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619906.534055:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571317632 00000100:00100000:2.0:1788619906.534069:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571317632 00000100:00100000:2.0:1788619906.534193:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 521 00000100:00100000:2.0:1788619906.534207:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed61880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571317632:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619906.534249:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619906.534277:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041aa2bf0/0x1cd7fe5368733bee lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b21a expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619906.534405:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c4a4ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589045504:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619906.534526:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccb40 going to sleep for 11 seconds 00000100:00100000:1.0:1788619906.537044:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c4a4ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589045504:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619906.537098:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000041aa2bf0/0x1cd7fe5368733bee lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b21a expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619906.537130:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041aa2bf0/0x1cd7fe5368733bee lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b21a expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619906.537145:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7a00, rc 0) 00000100:00100000:1.0:1788619906.537223:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed61880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571317632:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3019us (3301us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619906.537241:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 521 00000100:00100000:1.0:1788619906.541863:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619906.542112:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571317760 00000100:00100000:0.0:1788619906.542126:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571317760 00000100:00100000:0.0:1788619906.542201:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 522 00000100:00100000:0.0:1788619906.542212:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571317760:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619906.542246:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619906.542270:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002d38f023/0x1cd7fe5368733bf5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b221 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619906.542381:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed60a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589045632:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619906.542505:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619906.545878:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed60a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589045632:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619906.545957:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002d38f023/0x1cd7fe5368733bf5 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b221 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.546004:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d38f023/0x1cd7fe5368733bf5 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b221 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.546026:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78ccca6e00, rc 0) 00000100:00100000:0.0:1788619906.546129:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571317760:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3917us (4281us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619906.546140:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 522 00000100:00100000:3.0:1788619906.553075:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619906.553276:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571317888 00000100:00100000:2.0:1788619906.553289:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571317888 00002000:00100000:2.0:1788619906.553411:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78dfa14380 x1875503571317888/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:2/0 lens 488/0 e 0 to 0 dl 1788619917 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619906.553426:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78dfa14380 with x1875503571317888 00000100:00100000:2.0:1788619906.553473:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 375 00000100:00100000:2.0:1788619906.553484:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa14380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2352:x1875503571317888:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619906.553843:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa14380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2352:x1875503571317888:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 359us (783us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619906.553853:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 375 00002000:00100000:2.0:1788619906.553860:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78dfa14380 x1875503571317888/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:2/0 lens 488/440 e 0 to 0 dl 1788619917 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619906.553869:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78dfa14380 with x1875503571317888 00000100:00100000:1.0:1788619906.557706:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619906.557838:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571318016 00000100:00100000:0.0:1788619906.557851:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571318016 00000100:00100000:0.0:1788619906.557945:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 523 00000100:00100000:0.0:1788619906.557958:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571318016:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619906.557993:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619906.558020:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006d21ef6a/0x1cd7fe5368733bfc lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b228 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619906.558102:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed60a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589045760:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619906.558213:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619906.561316:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed60a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589045760:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619906.561419:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000006d21ef6a/0x1cd7fe5368733bfc lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b228 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.561452:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006d21ef6a/0x1cd7fe5368733bfc lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b228 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.561470:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78ccca6400, rc 0) 00000100:00100000:2.0:1788619906.561544:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571318016:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3588us (3859us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619906.561555:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 523 00000100:00100000:2.0:1788619906.566023:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619906.566153:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571318144 00000100:00100000:1.0:1788619906.566166:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571318144 00000100:00100000:1.0:1788619906.566360:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 524 00000100:00100000:1.0:1788619906.566373:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571318144:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619906.566397:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619906.566421:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb562d34/0x1cd7fe5368733c03 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b22f expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619906.566504:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c71f3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589045888:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619906.566611:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963300 going to sleep for 11 seconds 00000100:00100000:2.0:1788619906.570035:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c71f3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589045888:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619906.570094:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bb562d34/0x1cd7fe5368733c03 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b22f expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.570126:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb562d34/0x1cd7fe5368733c03 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b22f expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.570142:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5022c00, rc 0) 00000100:00100000:2.0:1788619906.570366:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571318144:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3994us (4355us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619906.570377:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 524 00000100:00100000:3.0:1788619906.573962:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619906.574147:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571318272 00000100:00100000:1.0:1788619906.574162:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571318272 00002000:00100000:1.0:1788619906.574407:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78dfa15f80 x1875503571318272/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:2/0 lens 488/0 e 0 to 0 dl 1788619917 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619906.574424:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78dfa15f80 with x1875503571318272 00000100:00100000:1.0:1788619906.574491:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 376 00000100:00100000:1.0:1788619906.574504:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa15f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2355:x1875503571318272:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619906.575218:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa15f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2355:x1875503571318272:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 713us (1274us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619906.575229:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 376 00002000:00100000:1.0:1788619906.575308:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78dfa15f80 x1875503571318272/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:2/0 lens 488/440 e 0 to 0 dl 1788619917 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619906.575317:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78dfa15f80 with x1875503571318272 00000100:00100000:3.0:1788619906.578784:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619906.579038:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571318400 00000100:00100000:1.0:1788619906.579052:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571318400 00000100:00100000:1.0:1788619906.579147:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 525 00000100:00100000:1.0:1788619906.579159:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa14a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571318400:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619906.579185:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619906.579209:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005bc4532a/0x1cd7fe5368733c0a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b236 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619906.579429:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c71f3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589046016:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619906.579531:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963840 going to sleep for 11 seconds 00000100:00100000:1.0:1788619906.583529:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c71f3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589046016:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619906.583587:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005bc4532a/0x1cd7fe5368733c0a lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b236 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619906.583619:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005bc4532a/0x1cd7fe5368733c0a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b236 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619906.583634:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5022400, rc 0) 00000100:00100000:1.0:1788619906.583760:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa14a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571318400:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4601us (4992us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619906.583771:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 525 00000100:00100000:0.0:1788619906.587828:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619906.587931:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571318528 00000100:00100000:2.0:1788619906.587944:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571318528 00000100:00100000:2.0:1788619906.588031:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 526 00000100:00100000:2.0:1788619906.588043:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed62300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571318528:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619906.588065:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619906.588089:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1c87370/0x1cd7fe5368733c11 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b23d expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619906.588169:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5026680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589046144:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619906.588842:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc240 going to sleep for 11 seconds 00000100:00100000:2.0:1788619906.592022:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5026680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589046144:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619906.592082:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b1c87370/0x1cd7fe5368733c11 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b23d expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.592142:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1c87370/0x1cd7fe5368733c11 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b23d expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.592159:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6200, rc 0) 00000100:00100000:2.0:1788619906.592370:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed62300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571318528:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4327us (4556us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619906.592382:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 526 00000100:00100000:1.0:1788619906.602786:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619906.603071:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571318656 00000100:00100000:3.0:1788619906.603086:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571318656 00002000:00100000:3.0:1788619906.603155:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c71f3800 x1875503571318656/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:2/0 lens 488/0 e 0 to 0 dl 1788619917 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619906.603169:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c71f3800 with x1875503571318656 00000100:00100000:3.0:1788619906.603280:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 377 00000100:00100000:3.0:1788619906.603292:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2355:x1875503571318656:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619906.603723:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2355:x1875503571318656:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 432us (951us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619906.603732:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 377 00002000:00100000:3.0:1788619906.603740:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c71f3800 x1875503571318656/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:2/0 lens 488/440 e 0 to 0 dl 1788619917 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619906.603748:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c71f3800 with x1875503571318656 00000100:00100000:0.0:1788619906.608244:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619906.608556:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571318784 00000100:00100000:2.0:1788619906.608569:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571318784 00000100:00100000:2.0:1788619906.608715:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 527 00000100:00100000:2.0:1788619906.608729:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed60a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571318784:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619906.608754:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619906.608775:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5ec8983/0x1cd7fe5368733c18 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b244 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619906.608858:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5026680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589046272:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619906.608982:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc240 going to sleep for 11 seconds 00000100:00100000:0.0:1788619906.613822:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5026680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589046272:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619906.613882:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a5ec8983/0x1cd7fe5368733c18 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b244 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.613916:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5ec8983/0x1cd7fe5368733c18 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b244 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.613932:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6600, rc 0) 00000100:00100000:0.0:1788619906.614058:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed60a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571318784:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5332us (5829us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619906.614094:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 527 00000100:00100000:1.0:1788619906.621750:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619906.621911:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571318912 00000100:00100000:3.0:1788619906.621927:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571318912 00002000:00100000:3.0:1788619906.622046:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c71f3100 x1875503571318912/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:2/0 lens 488/0 e 0 to 0 dl 1788619917 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619906.622061:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c71f3100 with x1875503571318912 00000100:00100000:3.0:1788619906.622106:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 378 00000100:00100000:3.0:1788619906.622117:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2355:x1875503571318912:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:3.0:1788619906.630019:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 782531 -> 888832 00000004:00010000:3.0:1788619906.630299:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619905 -> 1788619906 00000004:00010000:3.0:1788619906.630302:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619905 -> 1788619904 00000004:00010000:3.0:1788619906.630305:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619906 -> 1788619906 00000004:00010000:3.0:1788619906.630308:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 774 -> 774 00000100:00100000:3.0:1788619906.633785:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2355:x1875503571318912:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 11649us (12031us total) trans 4294967563 rc 0/0 00000100:00100000:3.0:1788619906.633808:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 378 00002000:00100000:3.0:1788619906.633831:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c71f3100 x1875503571318912/t4294967563(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:2/0 lens 488/448 e 0 to 0 dl 1788619917 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619906.633845:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c71f3100 with x1875503571318912 00000100:00100000:2.0:1788619906.640556:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619906.640740:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571319040 00000100:00100000:1.0:1788619906.640754:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571319040 00000100:00100000:1.0:1788619906.641048:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 681 00000100:00100000:1.0:1788619906.641062:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5025180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2354:x1875503571319040:12345-192.168.201.5@tcp:16:fsx.0 00000100:00100000:1.0:1788619906.671585:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5025180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2354:x1875503571319040:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 30510us (31026us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619906.671605:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 681 00000001:00080000:1.0:1788619906.671654:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967562 is committed 00000001:00080000:1.0:1788619906.671665:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967561 is committed 00000001:00080000:1.0:1788619906.671707:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967563 is committed 00000100:00100000:1.0:1788619906.675138:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619906.675256:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571319168 00000100:00100000:3.0:1788619906.675271:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571319168 00000100:00100000:3.0:1788619906.675386:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 528 00000100:00100000:3.0:1788619906.675400:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571319168:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619906.675426:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619906.675452:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000095a50c3e/0x1cd7fe5368733c1f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b24b expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619906.675536:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78dfa15500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589046400:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619906.675664:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5c76600 going to sleep for 11 seconds 00000100:00100000:3.0:1788619906.678307:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78dfa15500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589046400:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619906.678408:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000095a50c3e/0x1cd7fe5368733c1f lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b24b expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619906.678446:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000095a50c3e/0x1cd7fe5368733c1f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b24b expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619906.678463:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d600, rc 0) 00000100:00100000:3.0:1788619906.678545:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571319168:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3146us (3420us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619906.678556:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 528 00000100:00100000:0.0:1788619906.682174:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619906.682285:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571319296 00000100:00100000:1.0:1788619906.682298:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571319296 00000100:00100000:1.0:1788619906.682397:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 529 00000100:00100000:1.0:1788619906.682410:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed62a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571319296:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619906.682434:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619906.682459:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb562d34/0x1cd7fe5368733c26 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b252 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619906.682537:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be2300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589046528:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619906.682647:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963840 going to sleep for 11 seconds 00000100:00100000:2.0:1788619906.685815:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be2300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589046528:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619906.685869:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bb562d34/0x1cd7fe5368733c26 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b252 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.685902:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb562d34/0x1cd7fe5368733c26 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b252 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.685935:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5022c00, rc 0) 00000100:00100000:2.0:1788619906.686024:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed62a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571319296:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3614us (3868us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619906.686033:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 529 00000100:00100000:2.0:1788619906.690263:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619906.690623:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571319424 00000100:00100000:0.0:1788619906.690638:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571319424 00000100:00100000:0.0:1788619906.690720:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 530 00000100:00100000:0.0:1788619906.690733:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5026680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:14908:x1875503571319424:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619906.690757:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619906.690782:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7278240/0x1cd7fe5368733c2d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b259 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.690840:0:7654:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000002aaba2be/0x1cd7fe5368733b38 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1462272->1466367) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b164 expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.690852:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=1s v=(4 1) 00010000:00010000:0.0:1788619906.690856:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(5 1) 00010000:00010000:0.0:1788619906.690863:0:7654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7278240/0x1cd7fe5368733c2d lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1486848->1507327] (req 1486848->1507327) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b259 expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.690914:0:7654:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000002aaba2be/0x1cd7fe5368733b38 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1462272->1466367) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b164 expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.690930:0:7654:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000002aaba2be/0x1cd7fe5368733b38 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1462272->1466367) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b164 expref: 5 pid: 6698 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619906.690948:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed61f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589046656:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619906.691090:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619906.695599:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619906.695745:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed61f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589046656:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619906.695810:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d7278240/0x1cd7fe5368733c2d lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1486848->1507327] (req 1486848->1507327) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b259 expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.695837:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78ccca7400, rc 0) 00000100:00100000:2.0:1788619906.695922:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5026680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+6:14908:x1875503571319424:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5190us (5689us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619906.695932:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 530 00000100:00100000:1.0:1788619906.696145:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571319552 00000100:00100000:1.0:1788619906.696155:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571319552 00010000:00010000:1.0:1788619906.696204:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000002aaba2be/0x1cd7fe5368733b38 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1462272->1466367) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b164 expref: 6 pid: 6698 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619906.696255:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 223 00000100:00100000:1.0:1788619906.696266:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed62d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571319552:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619906.696287:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619906.696318:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000002aaba2be/0x1cd7fe5368733b38 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1462272->1466367) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b164 expref: 6 pid: 6698 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619906.696400:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000002aaba2be/0x1cd7fe5368733b38 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1462272->1466367) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b164 expref: 6 pid: 6698 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619906.696435:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002aaba2be/0x1cd7fe5368733b38 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1462272->1466367) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b164 expref: 6 pid: 6698 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619906.696456:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(5 1) 00010000:00010000:1.0:1788619906.696462:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(5 1) 00010000:00010000:1.0:1788619906.696472:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000d7278240/0x1cd7fe5368733c2d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1486848->1507327) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b259 expref: 5 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619906.696486:0:5829:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7278240/0x1cd7fe5368733c2d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1486848->1507327) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b259 expref: 5 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619906.696531:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000d7278240/0x1cd7fe5368733c2d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1486848->1507327) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b259 expref: 5 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619906.696548:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be1f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1875503589046784:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619906.696622:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd9633c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619906.700271:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be1f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1875503589046784:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619906.700312:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619906.700434:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed62d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571319552:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 4168us (4852us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619906.700449:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 223 00000100:00100000:0.0:1788619906.704133:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619906.704448:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571319680 00000100:00100000:2.0:1788619906.704461:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571319680 00002000:00100000:2.0:1788619906.704535:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1486848->1490943) req@ffff8e78fed61f80 x1875503571319680/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:2/0 lens 488/0 e 0 to 0 dl 1788619917 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619906.704550:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fed61f80 with x1875503571319680 ext(1486848->1490943) 00002000:00010000:2.0:1788619906.704577:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fed61f80 00000100:00100000:2.0:1788619906.704622:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 267 00000100:00100000:2.0:1788619906.704635:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed61f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571319680:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619906.705112:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed61f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571319680:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 478us (994us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619906.705122:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 267 00002000:00100000:2.0:1788619906.705129:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1486848->1490943) req@ffff8e78fed61f80 x1875503571319680/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:2/0 lens 488/4536 e 0 to 0 dl 1788619917 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619906.705139:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fed61f80 with x1875503571319680 ext(1486848->1490943) 00002000:00010000:2.0:1788619906.705142:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fed61f80 00000100:00100000:0.0:1788619906.709732:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619906.710011:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571319808 00000100:00100000:2.0:1788619906.710025:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571319808 00002000:00100000:2.0:1788619906.710107:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1503232->1507327) req@ffff8e77c6b14a80 x1875503571319808/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:2/0 lens 488/0 e 0 to 0 dl 1788619917 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619906.710121:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6b14a80 with x1875503571319808 ext(1503232->1507327) 00002000:00010000:2.0:1788619906.710128:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6b14a80 00000100:00100000:2.0:1788619906.710171:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 268 00000100:00100000:2.0:1788619906.710183:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b14a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571319808:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619906.710527:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b14a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571319808:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 345us (810us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619906.710535:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 268 00002000:00100000:2.0:1788619906.710543:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1503232->1507327) req@ffff8e77c6b14a80 x1875503571319808/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:2/0 lens 488/4536 e 0 to 0 dl 1788619917 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619906.710552:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6b14a80 with x1875503571319808 ext(1503232->1507327) 00002000:00010000:2.0:1788619906.710577:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6b14a80 00000100:00100000:1.0:1788619906.715817:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619906.716003:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571319936 00000100:00100000:0.0:1788619906.716017:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571319936 00000100:00100000:0.0:1788619906.716097:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 531 00000100:00100000:0.0:1788619906.716111:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:14908:x1875503571319936:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619906.716137:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619906.716163:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca5ff342/0x1cd7fe5368733c34 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b260 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.716209:0:6698:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000d7278240/0x1cd7fe5368733c2d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1486848->1507327) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b259 expref: 5 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.716220:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(5 1) 00010000:00010000:0.0:1788619906.716225:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(6 1) 00010000:00010000:0.0:1788619906.716243:0:6698:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca5ff342/0x1cd7fe5368733c34 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1110016->1122303] (req 1110016->1122303) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b260 expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.716290:0:6698:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000d7278240/0x1cd7fe5368733c2d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1486848->1507327) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b259 expref: 5 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.716306:0:6698:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000d7278240/0x1cd7fe5368733c2d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1486848->1507327) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b259 expref: 5 pid: 7654 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619906.716378:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b16d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589046912:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619906.716511:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaacc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619906.719169:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b16d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589046912:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619906.719229:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ca5ff342/0x1cd7fe5368733c34 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1110016->1122303] (req 1110016->1122303) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b260 expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.719273:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78ccca7000, rc 0) 00000100:00100000:2.0:1788619906.719399:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:14908:x1875503571319936:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3290us (3595us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619906.719410:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 531 00000100:00100000:1.0:1788619906.722743:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619906.722930:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571320064 00000100:00100000:2.0:1788619906.722944:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571320064 00002000:00100000:2.0:1788619906.723029:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1486848->1507327) req@ffff8e77c4be2d80 x1875503571320064/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:2/0 lens 488/0 e 0 to 0 dl 1788619917 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619906.723042:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4be2d80 with x1875503571320064 ext(1486848->1507327) 00010000:00010000:2.0:1788619906.723054:0:6702:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000d7278240/0x1cd7fe5368733c2d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1486848->1507327) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b259 expref: 6 pid: 7654 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.723075:0:6702:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000d7278240/0x1cd7fe5368733c2d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1486848->1507327) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b259 expref: 6 pid: 7654 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788619906.723083:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff8e77c4be2d80 00000100:00100000:2.0:1788619906.723177:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 7 00000100:00100000:2.0:1788619906.723190:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571320064:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:2.0:1788619906.728313:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571320064:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 5105us (5565us total) trans 4294967369 rc 0/0 00000100:00100000:2.0:1788619906.728382:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 7 00002000:00100000:2.0:1788619906.728401:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1486848->1507327) req@ffff8e77c4be2d80 x1875503571320064/t4294967369(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:2/0 lens 488/448 e 0 to 0 dl 1788619917 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619906.728413:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4be2d80 with x1875503571320064 ext(1486848->1507327) 00010000:00010000:2.0:1788619906.728427:0:6702:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000d7278240/0x1cd7fe5368733c2d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1486848->1507327) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b259 expref: 8 pid: 7654 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.728453:0:6702:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000d7278240/0x1cd7fe5368733c2d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1486848->1507327) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b259 expref: 8 pid: 7654 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788619906.728462:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff8e77c4be2d80 00000100:00100000:1.0:1788619906.732056:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619906.732177:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571320192 00000100:00100000:0.0:1788619906.732190:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571320192 00010000:00010000:0.0:1788619906.732312:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000d7278240/0x1cd7fe5368733c2d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1486848->1507327) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b259 expref: 8 pid: 7654 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619906.732443:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 224 00000100:00100000:0.0:1788619906.732457:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571320192:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619906.732477:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:0.0:1788619906.732504:0:13330:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619905 -> 1788619906 00010000:00010000:0.0:1788619906.732513:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d7278240/0x1cd7fe5368733c2d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1486848->1507327) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b259 expref: 8 pid: 7654 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.732530:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d7278240/0x1cd7fe5368733c2d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1486848->1507327) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b259 expref: 8 pid: 7654 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.732562:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7278240/0x1cd7fe5368733c2d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1486848->1507327) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b259 expref: 8 pid: 7654 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.732586:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(6 1) 00010000:00010000:0.0:1788619906.732591:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(6 1) 00010000:00010000:0.0:1788619906.732603:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000ca5ff342/0x1cd7fe5368733c34 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1110016->1122303) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b260 expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.732616:0:13330:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca5ff342/0x1cd7fe5368733c34 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1110016->1122303) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b260 expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.732689:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000ca5ff342/0x1cd7fe5368733c34 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1110016->1122303) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b260 expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619906.732711:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b16a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589047040:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619906.733067:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:1.0:1788619906.738204:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619906.738360:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571320320 00000100:00100000:3.0:1788619906.738374:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571320320 00002000:00100000:3.0:1788619906.738447:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1110016->1114111) req@ffff8e77c4be0a80 x1875503571320320/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:2/0 lens 488/0 e 0 to 0 dl 1788619917 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619906.738462:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4be0a80 with x1875503571320320 ext(1110016->1114111) 00002000:00010000:3.0:1788619906.738470:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4be0a80 00000100:00100000:3.0:1788619906.738513:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 269 00000100:00100000:3.0:1788619906.738527:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571320320:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619906.739083:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571320320:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 555us (892us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619906.739092:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 269 00002000:00100000:3.0:1788619906.739101:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1110016->1114111) req@ffff8e77c4be0a80 x1875503571320320/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:2/0 lens 488/4536 e 0 to 0 dl 1788619917 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619906.739132:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4be0a80 with x1875503571320320 ext(1110016->1114111) 00002000:00010000:3.0:1788619906.739137:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4be0a80 00000100:00100000:3.0:1788619906.742586:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619906.742650:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571320448 00000100:00100000:3.0:1788619906.742662:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571320448 00002000:00100000:3.0:1788619906.742728:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1118208->1122303) req@ffff8e78dfa15500 x1875503571320448/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:2/0 lens 488/0 e 0 to 0 dl 1788619917 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619906.742742:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa15500 with x1875503571320448 ext(1118208->1122303) 00002000:00010000:3.0:1788619906.742749:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa15500 00000100:00100000:3.0:1788619906.742782:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 270 00000100:00100000:3.0:1788619906.742805:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa15500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571320448:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619906.743168:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa15500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571320448:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 364us (595us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619906.743177:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 270 00002000:00100000:3.0:1788619906.743184:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1118208->1122303) req@ffff8e78dfa15500 x1875503571320448/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:2/0 lens 488/4536 e 0 to 0 dl 1788619917 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619906.743193:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa15500 with x1875503571320448 ext(1118208->1122303) 00002000:00010000:3.0:1788619906.743198:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa15500 00000100:00100000:0.0:1788619906.744995:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b16a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589047040:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619906.745035:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619906.745114:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571320192:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 12659us (13074us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619906.745129:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 224 00000100:00100000:3.0:1788619906.748871:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619906.748988:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571320576 00000100:00100000:0.0:1788619906.749002:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571320576 00000100:00100000:0.0:1788619906.749080:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 682 00000100:00100000:0.0:1788619906.749092:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa16a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571320576:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619906.749121:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619906.749148:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6e00/0x1cd7fe5368733c3b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b267 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.749196:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7200/0x1cd7fe5368733b3f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b16b expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619906.749258:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b16a00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589047168:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619906.749397:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000004:00010000:0.0:1788619906.752135:0:11397:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619904 -> 1788619906 00000004:00010000:0.0:1788619906.752165:0:11397:0:(mdt_lvb.c:201:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks: 774 -> 1158 00000100:00100000:0.0:1788619906.752202:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b16a00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589047168:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619906.752246:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 888832 00010000:00010000:0.0:1788619906.752273:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6e00/0x1cd7fe5368733c3b lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b267 expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.752298:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619906.752302:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619906.752308:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6e00/0x1cd7fe5368733c3b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b267 expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.752366:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78ccca6e00, rc 0) 00000100:00100000:0.0:1788619906.752440:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa16a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571320576:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3349us (3585us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619906.752452:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 682 00000100:00100000:3.0:1788619906.755573:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619906.756012:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571320704 00000100:00100000:2.0:1788619906.756025:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571320704 00000100:00100000:2.0:1788619906.756110:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 683 00000100:00100000:2.0:1788619906.756123:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7583100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571320704:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619906.756152:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619906.756179:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6800/0x1cd7fe5368733c42 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b26e expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.756239:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7200/0x1cd7fe5368733b3f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b16b expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619906.756297:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6003100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589047296:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619906.756450:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccf00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619906.761243:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6003100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589047296:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619906.761300:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 888832 00010000:00010000:2.0:1788619906.761372:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6800/0x1cd7fe5368733c42 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b26e expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.761404:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619906.761408:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619906.761415:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6800/0x1cd7fe5368733c42 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b26e expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.761434:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6800, rc 0) 00000100:00100000:2.0:1788619906.761507:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7583100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571320704:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5387us (5946us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619906.761517:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 683 00000100:00100000:1.0:1788619906.766269:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619906.766559:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571320832 00000100:00100000:3.0:1788619906.766575:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571320832 00000100:00100000:3.0:1788619906.766732:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 379 00000100:00100000:3.0:1788619906.766745:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571320832:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:3.0:1788619906.767725:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 888832 -> 888832 00000004:00010000:3.0:1788619906.767735:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619906 -> 1788619906 00000004:00010000:3.0:1788619906.767738:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619906 -> 1788619904 00000004:00010000:3.0:1788619906.767741:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619906 -> 1788619906 00000004:00010000:3.0:1788619906.767744:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1158 -> 1158 00000100:00100000:3.0:1788619906.768116:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2353:x1875503571320832:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 1368us (1856us total) trans 4294967564 rc 0/0 00000100:00100000:3.0:1788619906.768128:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 379 00000100:00100000:3.0:1788619906.774468:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619906.774660:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571320960 00000100:00100000:2.0:1788619906.774673:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571320960 00000100:00100000:2.0:1788619906.774737:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 532 00000100:00100000:2.0:1788619906.774748:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7581500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571320960:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619906.774773:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619906.774807:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007527590a/0x1cd7fe5368733c49 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b275 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619906.774892:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6003100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589047424:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619906.775027:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccf00 going to sleep for 11 seconds 00002000:00010000:2.0:1788619906.786074:0:6696:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619905 -> 1788619906 00000100:00100000:2.0:1788619906.786122:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6003100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589047424:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619906.786174:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007527590a/0x1cd7fe5368733c49 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b275 expref: 8 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.786224:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007527590a/0x1cd7fe5368733c49 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b275 expref: 8 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.786260:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6800, rc 0) 00000100:00100000:2.0:1788619906.786823:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7581500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571320960:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 12072us (12365us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619906.786836:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 532 00000100:00100000:1.0:1788619906.790750:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619906.790830:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571321088 00000100:00100000:2.0:1788619906.790843:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571321088 00000100:00100000:2.0:1788619906.790924:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 533 00000100:00100000:2.0:1788619906.790937:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2353:x1875503571321088:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619906.790974:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619906.791002:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001dc9de0/0x1cd7fe5368733c50 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b27c expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619906.791085:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6003100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589047552:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619906.791207:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccf00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619906.796866:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6003100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589047552:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619906.796930:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000001dc9de0/0x1cd7fe5368733c50 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b27c expref: 8 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.796971:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001dc9de0/0x1cd7fe5368733c50 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b27c expref: 8 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.797027:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6a00, rc 0) 00000100:00100000:2.0:1788619906.797109:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2353:x1875503571321088:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6174us (6370us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619906.797119:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 533 00000100:00100000:1.0:1788619906.804322:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619906.805057:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571321216 00000100:00100000:0.0:1788619906.805071:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571321216 00000100:00100000:0.0:1788619906.805161:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 534 00000100:00100000:0.0:1788619906.805174:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+7:14908:x1875503571321216:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619906.805201:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619906.805239:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002d38f023/0x1cd7fe5368733c57 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b283 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.805292:0:7654:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000ca5ff342/0x1cd7fe5368733c34 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1110016->1122303) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b260 expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.805304:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(6 1) 00010000:00010000:0.0:1788619906.805378:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(7 1) 00010000:00010000:0.0:1788619906.805395:0:7654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d38f023/0x1cd7fe5368733c57 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1478656->1511423] (req 1478656->1511423) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b283 expref: 8 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.805446:0:7654:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000ca5ff342/0x1cd7fe5368733c34 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1110016->1122303) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b260 expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.805462:0:7654:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000ca5ff342/0x1cd7fe5368733c34 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1110016->1122303) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b260 expref: 5 pid: 6698 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619906.805478:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b16a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589047680:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619906.805582:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:2.0:1788619906.810095:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b16a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589047680:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619906.810148:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002d38f023/0x1cd7fe5368733c57 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1478656->1511423] (req 1478656->1511423) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b283 expref: 8 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.810175:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78ccca6e00, rc 0) 00000100:00100000:2.0:1788619906.813613:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+8:14908:x1875503571321216:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8421us (9287us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619906.813632:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 534 00000100:00100000:3.0:1788619906.814242:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619906.815387:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571321344 00000100:00100000:0.0:1788619906.815404:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571321344 00002000:00100000:0.0:1788619906.815472:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1110016->1122303) req@ffff8e78c7583480 x1875503571321344/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:2/0 lens 12776/0 e 0 to 0 dl 1788619917 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619906.815486:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7583480 with x1875503571321344 ext(1110016->1122303) 00010000:00010000:0.0:1788619906.815498:0:6701:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000ca5ff342/0x1cd7fe5368733c34 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1110016->1122303) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b260 expref: 6 pid: 6698 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.815517:0:6701:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000ca5ff342/0x1cd7fe5368733c34 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1110016->1122303) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b260 expref: 6 pid: 6698 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788619906.815526:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff8e78c7583480 00000100:00100000:0.0:1788619906.815564:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 8 00000100:00100000:0.0:1788619906.815574:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7583480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571321344:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:0.0:1788619906.816240:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7583480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+8:2355:x1875503571321344:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 652us (1987us total) trans 4294967370 rc 0/0 00000100:00100000:0.0:1788619906.816251:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 8 00002000:00100000:0.0:1788619906.816260:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1110016->1122303) req@ffff8e78c7583480 x1875503571321344/t4294967370(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:2/0 lens 12776/448 e 0 to 0 dl 1788619917 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619906.816270:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7583480 with x1875503571321344 ext(1110016->1122303) 00010000:00010000:0.0:1788619906.816278:0:6701:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000ca5ff342/0x1cd7fe5368733c34 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1110016->1122303) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b260 expref: 8 pid: 6698 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.816293:0:6701:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000ca5ff342/0x1cd7fe5368733c34 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1110016->1122303) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b260 expref: 8 pid: 6698 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788619906.816301:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff8e78c7583480 00000100:00100000:3.0:1788619906.827868:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619906.828161:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571321472 00000100:00100000:2.0:1788619906.828173:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571321472 00010000:00010000:2.0:1788619906.828232:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000ca5ff342/0x1cd7fe5368733c34 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1110016->1122303) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b260 expref: 8 pid: 6698 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619906.828293:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 225 00000100:00100000:2.0:1788619906.828305:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7582300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+8:2354:x1875503571321472:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619906.828671:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619906.828707:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000ca5ff342/0x1cd7fe5368733c34 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1110016->1122303) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b260 expref: 8 pid: 6698 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.828750:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000ca5ff342/0x1cd7fe5368733c34 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1110016->1122303) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b260 expref: 8 pid: 6698 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.828779:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca5ff342/0x1cd7fe5368733c34 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1110016->1122303) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b260 expref: 8 pid: 6698 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.828809:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(7 1) 00010000:00010000:2.0:1788619906.828813:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(7 1) 00010000:00010000:2.0:1788619906.828822:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000002d38f023/0x1cd7fe5368733c57 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1478656->1511423) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b283 expref: 7 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.828831:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d38f023/0x1cd7fe5368733c57 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1478656->1511423) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b283 expref: 7 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.828884:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000002d38f023/0x1cd7fe5368733c57 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1478656->1511423) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b283 expref: 7 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619906.828908:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6003100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589047808:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619906.829222:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccf00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619906.831952:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6003100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589047808:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619906.832126:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619906.832467:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7582300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571321472:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 4160us (4621us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619906.832484:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 225 00000100:00100000:3.0:1788619906.833820:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619906.833901:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571321600 00000100:00100000:0.0:1788619906.833914:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571321600 00002000:00100000:0.0:1788619906.834001:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1478656->1482751) req@ffff8e78df385880 x1875503571321600/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:2/0 lens 488/0 e 0 to 0 dl 1788619917 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619906.834016:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df385880 with x1875503571321600 ext(1478656->1482751) 00002000:00010000:0.0:1788619906.834024:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df385880 00000100:00100000:0.0:1788619906.834059:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 271 00000100:00100000:0.0:1788619906.834071:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df385880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571321600:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619906.834632:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df385880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571321600:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 560us (818us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619906.834647:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 271 00002000:00100000:0.0:1788619906.834669:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1478656->1482751) req@ffff8e78df385880 x1875503571321600/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:2/0 lens 488/4536 e 0 to 0 dl 1788619917 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619906.834677:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df385880 with x1875503571321600 ext(1478656->1482751) 00002000:00010000:0.0:1788619906.834680:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df385880 00000100:00100000:3.0:1788619906.840884:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619906.841091:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571321728 00000100:00100000:2.0:1788619906.841105:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571321728 00002000:00100000:2.0:1788619906.841181:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1507328->1511423) req@ffff8e78df387b80 x1875503571321728/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:2/0 lens 488/0 e 0 to 0 dl 1788619917 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619906.841194:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df387b80 with x1875503571321728 ext(1507328->1511423) 00002000:00010000:2.0:1788619906.841203:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df387b80 00000100:00100000:2.0:1788619906.841255:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 272 00000100:00100000:2.0:1788619906.841268:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df387b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571321728:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619906.842677:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df387b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571321728:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 1399us (1798us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619906.842694:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 272 00002000:00100000:1.0:1788619906.842708:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1507328->1511423) req@ffff8e78df387b80 x1875503571321728/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:2/0 lens 488/4536 e 0 to 0 dl 1788619917 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619906.842721:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df387b80 with x1875503571321728 ext(1507328->1511423) 00002000:00010000:1.0:1788619906.842729:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df387b80 00010000:00010000:3.0:1788619906.852225:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788619906.859980:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619906.860132:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571321856 00000100:00100000:3.0:1788619906.860145:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571321856 00002000:00100000:3.0:1788619906.860222:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6003100 x1875503571321856/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:2/0 lens 504/0 e 0 to 0 dl 1788619917 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619906.860247:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6003100 with x1875503571321856 00000100:00100000:3.0:1788619906.860298:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 380 00000100:00100000:3.0:1788619906.860311:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6003100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2353:x1875503571321856:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619906.864039:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619906.864619:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571321984 00000100:00100000:1.0:1788619906.864634:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571321984 00002000:00100000:1.0:1788619906.864783:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1974271) req@ffff8e77c6002300 x1875503571321984/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:2/0 lens 504/0 e 0 to 0 dl 1788619917 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619906.864809:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6002300 with x1875503571321984 ext(1048576->1974271) 00002000:00010000:1.0:1788619906.864817:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6002300 00000100:00100000:1.0:1788619906.864860:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 273 00000100:00100000:1.0:1788619906.864871:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6002300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571321984:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619906.892535:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6003100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2353:x1875503571321856:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 32210us (32563us total) trans 0 rc 835584/835584 00000100:00100000:3.0:1788619906.892555:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 380 00002000:00100000:3.0:1788619906.892572:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6003100 x1875503571321856/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:2/0 lens 504/440 e 0 to 0 dl 1788619917 ref 1 fl Complete:/600/0 rc 835584/835584 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619906.892604:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6003100 with x1875503571321856 00000100:00100000:2.0:1788619906.906816:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619906.906985:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571322112 00000100:00100000:3.0:1788619906.906999:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571322112 00000100:00100000:3.0:1788619906.907089:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 684 00000100:00100000:3.0:1788619906.907101:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6001f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571322112:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619906.907127:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619906.907153:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368733c5e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b28a expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619906.907195:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6002300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571321984:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 42325us (43172us total) trans 0 rc 892928/892928 00010000:00010000:3.0:1788619906.907202:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7200/0x1cd7fe5368733b3f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b16b expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619906.907206:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 273 00002000:00100000:1.0:1788619906.907220:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1974271) req@ffff8e77c6002300 x1875503571321984/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:2/0 lens 504/440 e 0 to 0 dl 1788619917 ref 1 fl Complete:/600/0 rc 892928/892928 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619906.907230:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6002300 with x1875503571321984 ext(1048576->1974271) 00002000:00010000:1.0:1788619906.907236:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6002300 00000100:00100000:3.0:1788619906.907276:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df385c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589047936:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619906.907389:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5c76b40 going to sleep for 11 seconds 00000100:00100000:2.0:1788619906.907711:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619906.908576:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571322240 00000100:00100000:1.0:1788619906.908589:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571322240 00000100:00100000:1.0:1788619906.908655:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 535 00000100:00100000:1.0:1788619906.908666:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6002680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571322240:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619906.908686:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619906.908704:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff771b43/0x1cd7fe5368733c65 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b291 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619906.908765:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df8b2680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589048064:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619906.908858:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd9636c0 going to sleep for 11 seconds 00000004:00010000:3.0:1788619906.912533:0:5845:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619904 -> 1788619906 00000100:00100000:3.0:1788619906.912591:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df385c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589047936:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619906.912630:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 888832 00010000:00010000:3.0:1788619906.912676:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368733c5e lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b28a expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619906.912709:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619906.912713:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619906.912721:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368733c5e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b28a expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619906.912743:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c600, rc 0) 00000100:00100000:3.0:1788619906.912860:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6001f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571322112:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5759us (6057us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619906.912871:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 684 00000100:00100000:0.0:1788619906.913023:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df8b2680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589048064:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619906.913059:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ff771b43/0x1cd7fe5368733c65 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b291 expref: 8 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.913078:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff771b43/0x1cd7fe5368733c65 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b291 expref: 8 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.913090:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa06400, rc 0) 00000100:00100000:0.0:1788619906.913134:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6002680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571322240:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4471us (5424us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619906.913142:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 535 00000100:00100000:1.0:1788619906.921084:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619906.921267:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571322368 00000100:00100000:0.0:1788619906.921281:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571322368 00000100:00100000:0.0:1788619906.921396:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 685 00000100:00100000:0.0:1788619906.921409:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571322368:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619906.921439:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619906.921468:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe5368733c6c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b298 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.921524:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7200/0x1cd7fe5368733b3f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b16b expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619906.921603:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b17800 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589048192:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619906.921731:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa540 going to sleep for 11 seconds 00000100:00100000:1.0:1788619906.925651:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619906.925916:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571322496 00000100:00100000:3.0:1788619906.925929:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571322496 00000100:00100000:3.0:1788619906.926011:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 536 00000100:00100000:3.0:1788619906.926035:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571322496:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619906.926063:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619906.926088:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe5368733c73 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b29f expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619906.926173:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df385c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589048320:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619906.926295:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5c76b40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619906.927282:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b17800 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589048192:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619906.927319:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 888832 00010000:00010000:0.0:1788619906.927389:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe5368733c6c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b298 expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.927416:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619906.927419:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619906.927426:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe5368733c6c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b298 expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.927444:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297000, rc 0) 00000100:00100000:0.0:1788619906.927520:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571322368:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6112us (6453us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619906.927530:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 685 00000100:00100000:3.0:1788619906.934299:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df385c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589048320:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619906.934395:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe5368733c73 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b29f expref: 8 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619906.934429:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe5368733c73 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b29f expref: 8 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619906.934446:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c600, rc 0) 00000100:00100000:3.0:1788619906.934521:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571322496:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8498us (8902us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619906.934532:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 536 00000100:00100000:2.0:1788619906.941009:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619906.941169:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571322752 00000100:00100000:3.0:1788619906.941203:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571322752 00000100:00100000:3.0:1788619906.941298:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 686 00000100:00100000:3.0:1788619906.941311:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6000e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571322752:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:3.0:1788619906.941431:0:9417:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619906.941471:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c505de00/0x1cd7fe5368733a90 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0bc expref: 9 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619906.941497:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619906.941504:0:9417:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788619906.941834:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368733c7a lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619906.941851:0:9417:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c0030400/0x1cd7fe5368733a82 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0b5 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619906.941862:0:9417:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368733c7a lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619906.941906:0:9417:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c0030400/0x1cd7fe5368733a82 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0b5 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619906.941925:0:9417:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c0030400/0x1cd7fe5368733a82 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0b5 expref: 13 pid: 5846 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619906.941943:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df385c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589048448:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619906.942065:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5c76b40 going to sleep for 11 seconds 00000100:00100000:3.0:1788619906.948859:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df385c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589048448:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619906.948907:0:9417:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368733c7a lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619906.950830:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619906.952104:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571322880 00000100:00100000:3.0:1788619906.952123:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571322880 00010000:00010000:2.0:1788619906.952455:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c0030400/0x1cd7fe5368733a82 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0b5 expref: 14 pid: 5846 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619906.952553:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 226 00000100:00100000:2.0:1788619906.952565:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6000000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571322880:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619906.952586:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619906.952595:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c0030400/0x1cd7fe5368733a82 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0b5 expref: 14 pid: 5846 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.952613:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c0030400/0x1cd7fe5368733a82 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0b5 expref: 14 pid: 5846 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.952689:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c0030400/0x1cd7fe5368733a82 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b0b5 expref: 14 pid: 5846 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.952714:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619906.952721:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368733c7a lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.952736:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368733c7a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.952746:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368733c7a lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.952921:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619906.953026:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6000000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571322880:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 461us (2208us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619906.953035:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 226 00010000:00010000:3.0:1788619906.953106:0:9417:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368733c7a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619906.953116:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368733c7a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788619906.953300:0:9417:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967565 00000004:00010000:3.0:1788619906.953395:0:9417:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 888832 -> 888832 00000004:00010000:3.0:1788619906.953398:0:9417:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619906 -> 1788619906 00000004:00010000:3.0:1788619906.953401:0:9417:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619906 -> 1788619904 00000004:00010000:3.0:1788619906.953404:0:9417:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619906 -> 1788619906 00000004:00010000:3.0:1788619906.953406:0:9417:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1158 -> 1158 00000004:00010000:3.0:1788619906.953414:0:9417:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000012d77741 reply state 00000000749c61b0 transno 4294967566 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368733c7a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619906.953424:0:9417:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788619906.953535:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6000e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571322752:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 12226us (12541us total) trans 4294967566 rc 0/0 00000100:00100000:3.0:1788619906.953545:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 686 00000020:00080000:3.0:1788619906.953549:0:9417:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:1.0:1788619906.957140:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619906.957696:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571323008 00000100:00100000:3.0:1788619906.957712:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571323008 00000100:00100000:3.0:1788619906.957783:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 687 00000100:00100000:3.0:1788619906.957921:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571323008:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619906.957952:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619906.957988:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368733c81 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b2a6 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619906.958062:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7200/0x1cd7fe5368733b3f lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b16b expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619906.958074:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368733c81 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2a6 expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619906.958121:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7200/0x1cd7fe5368733b3f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b16b expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619906.958138:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7200/0x1cd7fe5368733b3f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b16b expref: 12 pid: 11397 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619906.958155:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df385c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589048576:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619906.959594:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5c76b40 going to sleep for 11 seconds 00010000:00010000:0.0:1788619906.963762:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368733c7a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.963796:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368733c7a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.963806:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368733c7a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.963813:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c683c600 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788619906.963819:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368733c7a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.963858:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368733c7a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.963881:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619906.963884:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788619906.963891:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368733c7a lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.963899:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368733c7a lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619906.965400:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df385c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589048576:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619906.965453:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368733c81 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2a6 expref: 11 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619906.965478:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619906.965481:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619906.965486:0:5845:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368733c81 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2a6 expref: 11 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619906.965500:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d200, rc 0) 00000100:00100000:3.0:1788619906.965587:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571323008:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7669us (8459us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619906.965612:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 687 00000100:00100000:1.0:1788619906.969055:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619906.969151:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571323136 00000100:00100000:2.0:1788619906.969165:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571323136 00010000:00010000:2.0:1788619906.969239:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7200/0x1cd7fe5368733b3f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b16b expref: 13 pid: 11397 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619906.969296:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 227 00000100:00100000:2.0:1788619906.969309:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b1c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571323136:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619906.969368:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619906.969408:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7200/0x1cd7fe5368733b3f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b16b expref: 13 pid: 11397 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.969453:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7200/0x1cd7fe5368733b3f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b16b expref: 13 pid: 11397 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.969487:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7200/0x1cd7fe5368733b3f lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b16b expref: 13 pid: 11397 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.969513:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619906.969522:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368733c81 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2a6 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.969536:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368733c81 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2a6 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.969548:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368733c81 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2a6 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619906.969608:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368733c81 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2a6 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619906.969628:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4ba2d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589048704:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619906.969750:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc840 going to sleep for 11 seconds 00000100:00100000:0.0:1788619906.974608:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619906.974880:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4ba2d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589048704:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619906.974915:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619906.974996:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b1c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2355:x1875503571323136:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 5687us (5959us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619906.975031:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 227 00000100:00100000:1.0:1788619906.975568:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571323264 00000100:00100000:1.0:1788619906.975579:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571323264 00000100:00100000:1.0:1788619906.975655:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 537 00000100:00100000:1.0:1788619906.975666:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b17800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:14908:x1875503571323264:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619906.975688:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619906.975714:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0b04224/0x1cd7fe5368733c88 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b2ad expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619906.975754:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000002d38f023/0x1cd7fe5368733c57 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1478656->1511423) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b283 expref: 7 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619906.975765:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(7 1) 00010000:00010000:1.0:1788619906.975770:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(8 1) 00010000:00010000:1.0:1788619906.975786:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0b04224/0x1cd7fe5368733c88 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2ad expref: 8 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619906.975831:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000002d38f023/0x1cd7fe5368733c57 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1478656->1511423) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b283 expref: 7 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619906.975847:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000002d38f023/0x1cd7fe5368733c57 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1478656->1511423) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b283 expref: 7 pid: 7654 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619906.975865:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df8b0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589048832:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619906.975972:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd9636c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619906.982188:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df8b0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589048832:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619906.982253:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e0b04224/0x1cd7fe5368733c88 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2ad expref: 8 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619906.982284:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa06800, rc 0) 00000100:00100000:1.0:1788619906.982505:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b17800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+8:14908:x1875503571323264:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6839us (7911us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619906.982515:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 537 00000100:00100000:2.0:1788619906.984301:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619906.984598:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571323392 00000100:00100000:1.0:1788619906.984612:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571323392 00002000:00100000:1.0:1788619906.984688:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1478656->1511423) req@ffff8e77c4ba2d80 x1875503571323392/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:2/0 lens 488/0 e 0 to 0 dl 1788619917 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619906.984702:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4ba2d80 with x1875503571323392 ext(1478656->1511423) 00010000:00010000:1.0:1788619906.984715:0:6702:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000002d38f023/0x1cd7fe5368733c57 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1478656->1511423) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b283 expref: 8 pid: 7654 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619906.984753:0:6702:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000002d38f023/0x1cd7fe5368733c57 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1478656->1511423) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b283 expref: 8 pid: 7654 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788619906.984761:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff8e77c4ba2d80 00000100:00100000:1.0:1788619906.984876:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 9 00000100:00100000:1.0:1788619906.984889:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571323392:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:3.0:1788619906.990727:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+10:2354:x1875503571323392:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 5823us (6421us total) trans 4294967371 rc 0/0 00000100:00100000:3.0:1788619906.990746:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 9 00002000:00100000:3.0:1788619906.990762:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1478656->1511423) req@ffff8e77c4ba2d80 x1875503571323392/t4294967371(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:2/0 lens 488/448 e 0 to 0 dl 1788619917 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619906.990774:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4ba2d80 with x1875503571323392 ext(1478656->1511423) 00010000:00010000:3.0:1788619906.990915:0:6702:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000002d38f023/0x1cd7fe5368733c57 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1478656->1511423) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b283 expref: 10 pid: 7654 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619906.990941:0:6702:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000002d38f023/0x1cd7fe5368733c57 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1478656->1511423) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b283 expref: 10 pid: 7654 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788619906.990951:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff8e77c4ba2d80 00000100:00100000:2.0:1788619906.998015:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619906.998139:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571323520 00000100:00100000:0.0:1788619906.998154:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571323520 00010000:00010000:0.0:1788619906.998210:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000002d38f023/0x1cd7fe5368733c57 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1478656->1511423) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b283 expref: 10 pid: 7654 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619906.998275:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 228 00000100:00100000:0.0:1788619906.998288:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+10:2352:x1875503571323520:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619906.998308:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619906.998387:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000002d38f023/0x1cd7fe5368733c57 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1478656->1511423) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b283 expref: 10 pid: 7654 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.998405:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000002d38f023/0x1cd7fe5368733c57 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1478656->1511423) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b283 expref: 10 pid: 7654 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.998436:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d38f023/0x1cd7fe5368733c57 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1478656->1511423) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b283 expref: 10 pid: 7654 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.998473:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(8 1) 00010000:00010000:0.0:1788619906.998479:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(8 1) 00010000:00010000:0.0:1788619906.998492:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000e0b04224/0x1cd7fe5368733c88 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2ad expref: 7 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.998505:0:13330:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0b04224/0x1cd7fe5368733c88 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2ad expref: 7 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619906.998557:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000e0b04224/0x1cd7fe5368733c88 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2ad expref: 7 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619906.998574:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b17b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589048960:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619906.998698:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa540 going to sleep for 11 seconds 00000100:00100000:2.0:1788619907.002931:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619907.003189:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b17b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589048960:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619907.003243:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619907.003322:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2352:x1875503571323520:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 5035us (5322us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619907.003374:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 228 00000100:00100000:1.0:1788619907.003644:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571323648 00000100:00100000:1.0:1788619907.003654:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571323648 00002000:00010000:1.0:1788619907.003708:0:6701:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1378519->18446744073709551615) 00002000:00100000:1.0:1788619907.003713:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4ba3100 with x1875503571323648 ext(1378519->18446744073709551615) 00002000:00010000:1.0:1788619907.003727:0:6701:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4ba3100. 00000100:00100000:1.0:1788619907.003762:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 274 00000100:00100000:1.0:1788619907.003773:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+8:2355:x1875503571323648:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:1.0:1788619907.004290:0:6701:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1973002 -> 1378519 00002000:00010000:1.0:1788619907.004296:0:6701:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619906 -> 1788619906 00002000:00010000:1.0:1788619907.004299:0:6701:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619906 -> 1788619906 00002000:00010000:1.0:1788619907.004301:0:6701:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619906 -> 1788619906 00002000:00010000:1.0:1788619907.004304:0:6701:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 391 -> 391 00000100:00100000:1.0:1788619907.004400:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571323648:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 627us (1487us total) trans 4294967372 rc 0/0 00000100:00100000:1.0:1788619907.004408:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 274 00002000:00010000:1.0:1788619907.004411:0:6701:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1378519->18446744073709551615) 00002000:00100000:1.0:1788619907.004415:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4ba3100 with x1875503571323648 ext(1378519->18446744073709551615) 00002000:00010000:1.0:1788619907.004434:0:6701:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4ba3100. 00000100:00100000:1.0:1788619907.008683:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619907.008892:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571323776 00000100:00100000:0.0:1788619907.008915:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571323776 00000100:00100000:0.0:1788619907.009020:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 688 00000100:00100000:0.0:1788619907.009032:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571323776:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619907.009061:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619907.009110:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe5368733c8f lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b2b4 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.009242:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297c00/0x1cd7fe5368733c96 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.009263:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297c00/0x1cd7fe5368733c96 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.009279:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297c00/0x1cd7fe5368733c96 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619907.009317:0:11397:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1378519 00000004:00200000:0.0:1788619907.009322:0:11397:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619907.009402:0:11397:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297c00/0x1cd7fe5368733c96 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.009438:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe5368733c8f lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2b4 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.009460:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297c00/0x1cd7fe5368733c96 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2b4 expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.009469:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619907.009473:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619907.009478:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297c00, rc 0) 00000100:00100000:0.0:1788619907.009552:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+12:14908:x1875503571323776:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 522us (885us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619907.009559:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 688 00000100:00100000:2.0:1788619907.014578:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619907.014655:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571323904 00000100:00100000:3.0:1788619907.014666:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571323904 00000100:00100000:3.0:1788619907.014739:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 381 00000100:00100000:3.0:1788619907.014747:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571323904:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:3.0:1788619907.015137:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 888832 -> 888832 00000004:00010000:3.0:1788619907.015143:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619906 -> 1788619906 00000004:00010000:3.0:1788619907.015145:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619904 -> 1788619904 00000004:00010000:3.0:1788619907.015148:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619906 -> 1788619906 00000004:00010000:3.0:1788619907.015150:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1158 -> 1158 00000100:00100000:3.0:1788619907.015237:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571323904:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 488us (671us total) trans 4294967567 rc 0/0 00000100:00100000:3.0:1788619907.015266:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 381 00000100:00100000:2.0:1788619907.022158:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619907.022276:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619907.022518:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571324160 00000100:00100000:1.0:1788619907.022535:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571324160 00002000:00100000:1.0:1788619907.022613:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1380351) req@ffff8e78df965180 x1875503571324160/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:3/0 lens 488/0 e 0 to 0 dl 1788619918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619907.022627:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df965180 with x1875503571324160 ext(1048576->1380351) 00002000:00010000:1.0:1788619907.022635:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df965180 00000100:00100000:1.0:1788619907.022666:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 275 00000100:00100000:1.0:1788619907.022678:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df965180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571324160:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619907.028142:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571324032 00000100:00100000:3.0:1788619907.028177:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571324032 00002000:00100000:2.0:1788619907.029498:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4ba3480 x1875503571324032/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:3/0 lens 488/0 e 0 to 0 dl 1788619918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619907.029513:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4ba3480 with x1875503571324032 00000100:00100000:2.0:1788619907.029572:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 382 00000100:00100000:2.0:1788619907.029584:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571324032:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619907.031546:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df965180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571324160:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 8856us (9255us total) trans 0 rc 331776/331776 00000100:00100000:0.0:1788619907.031583:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 275 00002000:00100000:0.0:1788619907.031600:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1380351) req@ffff8e78df965180 x1875503571324160/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:3/0 lens 488/440 e 0 to 0 dl 1788619918 ref 1 fl Complete:/600/0 rc 331776/331776 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619907.031612:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df965180 with x1875503571324160 ext(1048576->1380351) 00002000:00010000:0.0:1788619907.031619:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df965180 00000100:00100000:2.0:1788619907.043522:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571324032:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 13921us (21362us total) trans 0 rc 888832/888832 00000100:00100000:2.0:1788619907.043542:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 382 00002000:00100000:2.0:1788619907.043562:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4ba3480 x1875503571324032/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:3/0 lens 488/440 e 0 to 0 dl 1788619918 ref 1 fl Complete:/600/0 rc 888832/888832 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619907.043576:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4ba3480 with x1875503571324032 00000100:00100000:3.0:1788619907.046508:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619907.046697:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571324288 00000100:00100000:2.0:1788619907.046710:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571324288 00000100:00100000:2.0:1788619907.046834:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 689 00000100:00100000:2.0:1788619907.046847:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df385c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+12:14908:x1875503571324288:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619907.046876:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619907.046905:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe5368733c9d lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b2bb expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.047064:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6200/0x1cd7fe5368733ca4 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.047083:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6200/0x1cd7fe5368733ca4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.047097:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6200/0x1cd7fe5368733ca4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788619907.047134:0:9417:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1378519 00000004:00200000:2.0:1788619907.047140:0:9417:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788619907.047161:0:9417:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6200/0x1cd7fe5368733ca4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.047195:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe5368733c9d lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2bb expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.047217:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6200/0x1cd7fe5368733ca4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2bb expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.047235:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619907.047238:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788619907.047241:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6200, rc 0) 00000100:00100000:2.0:1788619907.047431:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df385c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571324288:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 585us (936us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619907.047440:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 689 00000100:00100000:3.0:1788619907.050605:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619907.050667:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619907.050723:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571324544 00000100:00100000:2.0:1788619907.050735:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571324544 00000100:00100000:2.0:1788619907.050825:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 538 00000100:00100000:2.0:1788619907.050837:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df387480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571324544:12345-192.168.201.5@tcp:101:fsx.0 00000100:00100000:0.0:1788619907.050860:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571324416 00000100:00100000:0.0:1788619907.050865:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571324416 00010000:00010000:2.0:1788619907.050870:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619907.050901:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3b293bd/0x1cd7fe5368733cab lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b2c9 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619907.050910:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 690 00000100:00100000:0.0:1788619907.050918:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df385500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571324416:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619907.050929:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619907.050944:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe5368733cb2 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b2c2 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.050989:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368733c81 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2a6 expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619907.050990:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df966d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589049088:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619907.051016:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b17b80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589049216:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619907.051102:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619907.056078:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa540 going to sleep for 11 seconds 00000100:00100000:0.0:1788619907.059156:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df966d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589049088:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619907.059214:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f3b293bd/0x1cd7fe5368733cab lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2c9 expref: 10 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.059308:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3b293bd/0x1cd7fe5368733cab lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2c9 expref: 10 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.059452:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7e00, rc 0) 00000100:00100000:0.0:1788619907.059602:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df387480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571324544:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8766us (8933us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619907.059613:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 538 00000004:00010000:2.0:1788619907.060197:0:5845:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619904 -> 1788619906 00000100:00100000:2.0:1788619907.060238:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b17b80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589049216:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619907.060263:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 888832 00010000:00010000:2.0:1788619907.060279:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe5368733cb2 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2c2 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.060299:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619907.060302:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619907.060309:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe5368733cb2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2c2 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.060368:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297e00, rc 0) 00000100:00100000:2.0:1788619907.062200:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df385500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571324416:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 11266us (11594us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619907.062215:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 690 00000100:00100000:2.0:1788619907.065951:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619907.066016:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619907.066163:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571324672 00000100:00100000:1.0:1788619907.066178:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571324672 00000100:00100000:1.0:1788619907.066319:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 691 00000100:00100000:1.0:1788619907.066351:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df964e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571324672:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619907.066382:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619907.066412:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe5368733cb9 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b2d0 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619907.066469:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368733c81 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2a6 expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619907.066521:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df8b1180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589049344:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619907.066628:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd9636c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619907.069628:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571324800 00000100:00100000:3.0:1788619907.069650:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571324800 00000100:00100000:3.0:1788619907.069738:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 539 00000100:00100000:3.0:1788619907.069751:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df967b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2355:x1875503571324800:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619907.069776:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619907.069811:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007da7fefa/0x1cd7fe5368733cc0 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b2d7 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619907.069895:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df387800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589049472:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619907.070264:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5c76b40 going to sleep for 11 seconds 00000100:00100000:3.0:1788619907.072881:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df8b1180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589049344:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619907.072929:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 888832 00010000:00010000:3.0:1788619907.072967:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe5368733cb9 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2d0 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619907.072995:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619907.072998:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619907.073006:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe5368733cb9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2d0 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619907.073024:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa06400, rc 0) 00000100:00100000:3.0:1788619907.075576:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df964e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571324672:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 9209us (9540us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619907.075595:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 691 00000100:00100000:3.0:1788619907.077686:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df387800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589049472:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619907.077741:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007da7fefa/0x1cd7fe5368733cc0 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2d7 expref: 10 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619907.077773:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007da7fefa/0x1cd7fe5368733cc0 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2d7 expref: 10 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619907.077802:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c1495000, rc 0) 00000100:00100000:3.0:1788619907.077893:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df967b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2355:x1875503571324800:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8144us (11959us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619907.077912:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 539 00000100:00100000:2.0:1788619907.085061:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619907.086760:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571325056 00000100:00100000:3.0:1788619907.086778:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571325056 00000100:00100000:2.0:1788619907.087300:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 692 00000100:00100000:2.0:1788619907.087365:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df967480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571325056:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:2.0:1788619907.087444:0:9417:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619907.087483:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6200/0x1cd7fe5368733ca4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2bb expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.087506:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619907.087513:0:9417:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788619907.087803:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe5368733cc7 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.087823:0:9417:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297c00/0x1cd7fe5368733c96 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2b4 expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.087833:0:9417:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe5368733cc7 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.087877:0:9417:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297c00/0x1cd7fe5368733c96 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2b4 expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.087895:0:9417:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297c00/0x1cd7fe5368733c96 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2b4 expref: 12 pid: 11397 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619907.087910:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df966300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589049600:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619907.088044:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc300 going to sleep for 11 seconds 00000100:00100000:2.0:1788619907.090804:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df966300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589049600:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619907.090845:0:9417:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe5368733cc7 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619907.091649:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619907.091875:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571325184 00000100:00100000:0.0:1788619907.091886:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571325184 00010000:00010000:0.0:1788619907.091937:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297c00/0x1cd7fe5368733c96 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2b4 expref: 13 pid: 11397 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619907.091997:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 229 00000100:00100000:0.0:1788619907.092009:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8022c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+13:2355:x1875503571325184:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619907.092028:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619907.092036:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297c00/0x1cd7fe5368733c96 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2b4 expref: 13 pid: 11397 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.092051:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297c00/0x1cd7fe5368733c96 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2b4 expref: 13 pid: 11397 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.092077:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297c00/0x1cd7fe5368733c96 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2b4 expref: 13 pid: 11397 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.092097:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619907.092103:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe5368733cc7 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.092114:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe5368733cc7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.092135:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe5368733cc7 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.092177:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619907.092254:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8022c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571325184:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 247us (609us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619907.092262:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 229 00010000:00010000:3.0:1788619907.092506:0:9417:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe5368733cc7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619907.092515:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe5368733cc7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788619907.092672:0:9417:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967568 00000004:00010000:3.0:1788619907.092737:0:9417:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 888832 -> 888832 00000004:00010000:3.0:1788619907.092741:0:9417:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619906 -> 1788619906 00000004:00010000:3.0:1788619907.092743:0:9417:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619906 -> 1788619904 00000004:00010000:3.0:1788619907.092746:0:9417:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619906 -> 1788619906 00000004:00010000:3.0:1788619907.092756:0:9417:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1158 -> 1158 00000004:00010000:3.0:1788619907.092765:0:9417:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000009c99cb3a reply state 000000001d0d1955 transno 4294967569 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe5368733cc7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619907.092776:0:9417:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788619907.092879:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df967480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:14908:x1875503571325056:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 5550us (7831us total) trans 4294967569 rc 0/0 00000100:00100000:3.0:1788619907.092887:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 692 00000020:00080000:3.0:1788619907.092891:0:9417:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1788619907.096826:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619907.096901:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571325312 00000100:00100000:0.0:1788619907.096914:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571325312 00000100:00100000:0.0:1788619907.096997:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 693 00000100:00100000:0.0:1788619907.097009:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df966300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:14908:x1875503571325312:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619907.097042:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619907.097072:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe5368733cce lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b2de expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.097114:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368733c81 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2a6 expref: 11 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.097126:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe5368733cce lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2de expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.097170:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368733c81 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2a6 expref: 11 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.097186:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368733c81 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2a6 expref: 11 pid: 5845 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619907.097231:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b14380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589049728:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619907.097368:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa0c0 going to sleep for 11 seconds 00010000:00010000:3.0:1788619907.097890:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe5368733cc7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619907.097902:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe5368733cc7 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619907.097910:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe5368733cc7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619907.097917:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78cdcc6a00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:3.0:1788619907.097923:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe5368733cc7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619907.097932:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe5368733cc7 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619907.097951:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619907.097954:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788619907.097971:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe5368733cc7 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619907.097978:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe5368733cc7 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619907.103437:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b14380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589049728:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619907.103492:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe5368733cce lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2de expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.103513:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619907.103517:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619907.103522:0:5845:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe5368733cce lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2de expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.103533:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297e00, rc 0) 00000100:00100000:2.0:1788619907.103610:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df966300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:14908:x1875503571325312:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6602us (6796us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619907.103620:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 693 00000100:00100000:1.0:1788619907.106061:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619907.106173:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571325440 00000100:00100000:2.0:1788619907.106186:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571325440 00010000:00010000:2.0:1788619907.106255:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368733c81 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2a6 expref: 12 pid: 5845 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619907.106306:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 230 00000100:00100000:2.0:1788619907.106318:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571325440:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619907.106386:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619907.106423:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368733c81 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2a6 expref: 12 pid: 5845 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.106470:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368733c81 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2a6 expref: 12 pid: 5845 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.106498:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368733c81 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2a6 expref: 12 pid: 5845 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.106520:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619907.106527:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe5368733cce lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2de expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.106539:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe5368733cce lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2de expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.106550:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe5368733cce lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2de expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.106604:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe5368733cce lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2de expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619907.106623:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df967800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589049856:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619907.106729:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc300 going to sleep for 11 seconds 00000100:00100000:0.0:1788619907.113530:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619907.113667:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df967800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589049856:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619907.113701:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619907.113776:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571325440:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 7459us (7729us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619907.113812:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 230 00000100:00100000:3.0:1788619907.113915:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571325568 00000100:00100000:3.0:1788619907.113924:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571325568 00000100:00100000:3.0:1788619907.113996:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 540 00000100:00100000:3.0:1788619907.114007:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b15c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+9:14908:x1875503571325568:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619907.114030:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619907.114055:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007da7fefa/0x1cd7fe5368733cd5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b2e5 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619907.114096:0:6696:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000e0b04224/0x1cd7fe5368733c88 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2ad expref: 8 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619907.114107:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(8 1) 00010000:00010000:3.0:1788619907.114112:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(9 1) 00010000:00010000:3.0:1788619907.114118:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007da7fefa/0x1cd7fe5368733cd5 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2e5 expref: 10 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619907.114158:0:6696:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000e0b04224/0x1cd7fe5368733c88 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2ad expref: 8 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619907.114176:0:6696:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000e0b04224/0x1cd7fe5368733c88 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2ad expref: 8 pid: 6697 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619907.114193:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df387800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589049984:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619907.114259:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5c76b40 going to sleep for 11 seconds 00000100:00100000:3.0:1788619907.120613:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df387800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589049984:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619907.120663:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007da7fefa/0x1cd7fe5368733cd5 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2e5 expref: 10 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619907.120694:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c1495000, rc 0) 00000100:00100000:3.0:1788619907.120776:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b15c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+10:14908:x1875503571325568:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6770us (7262us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619907.120795:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 540 00000100:00100000:1.0:1788619907.123707:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619907.123840:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571325696 00000100:00100000:0.0:1788619907.123855:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571325696 00010000:00010000:0.0:1788619907.123914:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e0b04224/0x1cd7fe5368733c88 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2ad expref: 9 pid: 6697 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619907.123979:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 231 00000100:00100000:0.0:1788619907.123991:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b3800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571325696:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619907.124015:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619907.124046:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000e0b04224/0x1cd7fe5368733c88 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2ad expref: 9 pid: 6697 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.124064:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e0b04224/0x1cd7fe5368733c88 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2ad expref: 9 pid: 6697 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.124093:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0b04224/0x1cd7fe5368733c88 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2ad expref: 9 pid: 6697 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.124118:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(9 1) 00010000:00010000:0.0:1788619907.124123:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(9 1) 00010000:00010000:0.0:1788619907.124135:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000007da7fefa/0x1cd7fe5368733cd5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2e5 expref: 9 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.124172:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007da7fefa/0x1cd7fe5368733cd5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2e5 expref: 9 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.124236:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000007da7fefa/0x1cd7fe5368733cd5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2e5 expref: 9 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619907.124254:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b14000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589050112:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619907.124434:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaab40 going to sleep for 11 seconds 00000100:00100000:2.0:1788619907.128965:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b14000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589050112:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619907.129003:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619907.129077:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b3800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+8:2355:x1875503571325696:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 5087us (5388us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619907.129090:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 231 00000100:00100000:3.0:1788619907.135090:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619907.135150:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619907.135403:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571325952 00000100:00100000:0.0:1788619907.135418:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571325952 00002000:00010000:0.0:1788619907.135473:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:0.0:1788619907.135479:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df384a80 with x1875503571325952 ext(0->18446744073709551615) 00002000:00010000:0.0:1788619907.135495:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df384a80. 00000100:00100000:0.0:1788619907.135636:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 276 00000100:00100000:0.0:1788619907.135650:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df384a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+10:2353:x1875503571325952:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:0.0:1788619907.136083:0:6703:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1378519 -> 0 00002000:00010000:0.0:1788619907.136088:0:6703:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619906 -> 1788619906 00002000:00010000:0.0:1788619907.136091:0:6703:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619906 -> 1788619906 00002000:00010000:0.0:1788619907.136094:0:6703:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619906 -> 1788619906 00002000:00010000:0.0:1788619907.136096:0:6703:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 391 -> 391 00000100:00100000:0.0:1788619907.136188:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df384a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+11:2353:x1875503571325952:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 540us (1036us total) trans 4294967373 rc 0/0 00000100:00100000:0.0:1788619907.136198:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 276 00002000:00010000:0.0:1788619907.136235:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:0.0:1788619907.136238:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df384a80 with x1875503571325952 ext(0->18446744073709551615) 00002000:00010000:0.0:1788619907.136243:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df384a80. 00000100:00100000:1.0:1788619907.137163:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571325824 00000100:00100000:1.0:1788619907.137188:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571325824 00002000:00010000:1.0:1788619907.137244:0:5859:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:1.0:1788619907.137251:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df387800 with x1875503571325824 00000100:00100000:1.0:1788619907.137294:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 383 00000100:00100000:1.0:1788619907.137305:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df387800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2353:x1875503571325824:12345-192.168.201.5@tcp:10:fsx.0 00000004:00010000:1.0:1788619907.137925:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 888832 -> 358559 00000004:00010000:1.0:1788619907.137932:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619906 -> 1788619906 00000004:00010000:1.0:1788619907.137936:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619904 -> 1788619904 00000004:00010000:1.0:1788619907.137938:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619906 -> 1788619906 00000004:00010000:1.0:1788619907.137940:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1158 -> 1158 00000100:00100000:1.0:1788619907.138166:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df387800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2353:x1875503571325824:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 859us (3088us total) trans 4294967570 rc 0/0 00000100:00100000:1.0:1788619907.138177:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 383 00002000:00010000:1.0:1788619907.138181:0:5859:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:1.0:1788619907.138184:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df387800 with x1875503571325824 00000100:00100000:3.0:1788619907.142729:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619907.143009:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571326080 00000100:00100000:2.0:1788619907.143025:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571326080 00000100:00100000:2.0:1788619907.143108:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 694 00000100:00100000:2.0:1788619907.143121:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df386300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:14908:x1875503571326080:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619907.143175:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619907.143211:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6200/0x1cd7fe5368733cdc lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b2ec expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.143390:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe5368733ce3 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.143412:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe5368733ce3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.143426:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe5368733ce3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788619907.143463:0:5846:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 358559 00000004:00200000:2.0:1788619907.143469:0:5846:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788619907.143494:0:5846:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe5368733ce3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.143551:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6200/0x1cd7fe5368733cdc lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2ec expref: 18 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.143574:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe5368733ce3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2ec expref: 17 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.143585:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619907.143588:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788619907.143593:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6a00, rc 0) 00000100:00100000:2.0:1788619907.143677:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df386300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+17:14908:x1875503571326080:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 558us (963us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619907.143685:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 694 00000100:00100000:2.0:1788619907.148472:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619907.148712:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571326208 00000100:00100000:3.0:1788619907.148725:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571326208 00000100:00100000:3.0:1788619907.148811:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 695 00000100:00100000:3.0:1788619907.148825:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df967800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571326208:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619907.148852:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619907.148881:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1495a00/0x1cd7fe5368733cea lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b2f3 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619907.149020:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1494200/0x1cd7fe5368733cf1 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619907.149042:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1494200/0x1cd7fe5368733cf1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619907.149058:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1494200/0x1cd7fe5368733cf1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788619907.149118:0:9417:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 358559 00000004:00200000:3.0:1788619907.149122:0:9417:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788619907.149145:0:9417:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1494200/0x1cd7fe5368733cf1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619907.149180:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1495a00/0x1cd7fe5368733cea lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2f3 expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619907.149201:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1494200/0x1cd7fe5368733cf1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2f3 expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619907.149212:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619907.149216:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788619907.149392:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c1494200, rc 0) 00000100:00100000:3.0:1788619907.149483:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df967800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+12:14908:x1875503571326208:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 661us (1029us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619907.149492:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 695 00000100:00100000:3.0:1788619907.152987:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619907.153148:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571326336 00000100:00100000:2.0:1788619907.153163:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571326336 00000100:00100000:2.0:1788619907.153255:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 696 00000100:00100000:2.0:1788619907.153269:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df385f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571326336:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619907.153298:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619907.153377:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6200/0x1cd7fe5368733cf8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b2fa expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.153428:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe5368733cce lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2de expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619907.153483:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df966680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589050240:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619907.153594:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc300 going to sleep for 11 seconds 00000100:00100000:1.0:1788619907.157067:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000004:00010000:0.0:1788619907.157270:0:5845:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619904 -> 1788619906 00000100:00100000:0.0:1788619907.157306:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df966680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589050240:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619907.157380:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 358559 00010000:00010000:0.0:1788619907.157408:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6200/0x1cd7fe5368733cf8 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2fa expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.157435:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619907.157439:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619907.157446:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6200/0x1cd7fe5368733cf8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2fa expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.157464:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6200, rc 0) 00000100:00100000:0.0:1788619907.157542:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df385f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571326336:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4276us (4572us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619907.157552:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 696 00000100:00100000:2.0:1788619907.158924:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571326464 00000100:00100000:2.0:1788619907.158941:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571326464 00000100:00100000:2.0:1788619907.159033:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 541 00000100:00100000:2.0:1788619907.159045:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+8:2355:x1875503571326464:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619907.159068:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619907.159090:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3b293bd/0x1cd7fe5368733cff lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b301 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619907.159167:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df965f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589050368:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619907.159293:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619907.164464:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df965f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589050368:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619907.164527:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f3b293bd/0x1cd7fe5368733cff lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b301 expref: 9 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.164561:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3b293bd/0x1cd7fe5368733cff lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b301 expref: 9 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.164577:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7e00, rc 0) 00000100:00100000:2.0:1788619907.164657:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+8:2355:x1875503571326464:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5614us (7604us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619907.164667:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 541 00000100:00100000:0.0:1788619907.172623:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619907.172680:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619907.172794:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571326592 00000100:00100000:1.0:1788619907.172807:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571326592 00000100:00100000:1.0:1788619907.172898:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 697 00000100:00100000:1.0:1788619907.172917:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b14000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571326592:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619907.172947:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619907.172986:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07600/0x1cd7fe5368733d06 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b308 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619907.173037:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe5368733cce lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2de expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619907.173116:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df8b3480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589050496:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619907.173231:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd9633c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619907.173808:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571326720 00000100:00100000:2.0:1788619907.173815:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571326720 00000100:00100000:2.0:1788619907.173877:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 542 00000100:00100000:2.0:1788619907.173887:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b14380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+8:2354:x1875503571326720:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619907.173915:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619907.173931:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3b293bd/0x1cd7fe5368733d0d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b30f expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619907.173998:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df965f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589050624:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619907.174051:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619907.179463:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df965f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589050624:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619907.179521:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f3b293bd/0x1cd7fe5368733d0d lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b30f expref: 9 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.179553:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3b293bd/0x1cd7fe5368733d0d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b30f expref: 9 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.179571:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7e00, rc 0) 00000100:00100000:2.0:1788619907.179644:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b14380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+8:2354:x1875503571326720:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5758us (6962us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619907.179654:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 542 00000100:00100000:1.0:1788619907.180241:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df8b3480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589050496:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619907.180263:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 358559 00010000:00010000:1.0:1788619907.180282:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07600/0x1cd7fe5368733d06 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b308 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619907.180301:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619907.180304:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619907.180310:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07600/0x1cd7fe5368733d06 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b308 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619907.180371:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa07600, rc 0) 00000100:00100000:1.0:1788619907.180420:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b14000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571326592:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7507us (7811us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619907.180428:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 697 00000100:00100000:0.0:1788619907.189630:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619907.189725:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571326848 00000100:00100000:2.0:1788619907.189739:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571326848 00002000:00100000:2.0:1788619907.189819:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b17b80 x1875503571326848/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:3/0 lens 488/0 e 0 to 0 dl 1788619918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619907.189832:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b17b80 with x1875503571326848 00000100:00100000:2.0:1788619907.189875:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 384 00000100:00100000:2.0:1788619907.189887:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b17b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2353:x1875503571326848:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619907.227805:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b17b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2353:x1875503571326848:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 37892us (38162us total) trans 0 rc 360448/360448 00000100:00100000:2.0:1788619907.227824:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 384 00002000:00100000:2.0:1788619907.228020:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b17b80 x1875503571326848/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:3/0 lens 488/440 e 0 to 0 dl 1788619918 ref 1 fl Complete:/600/0 rc 360448/360448 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619907.228034:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b17b80 with x1875503571326848 00000100:00100000:3.0:1788619907.235431:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619907.235556:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571327104 00000100:00100000:2.0:1788619907.235570:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571327104 00000100:00100000:2.0:1788619907.235644:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 698 00000100:00100000:2.0:1788619907.235657:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df387100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+17:14908:x1875503571327104:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:2.0:1788619907.235735:0:5844:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619907.235790:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe5368733ce3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2ec expref: 17 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.235815:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619907.235821:0:5844:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788619907.236734:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7e00/0x1cd7fe5368733d14 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.236761:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1494200/0x1cd7fe5368733cf1 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2f3 expref: 11 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.236771:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7e00/0x1cd7fe5368733d14 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.236817:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1494200/0x1cd7fe5368733cf1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2f3 expref: 11 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.237565:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1494200/0x1cd7fe5368733cf1 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2f3 expref: 11 pid: 9417 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619907.237588:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df965f80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589050752:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619907.239448:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619907.243609:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619907.243665:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571327232 00000100:00100000:0.0:1788619907.243676:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571327232 00010000:00010000:0.0:1788619907.243738:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1494200/0x1cd7fe5368733cf1 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2f3 expref: 12 pid: 9417 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619907.244735:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df965f80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589050752:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619907.244771:0:5844:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7e00/0x1cd7fe5368733d14 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619907.245519:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 232 00000100:00100000:0.0:1788619907.245561:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b16a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571327232:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619907.245587:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619907.245599:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1494200/0x1cd7fe5368733cf1 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2f3 expref: 12 pid: 9417 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.245617:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1494200/0x1cd7fe5368733cf1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2f3 expref: 12 pid: 9417 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.245646:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1494200/0x1cd7fe5368733cf1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2f3 expref: 12 pid: 9417 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.245666:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619907.245672:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7e00/0x1cd7fe5368733d14 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.245684:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7e00/0x1cd7fe5368733d14 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.245694:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7e00/0x1cd7fe5368733d14 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.245729:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619907.245810:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b16a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571327232:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 257us (2215us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619907.245820:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 232 00010000:00010000:3.0:1788619907.246150:0:5844:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7e00/0x1cd7fe5368733d14 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619907.246161:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7e00/0x1cd7fe5368733d14 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788619907.246376:0:5844:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967571 00000004:00010000:3.0:1788619907.246444:0:5844:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 358559 -> 358559 00000004:00010000:3.0:1788619907.246449:0:5844:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619906 -> 1788619907 00000004:00010000:3.0:1788619907.246452:0:5844:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619906 -> 1788619904 00000004:00010000:3.0:1788619907.246455:0:5844:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619906 -> 1788619907 00000004:00010000:3.0:1788619907.246457:0:5844:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1158 -> 1158 00000004:00010000:3.0:1788619907.246465:0:5844:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000009bf5d9d0 reply state 00000000089c6b44 transno 4294967572 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7e00/0x1cd7fe5368733d14 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619907.246475:0:5844:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788619907.246567:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df387100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+19:14908:x1875503571327104:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 10912us (11152us total) trans 4294967572 rc 0/0 00000100:00100000:3.0:1788619907.246575:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 698 00000020:00080000:3.0:1788619907.246579:0:5844:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788619907.249423:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7e00/0x1cd7fe5368733d14 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.249445:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7e00/0x1cd7fe5368733d14 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.249473:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7e00/0x1cd7fe5368733d14 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.249480:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78cdcc7e00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788619907.249486:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7e00/0x1cd7fe5368733d14 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.249495:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7e00/0x1cd7fe5368733d14 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.249516:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619907.249520:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788619907.249527:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7e00/0x1cd7fe5368733d14 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.249534:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7e00/0x1cd7fe5368733d14 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619907.252166:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619907.252316:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571327360 00000100:00100000:0.0:1788619907.252364:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571327360 00002000:00010000:0.0:1788619907.252424:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1633436->18446744073709551615) 00002000:00100000:0.0:1788619907.252430:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df386a00 with x1875503571327360 ext(1633436->18446744073709551615) 00002000:00010000:0.0:1788619907.252445:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df386a00. 00000100:00100000:0.0:1788619907.252484:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 277 00000100:00100000:0.0:1788619907.252494:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df386a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+11:2352:x1875503571327360:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:0.0:1788619907.252798:0:6702:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 0 -> 1633436 00002000:00010000:0.0:1788619907.252803:0:6702:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619906 -> 1788619907 00002000:00010000:0.0:1788619907.252806:0:6702:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619906 -> 1788619906 00002000:00010000:0.0:1788619907.252823:0:6702:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619906 -> 1788619907 00002000:00010000:0.0:1788619907.252826:0:6702:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 391 -> 391 00000100:00100000:0.0:1788619907.252947:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df386a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2352:x1875503571327360:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 453us (793us total) trans 4294967374 rc 0/0 00000100:00100000:0.0:1788619907.252966:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 277 00002000:00010000:0.0:1788619907.252970:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1633436->18446744073709551615) 00002000:00100000:0.0:1788619907.252973:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df386a00 with x1875503571327360 ext(1633436->18446744073709551615) 00002000:00010000:0.0:1788619907.252978:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df386a00. 00000100:00100000:3.0:1788619907.262107:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619907.265818:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571327488 00000100:00100000:0.0:1788619907.265963:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571327488 00000100:00100000:0.0:1788619907.267469:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 699 00000100:00100000:0.0:1788619907.267496:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df384e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+18:14908:x1875503571327488:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619907.267538:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619907.267566:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297800/0x1cd7fe5368733d1b lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b316 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.267696:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe5368733d22 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.267720:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe5368733d22 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.267736:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe5368733d22 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619907.267775:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1633436 00000004:00200000:0.0:1788619907.267779:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619907.267802:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe5368733d22 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.267977:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297800/0x1cd7fe5368733d1b lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b316 expref: 20 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.268000:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe5368733d22 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b316 expref: 19 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.268010:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619907.268013:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619907.268017:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297000, rc 0) 00000100:00100000:0.0:1788619907.268105:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df384e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+19:14908:x1875503571327488:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 624us (6779us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619907.268112:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 699 00000100:00100000:1.0:1788619907.277806:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619907.278062:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571327616 00000100:00100000:3.0:1788619907.278077:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571327616 00000100:00100000:3.0:1788619907.278241:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 385 00000100:00100000:3.0:1788619907.278254:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2354:x1875503571327616:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:3.0:1788619907.278660:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 358559 -> 358559 00000004:00010000:3.0:1788619907.278666:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619907 -> 1788619906 00000004:00010000:3.0:1788619907.278668:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619904 -> 1788619907 00000004:00010000:3.0:1788619907.278671:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619907 -> 1788619906 00000004:00010000:3.0:1788619907.278674:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1158 -> 1158 00000100:00100000:3.0:1788619907.278772:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2354:x1875503571327616:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 519us (979us total) trans 4294967573 rc 0/0 00000100:00100000:3.0:1788619907.278801:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 385 00000100:00100000:1.0:1788619907.285011:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619907.285157:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571327744 00000100:00100000:0.0:1788619907.285169:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571327744 00000100:00100000:0.0:1788619907.285236:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 700 00000100:00100000:0.0:1788619907.285249:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571327744:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619907.285277:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619907.285301:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297800/0x1cd7fe5368733d29 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b31d expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.285386:0:11397:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe5368733cce lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2de expref: 19 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.285396:0:11397:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297800/0x1cd7fe5368733d29 lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b31d expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.285454:0:11397:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe5368733cce lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2de expref: 19 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.285465:0:11397:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe5368733cce lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2de expref: 19 pid: 5845 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619907.285478:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b16d80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589050880:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619907.285561:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa180 going to sleep for 11 seconds 00000100:00100000:2.0:1788619907.290651:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619907.290773:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571327872 00000100:00100000:1.0:1788619907.290786:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571327872 00010000:00010000:1.0:1788619907.290842:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe5368733cce lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2de expref: 20 pid: 5845 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619907.290908:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 233 00000100:00100000:1.0:1788619907.290922:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df965f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2354:x1875503571327872:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619907.290975:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619907.291007:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe5368733cce lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2de expref: 20 pid: 5845 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619907.291020:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe5368733cce lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2de expref: 20 pid: 5845 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619907.291044:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619907.291049:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297800/0x1cd7fe5368733d29 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b31d expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619907.291058:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297800/0x1cd7fe5368733d29 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b31d expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619907.291066:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297800/0x1cd7fe5368733d29 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b31d expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619907.291105:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297800/0x1cd7fe5368733d29 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b31d expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619907.291116:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df8b1c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589051008:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619907.291207:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd9633c0 going to sleep for 11 seconds 00010000:00010000:0.0:1788619907.291674:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe5368733cce lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2de expref: 20 pid: 5845 timeout: 499 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619907.291697:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b16d80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589050880:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619907.291728:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297800/0x1cd7fe5368733d29 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b31d expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.291743:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619907.291746:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619907.291749:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297800, rc 0) 00000100:00100000:0.0:1788619907.291790:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+12:14908:x1875503571327744:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6543us (6790us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619907.291799:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 700 00000100:00100000:1.0:1788619907.299065:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df8b1c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589051008:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619907.299103:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619907.299183:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df965f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2354:x1875503571327872:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 8261us (8545us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619907.299229:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 233 00000100:00100000:0.0:1788619907.303949:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619907.304456:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571328000 00000100:00100000:3.0:1788619907.304468:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571328000 00000100:00100000:3.0:1788619907.304532:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 543 00000100:00100000:3.0:1788619907.304544:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b16d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+8:2354:x1875503571328000:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619907.304568:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619907.304591:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000064837316/0x1cd7fe5368733d30 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b324 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619907.304668:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df385180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589051136:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619907.304773:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5c76b40 going to sleep for 11 seconds 00002000:00010000:3.0:1788619907.308041:0:6698:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619906 -> 1788619907 00000100:00100000:3.0:1788619907.308079:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df385180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589051136:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619907.308119:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000064837316/0x1cd7fe5368733d30 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b324 expref: 9 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619907.308145:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000064837316/0x1cd7fe5368733d30 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b324 expref: 9 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619907.308158:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c1495a00, rc 0) 00000100:00100000:3.0:1788619907.308221:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b16d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+8:2354:x1875503571328000:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3679us (4289us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619907.308232:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 543 00000100:00100000:2.0:1788619907.312276:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619907.312559:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571328128 00000100:00100000:1.0:1788619907.312573:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571328128 00002000:00100000:1.0:1788619907.312708:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df966680 x1875503571328128/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:3/0 lens 504/0 e 0 to 0 dl 1788619918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619907.312721:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df966680 with x1875503571328128 00000100:00100000:1.0:1788619907.312868:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 386 00000100:00100000:1.0:1788619907.312879:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df966680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571328128:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619907.320697:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df966680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571328128:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 7803us (8420us total) trans 0 rc 360448/360448 00000100:00100000:0.0:1788619907.320715:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 386 00002000:00100000:0.0:1788619907.320746:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df966680 x1875503571328128/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:3/0 lens 504/440 e 0 to 0 dl 1788619918 ref 1 fl Complete:/600/0 rc 360448/360448 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619907.320758:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df966680 with x1875503571328128 00000100:00100000:2.0:1788619907.325770:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619907.325997:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571328256 00000100:00100000:3.0:1788619907.326030:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571328256 00002000:00100000:3.0:1788619907.326096:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df966d80 x1875503571328256/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:3/0 lens 488/0 e 0 to 0 dl 1788619918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619907.326109:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df966d80 with x1875503571328256 00000100:00100000:3.0:1788619907.326231:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 387 00000100:00100000:3.0:1788619907.326243:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df966d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571328256:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619907.326667:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df966d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571328256:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 425us (909us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619907.326676:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 387 00002000:00100000:3.0:1788619907.326684:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df966d80 x1875503571328256/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:3/0 lens 488/440 e 0 to 0 dl 1788619918 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619907.326692:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df966d80 with x1875503571328256 00000100:00100000:1.0:1788619907.335974:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619907.336146:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571328384 00000100:00100000:0.0:1788619907.336159:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571328384 00002000:00100000:0.0:1788619907.336434:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df8b1c00 x1875503571328384/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:3/0 lens 488/0 e 0 to 0 dl 1788619918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619907.336448:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df8b1c00 with x1875503571328384 00000100:00100000:2.0:1788619907.336702:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 388 00000100:00100000:2.0:1788619907.336718:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2352:x1875503571328384:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619907.337657:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2352:x1875503571328384:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 936us (1699us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619907.337670:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 388 00002000:00100000:2.0:1788619907.337681:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df8b1c00 x1875503571328384/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:3/0 lens 488/440 e 0 to 0 dl 1788619918 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619907.337693:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df8b1c00 with x1875503571328384 00000100:00100000:2.0:1788619907.342794:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619907.343034:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571328512 00000100:00100000:3.0:1788619907.343047:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571328512 00002000:00100000:3.0:1788619907.343116:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fe060000 x1875503571328512/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:3/0 lens 488/0 e 0 to 0 dl 1788619918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619907.343131:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fe060000 with x1875503571328512 00000100:00100000:3.0:1788619907.343241:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 389 00000100:00100000:3.0:1788619907.343252:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe060000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2352:x1875503571328512:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619907.343704:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe060000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2352:x1875503571328512:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 449us (921us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619907.343714:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 389 00002000:00100000:3.0:1788619907.343723:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fe060000 x1875503571328512/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:3/0 lens 488/440 e 0 to 0 dl 1788619918 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619907.343731:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fe060000 with x1875503571328512 00000100:00100000:1.0:1788619907.349310:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619907.349491:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571328640 00000100:00100000:2.0:1788619907.349506:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571328640 00002000:00100000:2.0:1788619907.349589:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df8b1180 x1875503571328640/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:3/0 lens 488/0 e 0 to 0 dl 1788619918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619907.349601:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df8b1180 with x1875503571328640 00000100:00100000:2.0:1788619907.349650:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 390 00000100:00100000:2.0:1788619907.349660:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2353:x1875503571328640:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619907.350265:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2353:x1875503571328640:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 603us (968us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619907.350275:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 390 00002000:00100000:2.0:1788619907.350283:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df8b1180 x1875503571328640/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:3/0 lens 488/440 e 0 to 0 dl 1788619918 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619907.350292:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df8b1180 with x1875503571328640 00000100:00100000:1.0:1788619907.356476:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619907.356967:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571328768 00000100:00100000:3.0:1788619907.356983:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571328768 00002000:00100000:3.0:1788619907.357050:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df8b2680 x1875503571328768/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:3/0 lens 488/0 e 0 to 0 dl 1788619918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619907.357063:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df8b2680 with x1875503571328768 00000100:00100000:3.0:1788619907.357110:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 391 00000100:00100000:3.0:1788619907.357122:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2352:x1875503571328768:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619907.358006:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2352:x1875503571328768:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 880us (1541us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619907.358017:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 391 00002000:00100000:3.0:1788619907.358028:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df8b2680 x1875503571328768/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:3/0 lens 488/440 e 0 to 0 dl 1788619918 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619907.358037:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df8b2680 with x1875503571328768 00000100:00100000:3.0:1788619907.363371:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619907.363480:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571328896 00000100:00100000:1.0:1788619907.363493:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571328896 00002000:00100000:1.0:1788619907.363696:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df385180 x1875503571328896/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:3/0 lens 488/0 e 0 to 0 dl 1788619918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619907.363709:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df385180 with x1875503571328896 00000100:00100000:1.0:1788619907.363815:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 392 00000100:00100000:1.0:1788619907.363826:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df385180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571328896:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619907.364494:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df385180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571328896:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 666us (1136us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619907.364505:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 392 00002000:00100000:1.0:1788619907.364526:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df385180 x1875503571328896/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:3/0 lens 488/440 e 0 to 0 dl 1788619918 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619907.364535:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df385180 with x1875503571328896 00000100:00100000:3.0:1788619907.369041:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619907.369743:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571329024 00000100:00100000:2.0:1788619907.369758:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571329024 00002000:00100000:2.0:1788619907.370038:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df386d80 x1875503571329024/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:3/0 lens 488/0 e 0 to 0 dl 1788619918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619907.370054:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df386d80 with x1875503571329024 00000100:00100000:2.0:1788619907.370090:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 393 00000100:00100000:2.0:1788619907.370102:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df386d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571329024:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619907.370675:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df386d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571329024:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 571us (1645us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619907.370686:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 393 00002000:00100000:2.0:1788619907.370696:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df386d80 x1875503571329024/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:3/0 lens 488/440 e 0 to 0 dl 1788619918 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619907.370705:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df386d80 with x1875503571329024 00000100:00100000:1.0:1788619907.376354:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619907.377378:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571329152 00000100:00100000:2.0:1788619907.377395:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571329152 00002000:00100000:2.0:1788619907.377470:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c50c2680 x1875503571329152/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:3/0 lens 488/0 e 0 to 0 dl 1788619918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619907.377484:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c50c2680 with x1875503571329152 00000100:00100000:2.0:1788619907.377528:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 394 00000100:00100000:2.0:1788619907.377560:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571329152:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619907.378138:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571329152:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 576us (1826us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619907.378149:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 394 00002000:00100000:2.0:1788619907.378157:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c50c2680 x1875503571329152/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:3/0 lens 488/440 e 0 to 0 dl 1788619918 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619907.378166:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c50c2680 with x1875503571329152 00000100:00100000:1.0:1788619907.383072:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619907.383182:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571329280 00000100:00100000:3.0:1788619907.383195:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571329280 00000100:00100000:3.0:1788619907.383286:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 701 00000100:00100000:3.0:1788619907.383298:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:14908:x1875503571329280:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619907.383374:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619907.383405:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368733d37 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b32b expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619907.383789:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368733d3e lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619907.383811:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368733d3e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619907.383922:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368733d3e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788619907.383972:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1633436 00000004:00200000:3.0:1788619907.383978:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788619907.384006:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368733d3e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619907.384039:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368733d37 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b32b expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619907.384061:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368733d3e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b32b expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619907.384070:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619907.384073:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788619907.384078:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d400, rc 0) 00000100:00100000:3.0:1788619907.384197:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+13:14908:x1875503571329280:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 901us (1139us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619907.384205:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 701 00000100:00100000:0.0:1788619907.390917:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619907.391193:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571329408 00000100:00100000:3.0:1788619907.391205:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571329408 00000100:00100000:3.0:1788619907.391282:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 544 00000100:00100000:3.0:1788619907.391293:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed62d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+8:2354:x1875503571329408:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619907.391314:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619907.391375:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe5368733d45 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b332 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619907.391474:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df386680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589051264:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619907.392772:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5c76b40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619907.396715:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df386680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589051264:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619907.396770:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe5368733d45 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b332 expref: 9 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.396812:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe5368733d45 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b332 expref: 9 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.396829:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c600, rc 0) 00000100:00100000:0.0:1788619907.397088:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed62d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+8:2354:x1875503571329408:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5794us (6183us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619907.397099:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 544 00000100:00100000:1.0:1788619907.407017:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619907.407375:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571329536 00000100:00100000:2.0:1788619907.407389:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571329536 00000100:00100000:2.0:1788619907.407466:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 545 00000100:00100000:2.0:1788619907.407478:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+8:2354:x1875503571329536:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619907.407500:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619907.407522:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000860f3b75/0x1cd7fe5368733d4c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b339 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619907.407602:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fe063b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589051392:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619907.407715:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccb40 going to sleep for 11 seconds 00000100:00100000:2.0:1788619907.413502:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe063b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589051392:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619907.413736:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000860f3b75/0x1cd7fe5368733d4c lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b339 expref: 9 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.414050:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000860f3b75/0x1cd7fe5368733d4c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b339 expref: 9 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.414099:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7200, rc 0) 00000100:00100000:2.0:1788619907.414309:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+8:2354:x1875503571329536:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6832us (7307us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619907.414321:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 545 00000100:00100000:0.0:1788619907.421168:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619907.421422:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571329664 00000100:00100000:2.0:1788619907.421437:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571329664 00000100:00100000:2.0:1788619907.421572:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 702 00000100:00100000:2.0:1788619907.421584:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+13:14908:x1875503571329664:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619907.421612:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619907.421623:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1788619907.421685:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297800/0x1cd7fe5368733d29 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b31d expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.421711:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619907.421718:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788619907.421736:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7200/0x1cd7fe5368733d53 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b340 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.421771:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7200/0x1cd7fe5368733d53 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b340 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.421797:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7200/0x1cd7fe5368733d53 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b340 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.421811:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619907.421832:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619907.421844:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7200, rc 0) 00000100:00100000:2.0:1788619907.422226:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+13:14908:x1875503571329664:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 642us (1072us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619907.422235:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 702 00000100:00100000:1.0:1788619907.434878:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619907.437394:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571329792 00000100:00100000:0.0:1788619907.437417:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571329792 00000100:00100000:0.0:1788619907.437676:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 395 00000100:00100000:0.0:1788619907.437693:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+13:2353:x1875503571329792:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:0.0:1788619907.438808:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 358559 -> 358559 00000004:00010000:0.0:1788619907.438818:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619906 -> 1788619906 00000004:00010000:0.0:1788619907.438821:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619907 -> 1788619907 00000004:00010000:0.0:1788619907.438824:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619906 -> 1788619906 00000004:00010000:0.0:1788619907.438826:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1158 -> 1158 00000100:00100000:0.0:1788619907.439183:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+14:2353:x1875503571329792:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 1490us (4315us total) trans 4294967574 rc 0/0 00000100:00100000:0.0:1788619907.445006:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 395 00000100:00100000:0.0:1788619907.450182:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619907.450363:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571329920 00000100:00100000:1.0:1788619907.450378:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571329920 00000100:00100000:1.0:1788619907.450457:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 703 00000100:00100000:1.0:1788619907.450471:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7582300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2354:x1875503571329920:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619907.450505:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619907.450604:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07200/0x1cd7fe5368733d5a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b347 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619907.450731:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7200/0x1cd7fe5368733d53 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b340 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619907.450781:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c0380 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589051520:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619907.451215:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd9633c0 going to sleep for 11 seconds 00000004:00010000:0.0:1788619907.455259:0:11397:0:(mdt_lvb.c:183:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime: 1788619906 -> 1788619907 00000004:00010000:0.0:1788619907.455271:0:11397:0:(mdt_lvb.c:195:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime: 1788619906 -> 1788619907 00000100:00100000:0.0:1788619907.455303:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c0380 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589051520:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619907.455368:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 358559 00010000:00010000:0.0:1788619907.455394:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07200/0x1cd7fe5368733d5a lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b347 expref: 20 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.455423:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619907.455427:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619907.455433:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07200/0x1cd7fe5368733d5a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b347 expref: 20 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.455452:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa07200, rc 0) 00000100:00100000:0.0:1788619907.455536:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7582300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2354:x1875503571329920:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5067us (5368us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619907.455546:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 703 00000100:00100000:0.0:1788619907.459517:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619907.460688:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571330048 00000100:00100000:3.0:1788619907.460703:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571330048 00000100:00100000:3.0:1788619907.460783:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 704 00000100:00100000:3.0:1788619907.460794:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6000000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2353:x1875503571330048:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619907.460821:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619907.461160:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368733d61 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b34e expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619907.461228:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7200/0x1cd7fe5368733d53 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b340 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619907.461286:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df384000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589051648:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619907.461441:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5c76e40 going to sleep for 11 seconds 00000100:00100000:3.0:1788619907.466269:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df384000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589051648:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619907.466307:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 358559 00010000:00010000:3.0:1788619907.466380:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368733d61 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b34e expref: 20 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619907.466411:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619907.466415:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619907.466422:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368733d61 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b34e expref: 20 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619907.466441:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683cc00, rc 0) 00000100:00100000:3.0:1788619907.466544:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6000000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2353:x1875503571330048:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5750us (7046us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619907.466554:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 704 00000100:00100000:1.0:1788619907.482491:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619907.482672:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571330176 00000100:00100000:2.0:1788619907.482685:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571330176 00000100:00100000:2.0:1788619907.482761:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 705 00000100:00100000:2.0:1788619907.482772:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2352:x1875503571330176:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619907.482798:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619907.482824:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6400/0x1cd7fe5368733d68 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b355 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.482885:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7200/0x1cd7fe5368733d53 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b340 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619907.482939:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fe063b80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589051776:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619907.483062:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccb40 going to sleep for 11 seconds 00000100:00100000:2.0:1788619907.486743:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe063b80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589051776:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619907.486781:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 358559 00010000:00010000:2.0:1788619907.486806:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6400/0x1cd7fe5368733d68 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b355 expref: 20 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.486834:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619907.486836:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619907.486842:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6400/0x1cd7fe5368733d68 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b355 expref: 20 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.487076:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6400, rc 0) 00000100:00100000:2.0:1788619907.487161:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2352:x1875503571330176:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4388us (4684us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619907.487171:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 705 00000100:00100000:1.0:1788619907.492255:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619907.498501:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571330304 00000100:00100000:2.0:1788619907.498536:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571330304 00002000:00100000:2.0:1788619907.499029:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1261568->1265663) req@ffff8e77c50c3800 x1875503571330304/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:3/0 lens 488/0 e 0 to 0 dl 1788619918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619907.499044:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c50c3800 with x1875503571330304 ext(1261568->1265663) 00002000:00010000:2.0:1788619907.499053:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c50c3800 00000100:00100000:2.0:1788619907.499100:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 278 00000100:00100000:2.0:1788619907.499113:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2353:x1875503571330304:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619907.500456:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2353:x1875503571330304:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 1329us (8198us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619907.500472:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 278 00002000:00100000:2.0:1788619907.500486:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1261568->1265663) req@ffff8e77c50c3800 x1875503571330304/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:3/0 lens 488/4536 e 0 to 0 dl 1788619918 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619907.500499:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c50c3800 with x1875503571330304 ext(1261568->1265663) 00002000:00010000:2.0:1788619907.500507:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c50c3800 00000100:00100000:1.0:1788619907.515476:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619907.515638:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571330432 00000100:00100000:2.0:1788619907.515651:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571330432 00002000:00100000:2.0:1788619907.515715:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1265664->1269759) req@ffff8e77c50c2a00 x1875503571330432/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:3/0 lens 488/0 e 0 to 0 dl 1788619918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619907.515749:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c50c2a00 with x1875503571330432 ext(1265664->1269759) 00002000:00010000:2.0:1788619907.515756:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c50c2a00 00000100:00100000:2.0:1788619907.515796:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 279 00000100:00100000:2.0:1788619907.515807:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2352:x1875503571330432:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619907.516469:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2352:x1875503571330432:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 660us (1003us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619907.516480:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 279 00002000:00100000:2.0:1788619907.516488:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1265664->1269759) req@ffff8e77c50c2a00 x1875503571330432/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:3/0 lens 488/4536 e 0 to 0 dl 1788619918 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619907.516497:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c50c2a00 with x1875503571330432 ext(1265664->1269759) 00002000:00010000:2.0:1788619907.516502:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c50c2a00 00000100:00100000:0.0:1788619907.525367:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619907.525476:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571330560 00000100:00100000:2.0:1788619907.525490:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571330560 00002000:00100000:2.0:1788619907.525568:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1269760->1273855) req@ffff8e77c4ba0a80 x1875503571330560/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:3/0 lens 488/0 e 0 to 0 dl 1788619918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619907.525580:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4ba0a80 with x1875503571330560 ext(1269760->1273855) 00002000:00010000:2.0:1788619907.525586:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4ba0a80 00000100:00100000:1.0:1788619907.525860:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 280 00000100:00100000:1.0:1788619907.525876:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2353:x1875503571330560:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619907.526259:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2353:x1875503571330560:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 386us (932us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619907.526268:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 280 00002000:00100000:1.0:1788619907.526277:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1269760->1273855) req@ffff8e77c4ba0a80 x1875503571330560/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:3/0 lens 488/4536 e 0 to 0 dl 1788619918 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619907.526310:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4ba0a80 with x1875503571330560 ext(1269760->1273855) 00002000:00010000:1.0:1788619907.526315:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4ba0a80 00000100:00100000:3.0:1788619907.536485:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619907.537770:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571330688 00000100:00100000:1.0:1788619907.537792:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571330688 00002000:00100000:1.0:1788619907.538012:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1273856->1277951) req@ffff8e78df384000 x1875503571330688/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:3/0 lens 488/0 e 0 to 0 dl 1788619918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619907.538027:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df384000 with x1875503571330688 ext(1273856->1277951) 00002000:00010000:1.0:1788619907.538035:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df384000 00000100:00100000:1.0:1788619907.538087:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 281 00000100:00100000:1.0:1788619907.538099:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df384000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2352:x1875503571330688:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619907.539475:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df384000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2352:x1875503571330688:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 1365us (2992us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619907.539489:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 281 00002000:00100000:1.0:1788619907.539503:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1273856->1277951) req@ffff8e78df384000 x1875503571330688/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:3/0 lens 488/4536 e 0 to 0 dl 1788619918 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619907.539527:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df384000 with x1875503571330688 ext(1273856->1277951) 00002000:00010000:1.0:1788619907.539534:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df384000 00000100:00100000:0.0:1788619907.564245:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619907.564394:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571330816 00000100:00100000:2.0:1788619907.564408:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571330816 00002000:00100000:2.0:1788619907.564479:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1277952->1282047) req@ffff8e78dfa38a80 x1875503571330816/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:3/0 lens 488/0 e 0 to 0 dl 1788619918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619907.564493:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa38a80 with x1875503571330816 ext(1277952->1282047) 00002000:00010000:2.0:1788619907.564501:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa38a80 00000100:00100000:2.0:1788619907.564566:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 282 00000100:00100000:2.0:1788619907.564576:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa38a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2352:x1875503571330816:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619907.564915:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa38a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2352:x1875503571330816:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 338us (683us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619907.564923:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 282 00002000:00100000:2.0:1788619907.564930:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1277952->1282047) req@ffff8e78dfa38a80 x1875503571330816/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:3/0 lens 488/4536 e 0 to 0 dl 1788619918 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619907.564939:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa38a80 with x1875503571330816 ext(1277952->1282047) 00002000:00010000:2.0:1788619907.564943:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa38a80 00000100:00100000:0.0:1788619907.587936:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619907.588286:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571330944 00000100:00100000:2.0:1788619907.588301:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571330944 00002000:00100000:2.0:1788619907.588395:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1282048->1286143) req@ffff8e78dfa3bb80 x1875503571330944/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:3/0 lens 488/0 e 0 to 0 dl 1788619918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619907.588409:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa3bb80 with x1875503571330944 ext(1282048->1286143) 00002000:00010000:2.0:1788619907.588417:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa3bb80 00000100:00100000:2.0:1788619907.588450:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 283 00000100:00100000:2.0:1788619907.588462:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa3bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2355:x1875503571330944:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619907.589110:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa3bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2355:x1875503571330944:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 646us (1185us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619907.589120:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 283 00002000:00100000:2.0:1788619907.589129:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1282048->1286143) req@ffff8e78dfa3bb80 x1875503571330944/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:3/0 lens 488/4536 e 0 to 0 dl 1788619918 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619907.589139:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa3bb80 with x1875503571330944 ext(1282048->1286143) 00002000:00010000:2.0:1788619907.589144:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa3bb80 00000100:00100000:3.0:1788619907.632709:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619907.633068:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571331072 00000100:00100000:1.0:1788619907.633082:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571331072 00002000:00100000:1.0:1788619907.633152:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1286144->1290239) req@ffff8e78df384380 x1875503571331072/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:3/0 lens 488/0 e 0 to 0 dl 1788619918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619907.633165:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df384380 with x1875503571331072 ext(1286144->1290239) 00002000:00010000:1.0:1788619907.633173:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df384380 00000100:00100000:1.0:1788619907.633247:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 284 00000100:00100000:1.0:1788619907.633258:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df384380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571331072:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619907.634051:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df384380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571331072:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 789us (1353us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619907.634062:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 284 00002000:00100000:1.0:1788619907.634070:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1286144->1290239) req@ffff8e78df384380 x1875503571331072/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:3/0 lens 488/4536 e 0 to 0 dl 1788619918 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619907.634080:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df384380 with x1875503571331072 ext(1286144->1290239) 00002000:00010000:1.0:1788619907.634085:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df384380 00000100:00100000:1.0:1788619907.640172:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619907.640409:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571331200 00000100:00100000:0.0:1788619907.640426:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571331200 00002000:00100000:0.0:1788619907.640499:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1290240->1294335) req@ffff8e77c50c3b80 x1875503571331200/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:3/0 lens 488/0 e 0 to 0 dl 1788619918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619907.640513:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c50c3b80 with x1875503571331200 ext(1290240->1294335) 00002000:00010000:0.0:1788619907.640521:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c50c3b80 00000100:00100000:0.0:1788619907.640544:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 285 00000100:00100000:0.0:1788619907.640555:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571331200:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619907.641046:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571331200:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 490us (888us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619907.641056:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 285 00002000:00100000:0.0:1788619907.641064:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1290240->1294335) req@ffff8e77c50c3b80 x1875503571331200/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:3/0 lens 488/4536 e 0 to 0 dl 1788619918 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619907.641074:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c50c3b80 with x1875503571331200 ext(1290240->1294335) 00002000:00010000:0.0:1788619907.641078:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c50c3b80 00000100:00100000:0.0:1788619907.653896:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619907.654069:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571331328 00000100:00100000:2.0:1788619907.654083:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571331328 00002000:00100000:2.0:1788619907.654150:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1294336->1298431) req@ffff8e78dfa3a680 x1875503571331328/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:3/0 lens 488/0 e 0 to 0 dl 1788619918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619907.654163:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa3a680 with x1875503571331328 ext(1294336->1298431) 00002000:00010000:2.0:1788619907.654171:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa3a680 00000100:00100000:2.0:1788619907.654210:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 286 00000100:00100000:2.0:1788619907.654230:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa3a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2353:x1875503571331328:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619907.654606:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa3a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2353:x1875503571331328:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 377us (724us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619907.654614:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 286 00002000:00100000:2.0:1788619907.654622:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1294336->1298431) req@ffff8e78dfa3a680 x1875503571331328/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:3/0 lens 488/4536 e 0 to 0 dl 1788619918 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619907.654643:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa3a680 with x1875503571331328 ext(1294336->1298431) 00002000:00010000:2.0:1788619907.654647:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa3a680 00000100:00100000:3.0:1788619907.661649:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619907.663795:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571331456 00000100:00100000:1.0:1788619907.663820:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571331456 00002000:00100000:1.0:1788619907.663910:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1298432->1302527) req@ffff8e78df386680 x1875503571331456/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:3/0 lens 488/0 e 0 to 0 dl 1788619918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619907.663924:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df386680 with x1875503571331456 ext(1298432->1302527) 00002000:00010000:1.0:1788619907.663931:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df386680 00000100:00100000:1.0:1788619907.663988:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 287 00000100:00100000:1.0:1788619907.664000:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df386680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2352:x1875503571331456:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619907.664395:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df386680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2352:x1875503571331456:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 396us (2761us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619907.664404:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 287 00002000:00100000:1.0:1788619907.664412:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1298432->1302527) req@ffff8e78df386680 x1875503571331456/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:3/0 lens 488/4536 e 0 to 0 dl 1788619918 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619907.664421:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df386680 with x1875503571331456 ext(1298432->1302527) 00002000:00010000:1.0:1788619907.664442:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df386680 00000100:00100000:1.0:1788619907.674479:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619907.679813:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571331584 00000100:00100000:0.0:1788619907.679839:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571331584 00002000:00100000:0.0:1788619907.680119:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1302528->1306623) req@ffff8e77c50c0a80 x1875503571331584/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:3/0 lens 488/0 e 0 to 0 dl 1788619918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619907.680133:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c50c0a80 with x1875503571331584 ext(1302528->1306623) 00002000:00010000:0.0:1788619907.680141:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c50c0a80 00000100:00100000:0.0:1788619907.680206:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 288 00000100:00100000:0.0:1788619907.680217:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2353:x1875503571331584:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619907.680618:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2353:x1875503571331584:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 400us (6151us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619907.680719:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 288 00002000:00100000:0.0:1788619907.680729:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1302528->1306623) req@ffff8e77c50c0a80 x1875503571331584/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:3/0 lens 488/4536 e 0 to 0 dl 1788619918 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619907.680739:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c50c0a80 with x1875503571331584 ext(1302528->1306623) 00002000:00010000:0.0:1788619907.680743:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c50c0a80 00000100:00100000:0.0:1788619907.688442:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619907.689149:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571331712 00000100:00100000:2.0:1788619907.689165:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571331712 00002000:00100000:2.0:1788619907.689298:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1306624->1310719) req@ffff8e78dfa3b800 x1875503571331712/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:3/0 lens 488/0 e 0 to 0 dl 1788619918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619907.689312:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa3b800 with x1875503571331712 ext(1306624->1310719) 00002000:00010000:2.0:1788619907.689320:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa3b800 00000100:00100000:2.0:1788619907.689443:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 289 00000100:00100000:2.0:1788619907.689455:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa3b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2352:x1875503571331712:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619907.689831:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa3b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2352:x1875503571331712:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 377us (1403us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619907.690279:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 289 00002000:00100000:2.0:1788619907.690291:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1306624->1310719) req@ffff8e78dfa3b800 x1875503571331712/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:3/0 lens 488/4536 e 0 to 0 dl 1788619918 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619907.690301:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa3b800 with x1875503571331712 ext(1306624->1310719) 00002000:00010000:2.0:1788619907.690306:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa3b800 00000100:00100000:3.0:1788619907.695756:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619907.697477:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571331840 00000100:00100000:1.0:1788619907.697496:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571331840 00002000:00100000:1.0:1788619907.697573:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1310720->1314815) req@ffff8e77c7788380 x1875503571331840/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:3/0 lens 488/0 e 0 to 0 dl 1788619918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619907.697583:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c7788380 with x1875503571331840 ext(1310720->1314815) 00002000:00010000:1.0:1788619907.697589:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c7788380 00000100:00100000:1.0:1788619907.697634:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 290 00000100:00100000:1.0:1788619907.697645:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c7788380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2352:x1875503571331840:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619907.698007:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c7788380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2352:x1875503571331840:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 363us (2264us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619907.698309:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 290 00002000:00100000:1.0:1788619907.698319:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1310720->1314815) req@ffff8e77c7788380 x1875503571331840/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:3/0 lens 488/4536 e 0 to 0 dl 1788619918 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619907.698772:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c7788380 with x1875503571331840 ext(1310720->1314815) 00002000:00010000:1.0:1788619907.698780:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c7788380 00000100:00100000:1.0:1788619907.708739:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619907.708877:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571331968 00000100:00100000:0.0:1788619907.708891:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571331968 00000100:00100000:0.0:1788619907.708976:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 706 00000100:00100000:0.0:1788619907.708988:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2352:x1875503571331968:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619907.709017:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619907.709049:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296a00/0x1cd7fe5368733d6f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b35c expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.709101:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7200/0x1cd7fe5368733d53 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b340 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619907.709156:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78dfa38000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589051904:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619907.709286:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaab40 going to sleep for 11 seconds 00000100:00100000:1.0:1788619907.716212:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78dfa38000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589051904:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619907.716252:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 358559 00010000:00010000:1.0:1788619907.716280:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296a00/0x1cd7fe5368733d6f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b35c expref: 20 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619907.716308:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619907.716311:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619907.716369:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296a00/0x1cd7fe5368733d6f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b35c expref: 20 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619907.716392:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296a00, rc 0) 00000100:00100000:1.0:1788619907.716544:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2352:x1875503571331968:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7558us (7821us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619907.716555:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 706 00000100:00100000:3.0:1788619907.723478:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619907.724067:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571332096 00000100:00100000:2.0:1788619907.724082:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571332096 00000100:00100000:2.0:1788619907.724164:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 707 00000100:00100000:2.0:1788619907.724177:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c778bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2352:x1875503571332096:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619907.724205:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619907.724233:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6400/0x1cd7fe5368733d76 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b363 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.724285:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7200/0x1cd7fe5368733d53 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b340 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619907.724903:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fe063b80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589052032:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619907.725155:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc300 going to sleep for 11 seconds 00000100:00100000:2.0:1788619907.740672:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe063b80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589052032:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619907.740711:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 358559 00010000:00010000:2.0:1788619907.740739:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6400/0x1cd7fe5368733d76 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b363 expref: 20 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.740767:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619907.740771:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619907.740797:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6400/0x1cd7fe5368733d76 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b363 expref: 20 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.740816:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6400, rc 0) 00000100:00100000:2.0:1788619907.740914:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c778bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2352:x1875503571332096:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 16738us (17450us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619907.740923:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 707 00000100:00100000:0.0:1788619907.752296:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619907.752493:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571332224 00000100:00100000:2.0:1788619907.752506:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571332224 00000100:00100000:2.0:1788619907.752590:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 708 00000100:00100000:2.0:1788619907.752604:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa3ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2353:x1875503571332224:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619907.752646:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619907.752677:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6400/0x1cd7fe5368733d7d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b36a expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.752731:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7200/0x1cd7fe5368733d53 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b340 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619907.752796:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fe063b80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589052160:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619907.752906:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc300 going to sleep for 11 seconds 00000100:00100000:2.0:1788619907.757232:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe063b80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589052160:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619907.758781:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 358559 00010000:00010000:2.0:1788619907.758825:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6400/0x1cd7fe5368733d7d lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b36a expref: 20 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.758857:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619907.758861:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619907.758895:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6400/0x1cd7fe5368733d7d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b36a expref: 20 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.758918:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6400, rc 0) 00000100:00100000:2.0:1788619907.759096:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa3ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2353:x1875503571332224:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6493us (6813us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619907.759106:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 708 00000100:00100000:0.0:1788619907.772654:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619907.776181:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571332352 00000100:00100000:1.0:1788619907.776202:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571332352 00000100:00100000:1.0:1788619907.776298:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 709 00000100:00100000:1.0:1788619907.776307:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa38700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2353:x1875503571332352:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619907.776385:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619907.776413:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe5368733d84 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b371 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619907.776464:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7200/0x1cd7fe5368733d53 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b340 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619907.776527:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4ba6a00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589052288:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619907.776649:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963f00 going to sleep for 11 seconds 00000100:00100000:1.0:1788619907.781007:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4ba6a00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589052288:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619907.781046:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 358559 00010000:00010000:1.0:1788619907.781076:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe5368733d84 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b371 expref: 20 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619907.781106:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619907.781110:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619907.781137:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe5368733d84 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b371 expref: 20 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619907.781158:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa06400, rc 0) 00000100:00100000:1.0:1788619907.781258:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa38700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2353:x1875503571332352:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4949us (8629us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619907.781269:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 709 00000100:00100000:1.0:1788619907.786047:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619907.787422:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571332480 00000100:00100000:2.0:1788619907.787450:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571332480 00002000:00100000:0.0:1788619907.787653:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1314816->1318911) req@ffff8e77c4ba6a00 x1875503571332480/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:3/0 lens 488/0 e 0 to 0 dl 1788619918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619907.787687:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4ba6a00 with x1875503571332480 ext(1314816->1318911) 00002000:00010000:0.0:1788619907.787695:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4ba6a00 00000100:00100000:0.0:1788619907.787737:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 291 00000100:00100000:0.0:1788619907.787749:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2352:x1875503571332480:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619907.788101:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2352:x1875503571332480:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 353us (2068us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619907.788121:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 291 00002000:00100000:0.0:1788619907.788129:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1314816->1318911) req@ffff8e77c4ba6a00 x1875503571332480/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:3/0 lens 488/4536 e 0 to 0 dl 1788619918 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619907.788138:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4ba6a00 with x1875503571332480 ext(1314816->1318911) 00002000:00010000:0.0:1788619907.788142:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4ba6a00 00000100:00100000:0.0:1788619907.793583:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619907.797392:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571332608 00000100:00100000:1.0:1788619907.797417:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571332608 00002000:00100000:1.0:1788619907.797512:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1318912->1323007) req@ffff8e78dfa39500 x1875503571332608/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:3/0 lens 488/0 e 0 to 0 dl 1788619918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619907.797525:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa39500 with x1875503571332608 ext(1318912->1323007) 00002000:00010000:1.0:1788619907.797533:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa39500 00000100:00100000:1.0:1788619907.797574:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 292 00000100:00100000:1.0:1788619907.797587:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa39500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2352:x1875503571332608:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619907.799591:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa39500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2352:x1875503571332608:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 1991us (6366us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619907.799608:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 292 00002000:00100000:1.0:1788619907.799623:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1318912->1323007) req@ffff8e78dfa39500 x1875503571332608/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:3/0 lens 488/4536 e 0 to 0 dl 1788619918 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619907.799646:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa39500 with x1875503571332608 ext(1318912->1323007) 00002000:00010000:1.0:1788619907.799655:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa39500 00000100:00100000:1.0:1788619907.809229:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619907.809450:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571332736 00000100:00100000:2.0:1788619907.809464:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571332736 00002000:00100000:2.0:1788619907.809527:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1323008->1327103) req@ffff8e77c4ba6300 x1875503571332736/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:3/0 lens 488/0 e 0 to 0 dl 1788619918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619907.809541:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4ba6300 with x1875503571332736 ext(1323008->1327103) 00002000:00010000:2.0:1788619907.809548:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4ba6300 00000100:00100000:2.0:1788619907.809589:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 293 00000100:00100000:2.0:1788619907.809600:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2352:x1875503571332736:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619907.810400:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2352:x1875503571332736:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 795us (1187us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619907.810413:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 293 00002000:00100000:2.0:1788619907.810421:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1323008->1327103) req@ffff8e77c4ba6300 x1875503571332736/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:3/0 lens 488/4536 e 0 to 0 dl 1788619918 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619907.810431:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4ba6300 with x1875503571332736 ext(1323008->1327103) 00002000:00010000:2.0:1788619907.811239:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4ba6300 00000100:00100000:3.0:1788619907.818004:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619907.818417:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571332864 00000100:00100000:1.0:1788619907.818463:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571332864 00002000:00100000:1.0:1788619907.818533:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1327104->1331199) req@ffff8e77c7789180 x1875503571332864/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:3/0 lens 488/0 e 0 to 0 dl 1788619918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619907.818547:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c7789180 with x1875503571332864 ext(1327104->1331199) 00002000:00010000:1.0:1788619907.818554:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c7789180 00000100:00100000:1.0:1788619907.818598:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 294 00000100:00100000:1.0:1788619907.818617:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c7789180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2352:x1875503571332864:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619907.818975:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c7789180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2352:x1875503571332864:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 366us (986us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619907.818984:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 294 00002000:00100000:1.0:1788619907.818992:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1327104->1331199) req@ffff8e77c7789180 x1875503571332864/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:3/0 lens 488/4536 e 0 to 0 dl 1788619918 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619907.819000:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c7789180 with x1875503571332864 ext(1327104->1331199) 00002000:00010000:1.0:1788619907.819004:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c7789180 00000100:00100000:0.0:1788619907.824223:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619907.824983:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571332992 00000100:00100000:2.0:1788619907.825001:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571332992 00002000:00100000:2.0:1788619907.825120:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1331200->1335295) req@ffff8e78dfa38380 x1875503571332992/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:3/0 lens 488/0 e 0 to 0 dl 1788619918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619907.825134:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa38380 with x1875503571332992 ext(1331200->1335295) 00002000:00010000:2.0:1788619907.825141:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa38380 00000100:00100000:2.0:1788619907.825186:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 295 00000100:00100000:2.0:1788619907.825196:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa38380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2352:x1875503571332992:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619907.825584:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa38380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2352:x1875503571332992:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 387us (1373us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619907.825592:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 295 00002000:00100000:2.0:1788619907.825599:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1331200->1335295) req@ffff8e78dfa38380 x1875503571332992/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:3/0 lens 488/4536 e 0 to 0 dl 1788619918 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619907.825608:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa38380 with x1875503571332992 ext(1331200->1335295) 00002000:00010000:2.0:1788619907.826830:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa38380 00000100:00100000:2.0:1788619907.833458:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619907.833863:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571333120 00000100:00100000:0.0:1788619907.833877:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571333120 00002000:00100000:0.0:1788619907.833959:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1335296->1339391) req@ffff8e78fe063b80 x1875503571333120/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:3/0 lens 488/0 e 0 to 0 dl 1788619918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619907.833973:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fe063b80 with x1875503571333120 ext(1335296->1339391) 00002000:00010000:0.0:1788619907.833981:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fe063b80 00000100:00100000:0.0:1788619907.834023:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 296 00000100:00100000:0.0:1788619907.834034:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe063b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2352:x1875503571333120:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619907.834466:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe063b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2352:x1875503571333120:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 432us (1022us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619907.834496:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 296 00002000:00100000:0.0:1788619907.834503:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1335296->1339391) req@ffff8e78fe063b80 x1875503571333120/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:3/0 lens 488/4536 e 0 to 0 dl 1788619918 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619907.834512:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fe063b80 with x1875503571333120 ext(1335296->1339391) 00002000:00010000:0.0:1788619907.834516:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fe063b80 00000100:00100000:3.0:1788619907.837891:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619907.838009:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571333248 00000100:00100000:1.0:1788619907.838022:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571333248 00002000:00100000:1.0:1788619907.838089:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1339392->1343487) req@ffff8e77c7788e00 x1875503571333248/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:3/0 lens 488/0 e 0 to 0 dl 1788619918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619907.838112:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c7788e00 with x1875503571333248 ext(1339392->1343487) 00002000:00010000:1.0:1788619907.838118:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c7788e00 00000100:00100000:1.0:1788619907.838159:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 297 00000100:00100000:1.0:1788619907.838169:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c7788e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2352:x1875503571333248:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619907.839853:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c7788e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2352:x1875503571333248:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 1659us (1950us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619907.839898:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 297 00002000:00100000:1.0:1788619907.839912:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1339392->1343487) req@ffff8e77c7788e00 x1875503571333248/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:3/0 lens 488/4536 e 0 to 0 dl 1788619918 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619907.839924:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c7788e00 with x1875503571333248 ext(1339392->1343487) 00002000:00010000:1.0:1788619907.839931:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c7788e00 00000100:00100000:0.0:1788619907.848403:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619907.848635:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571333376 00000100:00100000:2.0:1788619907.848649:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571333376 00000100:00100000:2.0:1788619907.848737:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 710 00000100:00100000:2.0:1788619907.848752:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa38000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2353:x1875503571333376:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619907.848783:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619907.848816:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7800/0x1cd7fe5368733d8b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b378 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.848880:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7200/0x1cd7fe5368733d53 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b340 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619907.848973:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fe062d80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589052416:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619907.849100:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619907.860364:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe062d80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589052416:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619907.860406:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 358559 00010000:00010000:3.0:1788619907.860433:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7800/0x1cd7fe5368733d8b lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b378 expref: 20 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619907.860460:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619907.860695:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619907.860703:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7800/0x1cd7fe5368733d8b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b378 expref: 20 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619907.861056:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7800, rc 0) 00000100:00100000:3.0:1788619907.861381:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa38000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2353:x1875503571333376:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 12578us (12941us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619907.861394:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 710 00000100:00100000:1.0:1788619907.871386:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619907.871504:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571333504 00000100:00100000:2.0:1788619907.871518:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571333504 00000100:00100000:2.0:1788619907.871590:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 711 00000100:00100000:2.0:1788619907.871603:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2352:x1875503571333504:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619907.871631:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619907.871657:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7600/0x1cd7fe5368733d92 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b37f expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.871707:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7200/0x1cd7fe5368733d53 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b340 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619907.871787:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fe061c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589052544:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619907.872263:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc840 going to sleep for 11 seconds 00000100:00100000:2.0:1788619907.876357:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe061c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589052544:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619907.876402:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 358559 00010000:00010000:2.0:1788619907.876428:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7600/0x1cd7fe5368733d92 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b37f expref: 20 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.876454:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619907.876458:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619907.876477:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7600/0x1cd7fe5368733d92 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b37f expref: 20 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.876522:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7600, rc 0) 00000100:00100000:2.0:1788619907.876611:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2352:x1875503571333504:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5008us (5241us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619907.876622:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 711 00010000:00010000:3.0:1788619907.878480:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788619907.883692:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619907.883964:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571333632 00000100:00100000:2.0:1788619907.883978:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571333632 00002000:00100000:2.0:1788619907.884046:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1089536->1093631) req@ffff8e77c4ba5880 x1875503571333632/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:3/0 lens 488/0 e 0 to 0 dl 1788619918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619907.884058:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4ba5880 with x1875503571333632 ext(1089536->1093631) 00002000:00010000:2.0:1788619907.884066:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4ba5880 00000100:00100000:2.0:1788619907.884109:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 298 00000100:00100000:2.0:1788619907.884121:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2352:x1875503571333632:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619907.884537:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2352:x1875503571333632:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 416us (859us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619907.884546:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 298 00002000:00100000:2.0:1788619907.884554:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1089536->1093631) req@ffff8e77c4ba5880 x1875503571333632/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:3/0 lens 488/4536 e 0 to 0 dl 1788619918 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619907.884563:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4ba5880 with x1875503571333632 ext(1089536->1093631) 00002000:00010000:2.0:1788619907.884567:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4ba5880 00000100:00100000:2.0:1788619907.890633:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619907.890964:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571333760 00000100:00100000:1.0:1788619907.890979:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571333760 00002000:00100000:1.0:1788619907.891053:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1093632->1097727) req@ffff8e78fe061c00 x1875503571333760/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:3/0 lens 488/0 e 0 to 0 dl 1788619918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619907.891068:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fe061c00 with x1875503571333760 ext(1093632->1097727) 00002000:00010000:1.0:1788619907.891075:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fe061c00 00000100:00100000:1.0:1788619907.891114:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 299 00000100:00100000:1.0:1788619907.891125:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe061c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2352:x1875503571333760:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619907.892032:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe061c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2352:x1875503571333760:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 900us (1408us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619907.892045:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 299 00002000:00100000:1.0:1788619907.892056:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1093632->1097727) req@ffff8e78fe061c00 x1875503571333760/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:3/0 lens 488/4536 e 0 to 0 dl 1788619918 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619907.892067:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fe061c00 with x1875503571333760 ext(1093632->1097727) 00002000:00010000:1.0:1788619907.892073:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fe061c00 00000100:00100000:3.0:1788619907.898159:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619907.898382:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571333888 00000100:00100000:0.0:1788619907.898397:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571333888 00002000:00100000:0.0:1788619907.898483:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1097728->1101823) req@ffff8e77c7788700 x1875503571333888/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:3/0 lens 488/0 e 0 to 0 dl 1788619918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619907.898497:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c7788700 with x1875503571333888 ext(1097728->1101823) 00002000:00010000:0.0:1788619907.898504:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c7788700 00000100:00100000:0.0:1788619907.898543:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 300 00000100:00100000:0.0:1788619907.898554:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c7788700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571333888:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619907.899176:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c7788700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571333888:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 621us (1028us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619907.899187:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 300 00002000:00100000:0.0:1788619907.899198:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1097728->1101823) req@ffff8e77c7788700 x1875503571333888/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:3/0 lens 488/4536 e 0 to 0 dl 1788619918 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619907.899979:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c7788700 with x1875503571333888 ext(1097728->1101823) 00002000:00010000:0.0:1788619907.899989:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c7788700 00000100:00100000:2.0:1788619907.904809:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619907.904999:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571334016 00000100:00100000:0.0:1788619907.905013:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571334016 00002000:00100000:0.0:1788619907.905101:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1101824->1105919) req@ffff8e78fe062d80 x1875503571334016/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:3/0 lens 488/0 e 0 to 0 dl 1788619918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619907.905115:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fe062d80 with x1875503571334016 ext(1101824->1105919) 00002000:00010000:0.0:1788619907.905123:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fe062d80 00000100:00100000:0.0:1788619907.905170:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 301 00000100:00100000:0.0:1788619907.905181:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe062d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2353:x1875503571334016:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619907.905616:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe062d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2353:x1875503571334016:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 435us (821us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619907.905625:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 301 00002000:00100000:0.0:1788619907.905633:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1101824->1105919) req@ffff8e78fe062d80 x1875503571334016/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:3/0 lens 488/4536 e 0 to 0 dl 1788619918 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619907.905641:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fe062d80 with x1875503571334016 ext(1101824->1105919) 00002000:00010000:0.0:1788619907.905646:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fe062d80 00000100:00100000:2.0:1788619907.918091:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619907.918378:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571334144 00000100:00100000:1.0:1788619907.918409:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571334144 00000100:00100000:1.0:1788619907.918496:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 712 00000100:00100000:1.0:1788619907.918508:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2354:x1875503571334144:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619907.918535:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619907.918563:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe5368733d99 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b386 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619907.918614:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7200/0x1cd7fe5368733d53 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b340 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619907.918683:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5694380 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589052672:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619907.918802:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963300 going to sleep for 11 seconds 00000100:00100000:1.0:1788619907.922546:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5694380 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589052672:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619907.922578:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 358559 00010000:00010000:1.0:1788619907.922602:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe5368733d99 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b386 expref: 20 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619907.922885:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619907.922890:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619907.922898:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe5368733d99 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b386 expref: 20 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619907.922918:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa07e00, rc 0) 00000100:00100000:1.0:1788619907.923071:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2354:x1875503571334144:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4565us (4992us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619907.923082:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 712 00000100:00100000:3.0:1788619907.929147:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619907.929426:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571334272 00000100:00100000:2.0:1788619907.929459:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571334272 00000100:00100000:2.0:1788619907.929612:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 713 00000100:00100000:2.0:1788619907.929784:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2352:x1875503571334272:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619907.929813:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619907.929984:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe5368733da0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b38d expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.930034:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7200/0x1cd7fe5368733d53 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b340 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619907.930086:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5ff4000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589052800:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619907.930197:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccf00 going to sleep for 11 seconds 00000100:00100000:3.0:1788619907.935190:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5ff4000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589052800:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619907.935260:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 358559 00010000:00010000:3.0:1788619907.935288:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe5368733da0 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b38d expref: 20 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619907.935314:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619907.935318:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619907.935371:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe5368733da0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b38d expref: 20 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619907.935392:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6000, rc 0) 00000100:00100000:3.0:1788619907.935480:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2352:x1875503571334272:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5709us (6348us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619907.935490:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 713 00000100:00100000:3.0:1788619907.943733:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619907.944252:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571334400 00000100:00100000:2.0:1788619907.944267:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571334400 00000100:00100000:2.0:1788619907.944385:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 546 00000100:00100000:2.0:1788619907.944398:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+8:14908:x1875503571334400:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619907.944423:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619907.944449:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007527590a/0x1cd7fe5368733da7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b394 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.944510:0:7654:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000007da7fefa/0x1cd7fe5368733cd5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2e5 expref: 11 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.944521:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=1s v=(9 1) 00010000:00010000:2.0:1788619907.944526:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(1 9) 00010000:00010000:2.0:1788619907.944533:0:7654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007527590a/0x1cd7fe5368733da7 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->1056767] (req 1048576->1056767) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b394 expref: 9 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.944606:0:7654:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000007da7fefa/0x1cd7fe5368733cd5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2e5 expref: 11 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619907.944634:0:7654:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000007da7fefa/0x1cd7fe5368733cd5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2e5 expref: 11 pid: 6696 timeout: 500 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619907.944651:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5ff7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589052928:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619907.944767:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc600 going to sleep for 11 seconds 00000100:00100000:2.0:1788619907.950098:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619907.950264:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5ff7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589052928:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619907.950309:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007527590a/0x1cd7fe5368733da7 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->1056767] (req 1048576->1056767) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b394 expref: 9 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619907.950378:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6800, rc 0) 00000100:00100000:0.0:1788619907.950446:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571334400:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6050us (6730us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619907.950456:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 546 00000100:00100000:1.0:1788619907.951370:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571334528 00000100:00100000:1.0:1788619907.951402:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571334528 00010000:00010000:1.0:1788619907.951454:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000007da7fefa/0x1cd7fe5368733cd5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2e5 expref: 12 pid: 6696 timeout: 500 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619907.951510:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 234 00000100:00100000:1.0:1788619907.951522:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571334528:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619907.951542:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619907.951573:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000007da7fefa/0x1cd7fe5368733cd5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2e5 expref: 12 pid: 6696 timeout: 500 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619907.951588:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000007da7fefa/0x1cd7fe5368733cd5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2e5 expref: 12 pid: 6696 timeout: 500 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619907.951747:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007da7fefa/0x1cd7fe5368733cd5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b2e5 expref: 12 pid: 6696 timeout: 500 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619907.951777:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(1 9) 00010000:00010000:1.0:1788619907.951782:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(1 9) 00010000:00010000:1.0:1788619907.951793:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000007527590a/0x1cd7fe5368733da7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1056767) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b394 expref: 8 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619907.951805:0:13330:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007527590a/0x1cd7fe5368733da7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1056767) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b394 expref: 8 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619907.951927:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000007527590a/0x1cd7fe5368733da7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1056767) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b394 expref: 8 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619907.951969:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5694380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589053056:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619907.952129:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963300 going to sleep for 11 seconds 00000100:00100000:2.0:1788619907.966441:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619907.966678:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571334656 00000100:00100000:3.0:1788619907.966691:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571334656 00002000:00100000:3.0:1788619907.966774:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1052672->1634303) req@ffff8e77c5ff7480 x1875503571334656/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:3/0 lens 488/0 e 0 to 0 dl 1788619918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619907.966795:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5ff7480 with x1875503571334656 ext(1052672->1634303) 00002000:00010000:3.0:1788619907.966802:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5ff7480 00000100:00100000:3.0:1788619907.967602:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 302 00000100:00100000:3.0:1788619907.967638:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571334656:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619907.970884:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5694380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589053056:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619907.970925:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619907.971036:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+11:2354:x1875503571334528:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 19514us (20951us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619907.971070:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 234 00000100:00100000:3.0:1788619907.978395:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619907.986665:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571334784 00000100:00100000:1.0:1788619907.986688:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571334784 00002000:00100000:1.0:1788619907.986763:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78cdcc2300 x1875503571334784/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:3/0 lens 488/0 e 0 to 0 dl 1788619918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619907.986786:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78cdcc2300 with x1875503571334784 00000100:00100000:1.0:1788619907.986823:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 396 00000100:00100000:1.0:1788619907.986841:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+14:2354:x1875503571334784:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619907.990794:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+14:2354:x1875503571334784:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 3939us (12399us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619907.990813:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 396 00002000:00100000:1.0:1788619907.990835:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78cdcc2300 x1875503571334784/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:3/0 lens 488/440 e 0 to 0 dl 1788619918 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619907.990847:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78cdcc2300 with x1875503571334784 00000100:00100000:0.0:1788619907.991224:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571334656:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 23607us (24798us total) trans 0 rc 581632/581632 00000100:00100000:0.0:1788619907.991235:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 302 00002000:00100000:0.0:1788619907.991244:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1052672->1634303) req@ffff8e77c5ff7480 x1875503571334656/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:3/0 lens 488/440 e 0 to 0 dl 1788619918 ref 1 fl Complete:/600/0 rc 581632/581632 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619907.991255:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5ff7480 with x1875503571334656 ext(1052672->1634303) 00002000:00010000:0.0:1788619907.991260:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5ff7480 00000100:00100000:3.0:1788619907.999134:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619907.999266:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571334912 00000100:00100000:0.0:1788619907.999280:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571334912 00000100:00100000:0.0:1788619907.999565:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 397 00000100:00100000:0.0:1788619907.999578:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+14:2354:x1875503571334912:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:0.0:1788619907.999860:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 358559 -> 358559 00000004:00010000:0.0:1788619907.999865:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619907 -> 1788619907 00000004:00010000:0.0:1788619907.999867:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619907 -> 1788619907 00000004:00010000:0.0:1788619907.999869:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619907 -> 1788619907 00000004:00010000:0.0:1788619907.999872:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1158 -> 1158 00000100:00100000:0.0:1788619907.999969:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+15:2354:x1875503571334912:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 393us (850us total) trans 4294967575 rc 0/0 00000100:00100000:0.0:1788619907.999977:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 397 00000100:00100000:3.0:1788619908.005145:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619908.005313:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571335040 00000100:00100000:1.0:1788619908.005344:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571335040 00002000:00100000:1.0:1788619908.005445:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1056767) req@ffff8e78cdcc3800 x1875503571335040/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:4/0 lens 8680/0 e 0 to 0 dl 1788619919 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619908.005458:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78cdcc3800 with x1875503571335040 ext(1048576->1056767) 00002000:00010000:1.0:1788619908.005465:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78cdcc3800 00000100:00100000:1.0:1788619908.005493:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 303 00000100:00100000:1.0:1788619908.005505:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571335040:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:1.0:1788619908.006232:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571335040:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 724us (1405us total) trans 4294967375 rc 0/0 00000100:00100000:1.0:1788619908.006244:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 303 00002000:00100000:1.0:1788619908.006253:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1056767) req@ffff8e78cdcc3800 x1875503571335040/t4294967375(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:4/0 lens 8680/448 e 0 to 0 dl 1788619919 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619908.006263:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78cdcc3800 with x1875503571335040 ext(1048576->1056767) 00002000:00010000:1.0:1788619908.006269:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78cdcc3800 00000100:00080000:1.0:1788619908.006548:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1788619908.011474:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619908.012793:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571335168 00000100:00100000:1.0:1788619908.012816:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571335168 00000100:00100000:1.0:1788619908.013166:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 304 00000100:00100000:1.0:1788619908.013182:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571335168:12345-192.168.201.5@tcp:22:fsx.0 00002000:00010000:1.0:1788619908.014919:0:6702:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1633436 -> 1633436 00002000:00010000:1.0:1788619908.014934:0:6702:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619907 -> 1788619907 00002000:00010000:1.0:1788619908.014938:0:6702:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619907 -> 1788619907 00002000:00010000:1.0:1788619908.014952:0:6702:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619907 -> 1788619907 00002000:00010000:1.0:1788619908.014955:0:6702:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 391 -> 391 00000100:00100000:1.0:1788619908.015174:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571335168:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 1985us (3707us total) trans 4294967376 rc 0/0 00000100:00100000:1.0:1788619908.015190:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 304 00000100:00100000:3.0:1788619908.021038:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619908.021098:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619908.021232:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571335424 00000100:00100000:1.0:1788619908.021246:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571335424 00000100:00100000:1.0:1788619908.021379:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 547 00000100:00100000:1.0:1788619908.021392:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+11:2355:x1875503571335424:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619908.021418:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619908.021462:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ce26d4f7/0x1cd7fe5368733dae lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b3a2 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619908.021539:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5697b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589053184:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619908.021645:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd9633c0 going to sleep for 11 seconds 00000100:00080000:2.0:1788619908.022376:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1788619908.022735:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571335296 00000100:00100000:0.0:1788619908.022745:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571335296 00000100:00100000:0.0:1788619908.022838:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 714 00000100:00100000:0.0:1788619908.022850:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2355:x1875503571335296:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619908.022885:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619908.022907:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296200/0x1cd7fe5368733db5 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b39b expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.022964:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7200/0x1cd7fe5368733d53 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b340 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619908.023013:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c30e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589053312:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619908.023108:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00080000:2.0:1788619908.023763:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1788619908.027423:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5697b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589053184:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619908.027480:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ce26d4f7/0x1cd7fe5368733dae lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3a2 expref: 12 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619908.027514:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ce26d4f7/0x1cd7fe5368733dae lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3a2 expref: 12 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619908.027529:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa07800, rc 0) 00000100:00100000:1.0:1788619908.027605:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+11:2355:x1875503571335424:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6215us (6505us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619908.027625:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 547 00000004:00010000:0.0:1788619908.027965:0:9417:0:(mdt_lvb.c:177:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size: 358559 -> 1048576 00000100:00100000:0.0:1788619908.027991:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c30e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589053312:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619908.028012:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:0.0:1788619908.028029:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296200/0x1cd7fe5368733db5 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b39b expref: 20 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.028047:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619908.028050:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619908.028055:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296200/0x1cd7fe5368733db5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b39b expref: 20 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.028068:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296200, rc 0) 00000100:00100000:0.0:1788619908.028098:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2355:x1875503571335296:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5253us (7079us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619908.028106:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 714 00000100:00100000:3.0:1788619908.034441:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619908.039175:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571335552 00000100:00100000:1.0:1788619908.039204:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571335552 00000100:00100000:1.0:1788619908.039445:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 715 00000100:00100000:1.0:1788619908.039458:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+19:14908:x1875503571335552:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619908.039489:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619908.039520:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07800/0x1cd7fe5368733dbc lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b3a9 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619908.039564:0:11397:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7200/0x1cd7fe5368733d53 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b340 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619908.039609:0:11397:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07800/0x1cd7fe5368733dbc lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3a9 expref: 20 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619908.039673:0:11397:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7200/0x1cd7fe5368733d53 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b340 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619908.039689:0:11397:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7200/0x1cd7fe5368733d53 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b340 expref: 14 pid: 5845 timeout: 500 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619908.039708:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5697b80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589053440:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619908.040159:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd9633c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619908.043267:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5697b80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589053440:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619908.043321:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07800/0x1cd7fe5368733dbc lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3a9 expref: 20 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619908.043486:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619908.043490:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619908.043497:0:11397:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07800/0x1cd7fe5368733dbc lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3a9 expref: 20 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619908.043511:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa07800, rc 0) 00000100:00100000:1.0:1788619908.043597:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+20:14908:x1875503571335552:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4140us (9284us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619908.043608:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 715 00000100:00100000:3.0:1788619908.047823:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619908.048225:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571335680 00000100:00100000:1.0:1788619908.048257:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571335680 00002000:00100000:1.0:1788619908.048393:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c501dc00 x1875503571335680/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:4/0 lens 488/0 e 0 to 0 dl 1788619919 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619908.048407:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c501dc00 with x1875503571335680 00010000:00010000:1.0:1788619908.048419:0:5857:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7200/0x1cd7fe5368733d53 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b340 expref: 15 pid: 5845 timeout: 500 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619908.048438:0:5857:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7200/0x1cd7fe5368733d53 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b340 expref: 15 pid: 5845 timeout: 500 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788619908.048446:0:5857:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e77c501dc00 00000100:00100000:1.0:1788619908.048493:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 18 00000100:00100000:1.0:1788619908.048505:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+15:2354:x1875503571335680:12345-192.168.201.5@tcp:4:fsx.0 00000100:00080000:2.0:1788619908.049218:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000004:00010000:1.0:1788619908.056111:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1048576 -> 1048576 00000004:00010000:1.0:1788619908.056126:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619907 -> 1788619907 00000004:00010000:1.0:1788619908.056129:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619907 -> 1788619907 00000004:00010000:1.0:1788619908.056132:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619907 -> 1788619907 00000004:00010000:1.0:1788619908.056134:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1158 -> 1158 00000100:00100000:1.0:1788619908.056685:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+17:2354:x1875503571335680:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 8171us (8870us total) trans 4294967576 rc 0/0 00000100:00100000:1.0:1788619908.056700:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 18 00002000:00100000:1.0:1788619908.056716:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c501dc00 x1875503571335680/t4294967576(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:4/0 lens 488/448 e 0 to 0 dl 1788619919 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619908.056728:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c501dc00 with x1875503571335680 00010000:00010000:1.0:1788619908.056742:0:5857:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7200/0x1cd7fe5368733d53 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b340 expref: 17 pid: 5845 timeout: 500 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619908.056782:0:5857:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7200/0x1cd7fe5368733d53 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b340 expref: 17 pid: 5845 timeout: 500 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788619908.056789:0:5857:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e77c501dc00 00000100:00100000:3.0:1788619908.062371:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619908.066171:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571335808 00000100:00100000:2.0:1788619908.066195:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571335808 00010000:00010000:2.0:1788619908.066862:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7200/0x1cd7fe5368733d53 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b340 expref: 17 pid: 5845 timeout: 500 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619908.066936:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 235 00000100:00100000:2.0:1788619908.066958:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+17:2355:x1875503571335808:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619908.066978:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619908.067015:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7200/0x1cd7fe5368733d53 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b340 expref: 17 pid: 5845 timeout: 500 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619908.067031:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7200/0x1cd7fe5368733d53 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b340 expref: 17 pid: 5845 timeout: 500 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619908.067082:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7200/0x1cd7fe5368733d53 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b340 expref: 17 pid: 5845 timeout: 500 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619908.067102:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619908.067109:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07800/0x1cd7fe5368733dbc lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3a9 expref: 19 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619908.067122:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07800/0x1cd7fe5368733dbc lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3a9 expref: 19 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619908.067133:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07800/0x1cd7fe5368733dbc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3a9 expref: 19 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619908.067184:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07800/0x1cd7fe5368733dbc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3a9 expref: 19 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619908.067200:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5ff5180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589053568:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619908.067369:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cca80 going to sleep for 11 seconds 00000100:00100000:2.0:1788619908.073710:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5ff5180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589053568:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619908.073748:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619908.074229:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571335808:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 7269us (12027us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619908.074245:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 235 00000100:00100000:0.0:1788619908.082481:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619908.083086:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571335936 00000100:00100000:3.0:1788619908.083101:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571335936 00002000:00100000:3.0:1788619908.083172:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5c30e00 x1875503571335936/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:4/0 lens 488/0 e 0 to 0 dl 1788619919 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619908.083186:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5c30e00 with x1875503571335936 00000100:00100000:3.0:1788619908.083239:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 398 00000100:00100000:3.0:1788619908.083250:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c30e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2355:x1875503571335936:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619908.104017:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c30e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2355:x1875503571335936:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 20750us (21534us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1788619908.104060:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 398 00002000:00100000:0.0:1788619908.104079:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5c30e00 x1875503571335936/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:4/0 lens 488/440 e 0 to 0 dl 1788619919 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619908.104092:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5c30e00 with x1875503571335936 00000100:00100000:0.0:1788619908.109316:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619908.109518:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571336064 00000100:00100000:1.0:1788619908.109529:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571336064 00002000:00100000:1.0:1788619908.109591:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5c33100 x1875503571336064/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:4/0 lens 488/0 e 0 to 0 dl 1788619919 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619908.109603:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5c33100 with x1875503571336064 00000100:00100000:1.0:1788619908.109723:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 399 00000100:00100000:1.0:1788619908.109733:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c33100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2355:x1875503571336064:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:1.0:1788619908.112453:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1048576 -> 1048576 00000004:00010000:1.0:1788619908.112463:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619907 -> 1788619907 00000004:00010000:1.0:1788619908.112465:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619907 -> 1788619907 00000004:00010000:1.0:1788619908.112467:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619907 -> 1788619907 00000004:00010000:1.0:1788619908.112469:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1158 -> 1158 00000100:00100000:1.0:1788619908.112873:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c33100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2355:x1875503571336064:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 3132us (3560us total) trans 4294967577 rc 0/0 00000100:00100000:1.0:1788619908.112887:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 399 00002000:00100000:1.0:1788619908.112904:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5c33100 x1875503571336064/t4294967577(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:4/0 lens 488/448 e 0 to 0 dl 1788619919 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619908.112915:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5c33100 with x1875503571336064 00000100:00100000:0.0:1788619908.116139:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619908.116203:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571336192 00000100:00100000:3.0:1788619908.116222:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571336192 00000100:00100000:3.0:1788619908.116287:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 716 00000100:00100000:3.0:1788619908.116298:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c30000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2354:x1875503571336192:12345-192.168.201.5@tcp:16:fsx.0 00000001:00080000:1.0:1788619908.147298:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967575 is committed 00000001:00080000:1.0:1788619908.147348:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967577 is committed 00000001:00080000:1.0:1788619908.147384:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967576 is committed 00000100:00100000:2.0:1788619908.147669:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c30000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571336192:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 31363us (31531us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619908.147695:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 716 00000100:00100000:2.0:1788619908.167237:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619908.167976:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571336320 00000100:00100000:3.0:1788619908.167992:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571336320 00000100:00100000:3.0:1788619908.168085:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 717 00000100:00100000:3.0:1788619908.168098:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571336320:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619908.168133:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619908.168163:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368733dc3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b3b0 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619908.168226:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07800/0x1cd7fe5368733dbc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3a9 expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619908.168308:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589053696:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619908.168477:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5c76a80 going to sleep for 11 seconds 00000100:00100000:0.0:1788619908.174724:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589053696:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619908.174763:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:0.0:1788619908.174792:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368733dc3 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3b0 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.174819:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619908.174823:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619908.174830:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368733dc3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3b0 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.176400:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c600, rc 0) 00000100:00100000:0.0:1788619908.176516:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571336320:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8416us (9290us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619908.176527:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 717 00000100:00100000:1.0:1788619908.184483:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619908.193917:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571336448 00000100:00100000:0.0:1788619908.193948:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571336448 00000100:00100000:0.0:1788619908.194046:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 718 00000100:00100000:0.0:1788619908.194059:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5697b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571336448:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619908.194088:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619908.194140:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe5368733dca lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b3b7 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.194199:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07800/0x1cd7fe5368733dbc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3a9 expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619908.194272:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c30700 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589053824:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619908.194455:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:0.0:1788619908.197570:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c30700 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589053824:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619908.197606:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:0.0:1788619908.197632:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe5368733dca lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3b7 expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.197658:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619908.197661:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619908.197667:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe5368733dca lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3b7 expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.197686:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297e00, rc 0) 00000100:00100000:0.0:1788619908.197883:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5697b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571336448:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3825us (13413us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619908.197894:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 718 00000100:00100000:3.0:1788619908.214629:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619908.216541:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571336576 00000100:00100000:0.0:1788619908.216567:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571336576 00000100:00100000:1.0:1788619908.222263:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 548 00000100:00100000:1.0:1788619908.222281:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+11:2353:x1875503571336576:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619908.222318:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619908.222362:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005221cf7b/0x1cd7fe5368733dd1 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b3be expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619908.222450:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5696a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589053952:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619908.222648:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd9639c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619908.251906:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5696a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589053952:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619908.251976:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005221cf7b/0x1cd7fe5368733dd1 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3be expref: 12 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619908.252012:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005221cf7b/0x1cd7fe5368733dd1 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3be expref: 12 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619908.252030:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa07600, rc 0) 00000100:00100000:2.0:1788619908.252139:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+11:2353:x1875503571336576:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 29861us (37526us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619908.252150:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 548 00000100:00100000:2.0:1788619908.270285:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619908.270591:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571336704 00000100:00100000:3.0:1788619908.270779:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571336704 00000100:00100000:3.0:1788619908.271154:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 549 00000100:00100000:3.0:1788619908.271169:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+11:14908:x1875503571336704:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619908.271194:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619908.271231:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000095a50c3e/0x1cd7fe5368733dd8 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b3c5 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619908.271279:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000007527590a/0x1cd7fe5368733da7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1056767) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b394 expref: 11 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619908.271290:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(1 9) 00010000:00010000:3.0:1788619908.271295:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 9) 00010000:00010000:3.0:1788619908.271301:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000095a50c3e/0x1cd7fe5368733dd8 lrc: 4/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1617920->1622015] (req 1617920->1622015) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3c5 expref: 12 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619908.271402:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000007527590a/0x1cd7fe5368733da7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1056767) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b394 expref: 11 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619908.271431:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000007527590a/0x1cd7fe5368733da7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1056767) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b394 expref: 11 pid: 7654 timeout: 500 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619908.271474:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900d5ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589054080:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619908.271583:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5c76480 going to sleep for 11 seconds 00000100:00100000:2.0:1788619908.280183:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900d5ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589054080:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619908.280297:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000095a50c3e/0x1cd7fe5368733dd8 lrc: 3/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1617920->1622015] (req 1617920->1622015) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3c5 expref: 12 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619908.280544:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d600, rc 0) 00000100:00100000:2.0:1788619908.281292:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+12:14908:x1875503571336704:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 10122us (11017us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619908.281306:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 549 00000100:00100000:0.0:1788619908.282119:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619908.283138:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571336832 00000100:00100000:2.0:1788619908.283152:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571336832 00010000:00010000:2.0:1788619908.283240:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000007527590a/0x1cd7fe5368733da7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1056767) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b394 expref: 12 pid: 7654 timeout: 500 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619908.283293:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 236 00000100:00100000:2.0:1788619908.283305:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8020c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571336832:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619908.283350:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619908.283380:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000007527590a/0x1cd7fe5368733da7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1056767) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b394 expref: 12 pid: 7654 timeout: 500 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619908.283399:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000007527590a/0x1cd7fe5368733da7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1056767) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b394 expref: 12 pid: 7654 timeout: 500 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619908.283430:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007527590a/0x1cd7fe5368733da7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1056767) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b394 expref: 12 pid: 7654 timeout: 500 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619908.283453:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 9) 00010000:00010000:2.0:1788619908.283458:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 9) 00010000:00010000:2.0:1788619908.283469:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000095a50c3e/0x1cd7fe5368733dd8 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1617920->1622015) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3c5 expref: 11 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619908.283480:0:5831:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000095a50c3e/0x1cd7fe5368733dd8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1617920->1622015) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3c5 expref: 11 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619908.283533:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000095a50c3e/0x1cd7fe5368733dd8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1617920->1622015) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3c5 expref: 11 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619908.283566:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5ff5880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5831:1875503589054208:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619908.284524:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc300 going to sleep for 11 seconds 00000100:00100000:3.0:1788619908.292481:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5ff5880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5831:1875503589054208:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619908.292523:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619908.292695:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8020c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571336832:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 9390us (10578us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619908.292710:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 236 00000100:00100000:0.0:1788619908.293586:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619908.294741:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571336960 00000100:00100000:3.0:1788619908.294757:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571336960 00002000:00100000:3.0:1788619908.294963:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1634303) req@ffff8e78c5c30700 x1875503571336960/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:4/0 lens 488/0 e 0 to 0 dl 1788619919 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619908.294977:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5c30700 with x1875503571336960 ext(1048576->1634303) 00002000:00010000:3.0:1788619908.294984:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5c30700 00000100:00100000:3.0:1788619908.295139:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 305 00000100:00100000:3.0:1788619908.295151:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c30700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571336960:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619908.304103:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c30700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571336960:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 8935us (10503us total) trans 0 rc 585728/585728 00000100:00100000:1.0:1788619908.304121:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 305 00002000:00100000:1.0:1788619908.304139:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1634303) req@ffff8e78c5c30700 x1875503571336960/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:4/0 lens 488/440 e 0 to 0 dl 1788619919 ref 1 fl Complete:/600/0 rc 585728/585728 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619908.304151:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5c30700 with x1875503571336960 ext(1048576->1634303) 00002000:00010000:1.0:1788619908.304160:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5c30700 00000100:00100000:2.0:1788619908.309670:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619908.309908:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571337088 00000100:00100000:0.0:1788619908.309923:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571337088 00000100:00100000:0.0:1788619908.310134:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 719 00000100:00100000:0.0:1788619908.310148:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571337088:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619908.310181:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619908.310222:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296200/0x1cd7fe5368733ddf lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b3cc expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.310267:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07800/0x1cd7fe5368733dbc lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3a9 expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.310280:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296200/0x1cd7fe5368733ddf lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3cc expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.310353:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07800/0x1cd7fe5368733dbc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3a9 expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.310373:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07800/0x1cd7fe5368733dbc lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3a9 expref: 12 pid: 11397 timeout: 500 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619908.310389:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c30a80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589054336:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619908.310503:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:0.0:1788619908.313734:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c30a80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589054336:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619908.313796:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296200/0x1cd7fe5368733ddf lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3cc expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.313822:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619908.313826:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619908.313980:0:5844:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296200/0x1cd7fe5368733ddf lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3cc expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.313997:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296200, rc 0) 00000100:00100000:1.0:1788619908.314225:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:14908:x1875503571337088:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4076us (4566us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619908.314236:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 719 00000100:00100000:1.0:1788619908.320671:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619908.320778:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571337216 00000100:00100000:3.0:1788619908.320793:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571337216 00010000:00010000:3.0:1788619908.320860:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07800/0x1cd7fe5368733dbc lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3a9 expref: 13 pid: 11397 timeout: 500 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619908.321250:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 237 00000100:00100000:2.0:1788619908.321268:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5694000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571337216:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619908.321291:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619908.321375:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07800/0x1cd7fe5368733dbc lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3a9 expref: 13 pid: 11397 timeout: 500 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619908.321393:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07800/0x1cd7fe5368733dbc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3a9 expref: 13 pid: 11397 timeout: 500 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619908.321421:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07800/0x1cd7fe5368733dbc lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3a9 expref: 13 pid: 11397 timeout: 500 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619908.321443:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619908.321451:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296200/0x1cd7fe5368733ddf lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3cc expref: 9 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619908.321465:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296200/0x1cd7fe5368733ddf lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3cc expref: 9 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619908.321475:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296200/0x1cd7fe5368733ddf lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3cc expref: 9 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619908.321557:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296200/0x1cd7fe5368733ddf lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3cc expref: 9 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619908.321573:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5ff6680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589054464:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619908.321680:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619908.330262:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5ff6680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589054464:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619908.330298:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619908.330420:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5694000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2355:x1875503571337216:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 9157us (9765us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619908.330436:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 237 00000100:00100000:0.0:1788619908.333478:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619908.333637:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571337344 00000100:00100000:3.0:1788619908.333651:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571337344 00002000:00100000:3.0:1788619908.333723:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5c30a80 x1875503571337344/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:4/0 lens 488/0 e 0 to 0 dl 1788619919 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619908.333773:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5c30a80 with x1875503571337344 00000100:00100000:3.0:1788619908.333823:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 400 00000100:00100000:3.0:1788619908.333961:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c30a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571337344:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619908.409508:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c30a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571337344:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 75657us (76030us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1788619908.409529:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 400 00002000:00100000:3.0:1788619908.409547:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5c30a80 x1875503571337344/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:4/0 lens 488/440 e 0 to 0 dl 1788619919 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619908.409559:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5c30a80 with x1875503571337344 00000100:00100000:1.0:1788619908.415752:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619908.416049:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571337472 00000100:00100000:0.0:1788619908.416064:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571337472 00000100:00100000:0.0:1788619908.416149:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 550 00000100:00100000:0.0:1788619908.416162:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5695c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571337472:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619908.416187:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619908.416222:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000084d1816c/0x1cd7fe5368733de6 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b3d3 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.416258:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=1s v=(2 9) 00010000:00010000:0.0:1788619908.416263:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 9) 00010000:00010000:0.0:1788619908.416277:0:7654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000084d1816c/0x1cd7fe5368733de6 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3d3 expref: 12 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.416304:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000084d1816c/0x1cd7fe5368733de6 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3d3 expref: 12 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.416321:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6e000, rc 0) 00000100:00100000:0.0:1788619908.416456:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5695c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571337472:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 296us (721us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619908.416464:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 550 00000100:00100000:0.0:1788619908.423491:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619908.423673:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571337600 00000100:00100000:3.0:1788619908.423688:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571337600 00000100:00100000:3.0:1788619908.423903:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 720 00000100:00100000:3.0:1788619908.423917:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c31180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2352:x1875503571337600:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619908.423956:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619908.423983:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368733ded lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b3da expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619908.424032:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368733ded lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3da expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788619908.424051:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:3.0:1788619908.424060:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368733ded lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3da expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619908.424076:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619908.424099:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619908.424105:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683cc00, rc 0) 00000100:00100000:3.0:1788619908.424189:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c31180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571337600:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 275us (714us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619908.424197:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 720 00000100:00100000:3.0:1788619908.439767:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619908.440425:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571337728 00000100:00100000:1.0:1788619908.440441:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571337728 00002000:00100000:1.0:1788619908.440512:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1634303) req@ffff8e7900d5e680 x1875503571337728/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:4/0 lens 488/0 e 0 to 0 dl 1788619919 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619908.440548:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e7900d5e680 with x1875503571337728 ext(1048576->1634303) 00002000:00010000:1.0:1788619908.440554:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e7900d5e680 00000100:00100000:1.0:1788619908.440862:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 306 00000100:00100000:1.0:1788619908.440877:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2352:x1875503571337728:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619908.456773:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2352:x1875503571337728:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 15873us (16993us total) trans 0 rc 585728/585728 00000100:00100000:1.0:1788619908.456793:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 306 00002000:00100000:1.0:1788619908.456812:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1634303) req@ffff8e7900d5e680 x1875503571337728/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:4/0 lens 488/440 e 0 to 0 dl 1788619919 ref 1 fl Complete:/600/0 rc 585728/585728 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619908.456840:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e7900d5e680 with x1875503571337728 ext(1048576->1634303) 00002000:00010000:1.0:1788619908.456849:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e7900d5e680 00000100:00100000:2.0:1788619908.465223:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619908.465453:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571337856 00000100:00100000:1.0:1788619908.465468:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571337856 00000100:00100000:1.0:1788619908.465545:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 551 00000100:00100000:1.0:1788619908.465559:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:14908:x1875503571337856:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619908.465584:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619908.465609:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c5c78e09/0x1cd7fe5368733df4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b3e1 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619908.465653:0:6698:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000095a50c3e/0x1cd7fe5368733dd8 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1617920->1622015) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3c5 expref: 13 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619908.465664:0:6698:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000084d1816c/0x1cd7fe5368733de6 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3d3 expref: 11 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619908.465693:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 9) 00010000:00010000:1.0:1788619908.465698:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 9) 00010000:00010000:1.0:1788619908.465705:0:6698:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c5c78e09/0x1cd7fe5368733df4 lrc: 5/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [1826816->1875967] (req 1826816->1875967) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3e1 expref: 13 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619908.465749:0:6698:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000084d1816c/0x1cd7fe5368733de6 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3d3 expref: 11 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619908.465766:0:6698:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000084d1816c/0x1cd7fe5368733de6 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3d3 expref: 11 pid: 7654 timeout: 501 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619908.465783:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5697100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589054592:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619908.466104:0:6698:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000095a50c3e/0x1cd7fe5368733dd8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1617920->1622015) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3c5 expref: 13 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619908.466116:0:6698:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000095a50c3e/0x1cd7fe5368733dd8 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1617920->1622015) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3c5 expref: 13 pid: 6697 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619908.466126:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5696a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589054720:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619908.466156:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963f00 going to sleep for 11 seconds 00000100:00100000:1.0:1788619908.477746:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5697100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589054592:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619908.477768:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5696a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589054720:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619908.478000:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c5c78e09/0x1cd7fe5368733df4 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [1826816->1875967] (req 1826816->1875967) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3e1 expref: 13 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619908.478030:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed1400, rc 0) 00000100:00100000:1.0:1788619908.478114:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571337856:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 12558us (12918us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619908.478124:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 551 00000100:00100000:3.0:1788619908.483466:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619908.483513:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619908.483873:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571337984 00000100:00100000:0.0:1788619908.483923:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571337984 00010000:00010000:0.0:1788619908.483988:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000095a50c3e/0x1cd7fe5368733dd8 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1617920->1622015) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3c5 expref: 13 pid: 6697 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619908.484033:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571338112 00000100:00100000:0.0:1788619908.484036:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571338112 00010000:00010000:0.0:1788619908.484051:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000084d1816c/0x1cd7fe5368733de6 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3d3 expref: 12 pid: 7654 timeout: 501 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619908.484083:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 238 00000100:00100000:0.0:1788619908.484093:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571337984:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619908.484114:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619908.484146:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000095a50c3e/0x1cd7fe5368733dd8 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1617920->1622015) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3c5 expref: 13 pid: 6697 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.484159:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000095a50c3e/0x1cd7fe5368733dd8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1617920->1622015) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3c5 expref: 13 pid: 6697 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.484189:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000095a50c3e/0x1cd7fe5368733dd8 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1617920->1622015) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3c5 expref: 13 pid: 6697 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.484212:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619908.484277:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571337984:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 185us (827us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619908.484285:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 238 00000100:00100000:0.0:1788619908.484306:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 239 00000100:00100000:0.0:1788619908.484312:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571338112:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619908.484384:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619908.484395:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000084d1816c/0x1cd7fe5368733de6 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3d3 expref: 12 pid: 7654 timeout: 501 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.484406:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000084d1816c/0x1cd7fe5368733de6 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3d3 expref: 12 pid: 7654 timeout: 501 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.484420:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000084d1816c/0x1cd7fe5368733de6 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3d3 expref: 12 pid: 7654 timeout: 501 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.484432:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 9) 00010000:00010000:0.0:1788619908.484436:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 9) 00010000:00010000:0.0:1788619908.484447:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000c5c78e09/0x1cd7fe5368733df4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1826816->1875967) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3e1 expref: 11 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.484459:0:13330:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c5c78e09/0x1cd7fe5368733df4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1826816->1875967) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3e1 expref: 11 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.484505:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000c5c78e09/0x1cd7fe5368733df4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1826816->1875967) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3e1 expref: 11 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619908.484523:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c31f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589054848:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619908.484591:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa540 going to sleep for 11 seconds 00000100:00100000:2.0:1788619908.493464:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619908.494011:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571338240 00000100:00100000:3.0:1788619908.494024:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571338240 00000100:00100000:3.0:1788619908.494103:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 552 00000100:00100000:3.0:1788619908.494116:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571338240:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619908.494141:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619908.494163:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe5368733dfb lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b3e8 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619908.494244:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900d5d180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589054976:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619908.494394:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5c76c00 going to sleep for 11 seconds 00000100:00100000:0.0:1788619908.494534:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c31f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589054848:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619908.494562:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619908.494611:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+13:2355:x1875503571338112:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 10298us (11097us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619908.494622:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 239 00002000:00010000:0.0:1788619908.498670:0:7654:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size: 1633436 -> 1873303 00002000:00010000:0.0:1788619908.498685:0:7654:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime: 1788619907 -> 1788619908 00002000:00010000:0.0:1788619908.498688:0:7654:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619907 -> 1788619908 00002000:00010000:0.0:1788619908.498690:0:7654:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime: 1788619907 -> 1788619908 00000100:00100000:0.0:1788619908.498726:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900d5d180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589054976:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619908.498786:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe5368733dfb lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3e8 expref: 12 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.498824:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe5368733dfb lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3e8 expref: 12 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.498845:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c600, rc 0) 00000100:00100000:0.0:1788619908.498923:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571338240:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4809us (5475us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619908.498942:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 552 00000100:00100000:1.0:1788619908.502799:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619908.502896:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571338368 00000100:00100000:2.0:1788619908.502909:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571338368 00000100:00100000:2.0:1788619908.502990:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 553 00000100:00100000:2.0:1788619908.503003:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5696a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571338368:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619908.503028:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619908.503053:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a016b877/0x1cd7fe5368733e02 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b3ef expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619908.503137:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5ff5500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589055104:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619908.503317:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619908.507845:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5ff5500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589055104:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619908.507901:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a016b877/0x1cd7fe5368733e02 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3ef expref: 12 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619908.507945:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a016b877/0x1cd7fe5368733e02 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3ef expref: 12 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619908.507962:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7000, rc 0) 00000100:00100000:3.0:1788619908.508121:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5696a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571338368:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5118us (5335us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619908.508131:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 553 00000100:00100000:2.0:1788619908.518481:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619908.520186:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571338496 00000100:00100000:1.0:1788619908.520216:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571338496 00000100:00100000:1.0:1788619908.520316:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 554 00000100:00100000:1.0:1788619908.520374:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571338496:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619908.520402:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619908.520428:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d4b76022/0x1cd7fe5368733e09 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b3f6 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619908.520474:0:6696:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000c5c78e09/0x1cd7fe5368733df4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1826816->1875967) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3e1 expref: 11 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619908.520486:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 9) 00010000:00010000:1.0:1788619908.520491:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(3 9) 00010000:00010000:1.0:1788619908.520497:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d4b76022/0x1cd7fe5368733e09 lrc: 4/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1384448->1392639] (req 1384448->1392639) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3f6 expref: 12 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619908.520544:0:6696:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000c5c78e09/0x1cd7fe5368733df4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1826816->1875967) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3e1 expref: 11 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619908.520587:0:6696:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000c5c78e09/0x1cd7fe5368733df4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1826816->1875967) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3e1 expref: 11 pid: 6698 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619908.520606:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5697100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589055232:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619908.520724:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963f00 going to sleep for 11 seconds 00000100:00100000:3.0:1788619908.528267:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619908.528595:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571338624 00000100:00100000:1.0:1788619908.528610:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571338624 00002000:00100000:1.0:1788619908.528679:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1826816->1873302) req@ffff8e7900d5d180 x1875503571338624/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:4/0 lens 488/0 e 0 to 0 dl 1788619919 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619908.528693:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e7900d5d180 with x1875503571338624 ext(1826816->1873302) 00010000:00010000:1.0:1788619908.528705:0:6701:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000c5c78e09/0x1cd7fe5368733df4 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1826816->1875967) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3e1 expref: 12 pid: 6698 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619908.528727:0:6701:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000c5c78e09/0x1cd7fe5368733df4 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1826816->1875967) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3e1 expref: 12 pid: 6698 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788619908.528735:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff8e7900d5d180 00000100:00100000:1.0:1788619908.528784:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 10 00000100:00100000:1.0:1788619908.528796:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571338624:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:2.0:1788619908.529840:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5697100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589055232:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619908.529899:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d4b76022/0x1cd7fe5368733e09 lrc: 3/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1384448->1392639] (req 1384448->1392639) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3f6 expref: 12 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619908.529924:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed0a00, rc 0) 00000100:00100000:2.0:1788619908.530003:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:14908:x1875503571338496:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 9675us (11534us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619908.530013:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 554 00000100:00100000:0.0:1788619908.538915:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571338624:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 10095us (10637us total) trans 4294967377 rc 0/0 00000100:00100000:0.0:1788619908.539824:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 10 00002000:00100000:0.0:1788619908.539852:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1826816->1873302) req@ffff8e7900d5d180 x1875503571338624/t4294967377(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:4/0 lens 488/448 e 0 to 0 dl 1788619919 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619908.539865:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e7900d5d180 with x1875503571338624 ext(1826816->1873302) 00010000:00010000:0.0:1788619908.539880:0:6701:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000c5c78e09/0x1cd7fe5368733df4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1826816->1875967) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3e1 expref: 14 pid: 6698 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.540030:0:6701:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000c5c78e09/0x1cd7fe5368733df4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1826816->1875967) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3e1 expref: 14 pid: 6698 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788619908.540042:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff8e7900d5d180 00000100:00100000:3.0:1788619908.544152:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619908.544587:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571338752 00000100:00100000:1.0:1788619908.544600:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571338752 00010000:00010000:1.0:1788619908.544655:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000c5c78e09/0x1cd7fe5368733df4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1826816->1875967) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3e1 expref: 14 pid: 6698 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619908.544735:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 240 00000100:00100000:1.0:1788619908.544747:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5ca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571338752:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619908.544768:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619908.544801:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000c5c78e09/0x1cd7fe5368733df4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1826816->1875967) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3e1 expref: 14 pid: 6698 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619908.544834:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000c5c78e09/0x1cd7fe5368733df4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1826816->1875967) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3e1 expref: 14 pid: 6698 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619908.544864:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c5c78e09/0x1cd7fe5368733df4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1826816->1875967) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3e1 expref: 14 pid: 6698 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619908.544887:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(3 9) 00010000:00010000:1.0:1788619908.544891:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(3 9) 00010000:00010000:1.0:1788619908.544913:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000d4b76022/0x1cd7fe5368733e09 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1384448->1392639) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3f6 expref: 11 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619908.544925:0:5829:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d4b76022/0x1cd7fe5368733e09 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1384448->1392639) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3f6 expref: 11 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619908.544984:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000d4b76022/0x1cd7fe5368733e09 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1384448->1392639) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3f6 expref: 11 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619908.545000:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5694380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1875503589055360:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619908.545116:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd9639c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619908.550669:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5694380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1875503589055360:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619908.550704:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619908.550795:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5ca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571338752:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 6049us (6656us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619908.550809:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 240 00000100:00100000:2.0:1788619908.558897:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619908.561713:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571338880 00000100:00100000:1.0:1788619908.561738:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571338880 00002000:00100000:1.0:1788619908.561824:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1875967) req@ffff8e77c5ff7b80 x1875503571338880/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:4/0 lens 488/0 e 0 to 0 dl 1788619919 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619908.561849:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5ff7b80 with x1875503571338880 ext(1048576->1875967) 00002000:00010000:1.0:1788619908.561857:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5ff7b80 00000100:00100000:1.0:1788619908.562132:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 307 00000100:00100000:1.0:1788619908.562147:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571338880:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619908.577540:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571338880:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 15380us (18645us total) trans 0 rc 827392/827392 00000100:00100000:0.0:1788619908.577560:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 307 00002000:00100000:0.0:1788619908.577577:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1875967) req@ffff8e77c5ff7b80 x1875503571338880/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:4/0 lens 488/440 e 0 to 0 dl 1788619919 ref 1 fl Complete:/600/0 rc 827392/827392 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619908.577590:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5ff7b80 with x1875503571338880 ext(1048576->1875967) 00002000:00010000:0.0:1788619908.577598:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5ff7b80 00000100:00100000:2.0:1788619908.586015:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619908.586150:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571339008 00000100:00100000:1.0:1788619908.586163:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571339008 00000100:00100000:1.0:1788619908.586299:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 555 00000100:00100000:1.0:1788619908.586314:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571339008:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619908.586556:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619908.586582:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000092f9790d/0x1cd7fe5368733e10 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b3fd expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619908.586618:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(3 9) 00010000:00010000:2.0:1788619908.586623:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(3 9) 00010000:00010000:2.0:1788619908.586636:0:6698:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000092f9790d/0x1cd7fe5368733e10 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3fd expref: 14 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619908.586658:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000092f9790d/0x1cd7fe5368733e10 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3fd expref: 14 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619908.586676:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7800, rc 0) 00000100:00100000:2.0:1788619908.594875:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2353:x1875503571339008:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 444us (755us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619908.594910:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 555 00000100:00100000:1.0:1788619908.599174:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619908.599247:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571339136 00000100:00100000:0.0:1788619908.599262:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571339136 00000100:00100000:0.0:1788619908.599420:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 721 00000100:00100000:0.0:1788619908.599433:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5694380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+10:14908:x1875503571339136:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619908.599462:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619908.599466:0:11397:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:0.0:1788619908.599528:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296200/0x1cd7fe5368733ddf lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3cc expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.599553:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619908.599560:0:11397:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788619908.599576:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368733e17 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b404 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.599631:0:11397:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368733ded lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3da expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.599642:0:11397:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368733e17 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b404 expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.599683:0:11397:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368733ded lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3da expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.599698:0:11397:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368733ded lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3da expref: 12 pid: 9417 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619908.599715:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c31f80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589055488:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619908.600200:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa540 going to sleep for 11 seconds 00000100:00100000:0.0:1788619908.602533:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c31f80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589055488:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619908.602587:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368733e17 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b404 expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.602611:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619908.602614:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619908.602619:0:11397:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368733e17 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b404 expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.602631:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6e000, rc 0) 00000100:00100000:3.0:1788619908.602827:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619908.603003:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571339264 00000100:00100000:1.0:1788619908.603017:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571339264 00000100:00100000:0.0:1788619908.603036:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5694380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+10:14908:x1875503571339136:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3604us (3873us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619908.603066:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 721 00010000:00010000:1.0:1788619908.603260:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368733ded lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3da expref: 13 pid: 9417 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619908.603309:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 241 00000100:00100000:1.0:1788619908.603320:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571339264:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619908.603390:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619908.603422:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368733ded lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3da expref: 13 pid: 9417 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619908.603434:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368733ded lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3da expref: 13 pid: 9417 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619908.603461:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368733ded lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3da expref: 13 pid: 9417 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619908.603473:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619908.603477:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368733e17 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b404 expref: 9 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619908.603486:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368733e17 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b404 expref: 9 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619908.603493:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368733e17 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b404 expref: 9 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619908.603539:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368733e17 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b404 expref: 9 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619908.603553:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5697100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589055616:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619908.603621:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963180 going to sleep for 11 seconds 00000100:00100000:1.0:1788619908.606915:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5697100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589055616:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619908.606961:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619908.607028:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2355:x1875503571339264:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3710us (4202us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619908.607039:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 241 00000100:00100000:3.0:1788619908.613575:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619908.613706:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571339392 00000100:00100000:1.0:1788619908.613718:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571339392 00000100:00100000:1.0:1788619908.613820:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 401 00000100:00100000:1.0:1788619908.613829:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571339392:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:0.0:1788619908.614479:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1048576 -> 1048576 00000004:00010000:0.0:1788619908.614486:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619907 -> 1788619907 00000004:00010000:0.0:1788619908.614489:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619907 -> 1788619907 00000004:00010000:0.0:1788619908.614491:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619907 -> 1788619907 00000004:00010000:0.0:1788619908.614493:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1158 -> 1030 00000100:00100000:0.0:1788619908.614649:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571339392:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 816us (1088us total) trans 4294967578 rc 0/0 00000100:00100000:0.0:1788619908.614659:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 401 00000100:00100000:3.0:1788619908.621485:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619908.622016:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571339520 00000100:00100000:1.0:1788619908.622033:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571339520 00000100:00100000:1.0:1788619908.622142:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 722 00000100:00100000:1.0:1788619908.622157:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b16a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2355:x1875503571339520:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619908.622187:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619908.622224:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f48400/0x1cd7fe5368733e1e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b40b expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619908.622276:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368733e17 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b404 expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619908.622378:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5697100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589055744:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619908.622498:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963180 going to sleep for 11 seconds 00000004:00010000:1.0:1788619908.626534:0:5844:0:(mdt_lvb.c:183:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime: 1788619907 -> 1788619908 00000004:00010000:1.0:1788619908.626550:0:5844:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619907 -> 1788619908 00000004:00010000:1.0:1788619908.626553:0:5844:0:(mdt_lvb.c:195:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime: 1788619907 -> 1788619908 00000100:00100000:1.0:1788619908.626610:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5697100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589055744:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619908.626638:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:1.0:1788619908.626664:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f48400/0x1cd7fe5368733e1e lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b40b expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619908.626689:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619908.626692:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619908.626697:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f48400/0x1cd7fe5368733e1e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b40b expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619908.626738:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f48400, rc 0) 00000100:00100000:1.0:1788619908.627251:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b16a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2355:x1875503571339520:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5091us (7335us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619908.627262:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 722 00000100:00100000:3.0:1788619908.638902:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619908.639294:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571339648 00000100:00100000:2.0:1788619908.639502:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571339648 00000100:00100000:0.0:1788619908.640480:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 723 00000100:00100000:0.0:1788619908.640500:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df965f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2355:x1875503571339648:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619908.640545:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619908.640576:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f400/0x1cd7fe5368733e25 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b412 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.640638:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368733e17 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b404 expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619908.640693:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c31f80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589055872:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619908.640810:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa540 going to sleep for 11 seconds 00000100:00100000:0.0:1788619908.649307:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c31f80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589055872:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619908.649618:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:0.0:1788619908.649646:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f400/0x1cd7fe5368733e25 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b412 expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.649704:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619908.649708:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619908.649715:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f400/0x1cd7fe5368733e25 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b412 expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.649735:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6f400, rc 0) 00000100:00100000:0.0:1788619908.649830:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df965f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2355:x1875503571339648:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 9337us (10941us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619908.649842:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 723 00000100:00100000:3.0:1788619908.658243:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619908.658568:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571339776 00000100:00100000:0.0:1788619908.658581:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571339776 00000100:00100000:0.0:1788619908.658660:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 724 00000100:00100000:0.0:1788619908.658673:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571339776:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619908.658699:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619908.658726:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f400/0x1cd7fe5368733e2c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b419 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.658775:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368733e17 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b404 expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619908.658828:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c31f80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589056000:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619908.658951:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa540 going to sleep for 11 seconds 00000100:00100000:0.0:1788619908.661727:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c31f80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589056000:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619908.662281:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:0.0:1788619908.662311:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f400/0x1cd7fe5368733e2c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b419 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.662415:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619908.662419:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619908.662428:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f400/0x1cd7fe5368733e2c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b419 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.662448:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6f400, rc 0) 00000100:00100000:0.0:1788619908.662535:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571339776:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3864us (4308us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619908.662544:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 724 00000100:00100000:3.0:1788619908.669528:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619908.669617:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571339904 00000100:00100000:1.0:1788619908.669634:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571339904 00000100:00100000:1.0:1788619908.669726:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 725 00000100:00100000:1.0:1788619908.669740:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d01500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2353:x1875503571339904:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619908.669771:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619908.669800:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f48400/0x1cd7fe5368733e33 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b420 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619908.669859:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368733e17 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b404 expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619908.669944:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5697100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589056128:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619908.670058:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963180 going to sleep for 11 seconds 00000100:00100000:1.0:1788619908.673548:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5697100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589056128:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619908.673589:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:1.0:1788619908.673804:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f48400/0x1cd7fe5368733e33 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b420 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619908.674084:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619908.674088:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619908.674096:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f48400/0x1cd7fe5368733e33 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b420 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619908.674149:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f48400, rc 0) 00000100:00100000:1.0:1788619908.674307:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d01500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2353:x1875503571339904:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4568us (4978us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619908.674318:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 725 00000100:00100000:3.0:1788619908.680017:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619908.680241:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571340032 00000100:00100000:0.0:1788619908.680253:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571340032 00000100:00100000:0.0:1788619908.680371:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 556 00000100:00100000:0.0:1788619908.680384:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d02680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571340032:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619908.680409:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619908.680433:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000244b3d37/0x1cd7fe5368733e3a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b427 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.680479:0:7654:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000d4b76022/0x1cd7fe5368733e09 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1384448->1392639) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3f6 expref: 11 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.680490:0:7654:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000092f9790d/0x1cd7fe5368733e10 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3fd expref: 15 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.680497:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(3 9) 00010000:00010000:0.0:1788619908.680503:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(3 9) 00010000:00010000:0.0:1788619908.680527:0:7654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000244b3d37/0x1cd7fe5368733e3a lrc: 5/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [1527808->1535999] (req 1527808->1535999) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b427 expref: 15 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.680572:0:7654:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000092f9790d/0x1cd7fe5368733e10 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3fd expref: 15 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.680590:0:7654:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000092f9790d/0x1cd7fe5368733e10 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3fd expref: 15 pid: 6698 timeout: 501 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619908.680766:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c31f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589056256:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619908.680890:0:7654:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000d4b76022/0x1cd7fe5368733e09 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1384448->1392639) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3f6 expref: 11 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.680900:0:7654:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000d4b76022/0x1cd7fe5368733e09 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1384448->1392639) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3f6 expref: 11 pid: 6696 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619908.680910:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c31c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589056384:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619908.680947:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa540 going to sleep for 11 seconds 00000100:00100000:0.0:1788619908.687097:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c31f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589056256:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619908.687122:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c31c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589056384:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619908.687172:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000244b3d37/0x1cd7fe5368733e3a lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [1527808->1535999] (req 1527808->1535999) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b427 expref: 15 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.687215:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6f400, rc 0) 00000100:00100000:0.0:1788619908.687372:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d02680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:14908:x1875503571340032:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6945us (7325us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619908.687384:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 556 00000100:00100000:3.0:1788619908.696603:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619908.696873:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571340160 00000100:00100000:2.0:1788619908.696887:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571340160 00010000:00010000:2.0:1788619908.696976:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000092f9790d/0x1cd7fe5368733e10 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3fd expref: 15 pid: 6698 timeout: 501 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619908.697084:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 242 00000100:00100000:2.0:1788619908.697097:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d03480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571340160:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619908.697118:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619908.697163:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000092f9790d/0x1cd7fe5368733e10 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3fd expref: 15 pid: 6698 timeout: 501 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619908.697184:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000092f9790d/0x1cd7fe5368733e10 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3fd expref: 15 pid: 6698 timeout: 501 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619908.697215:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000092f9790d/0x1cd7fe5368733e10 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3fd expref: 15 pid: 6698 timeout: 501 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619908.697247:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619908.697415:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d03480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571340160:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 319us (826us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619908.697423:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 242 00000100:00100000:3.0:1788619908.700503:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619908.700697:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571340288 00000100:00100000:0.0:1788619908.700710:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571340288 00010000:00010000:0.0:1788619908.700764:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000d4b76022/0x1cd7fe5368733e09 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1384448->1392639) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3f6 expref: 12 pid: 6696 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619908.701320:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 243 00000100:00100000:0.0:1788619908.701379:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8023450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+12:2353:x1875503571340288:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619908.701399:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619908.701433:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d4b76022/0x1cd7fe5368733e09 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1384448->1392639) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3f6 expref: 12 pid: 6696 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.701452:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d4b76022/0x1cd7fe5368733e09 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1384448->1392639) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3f6 expref: 12 pid: 6696 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.701483:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d4b76022/0x1cd7fe5368733e09 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1384448->1392639) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b3f6 expref: 12 pid: 6696 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.701508:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(3 9) 00010000:00010000:0.0:1788619908.701512:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(3 9) 00010000:00010000:0.0:1788619908.701553:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000244b3d37/0x1cd7fe5368733e3a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1527808->1535999) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b427 expref: 13 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.701565:0:13330:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000244b3d37/0x1cd7fe5368733e3a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1527808->1535999) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b427 expref: 13 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.701637:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000244b3d37/0x1cd7fe5368733e3a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1527808->1535999) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b427 expref: 13 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619908.701657:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c31c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589056512:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619908.701761:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa540 going to sleep for 11 seconds 00000100:00100000:2.0:1788619908.706887:0:3317:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b34000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3317:1875503589056896:0@lo:400:kworker.0 00000100:00100000:3.0:1788619908.707078:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b34380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589057024:0@lo:400:kworker.0 00000100:00100000:1.0:1788619908.707179:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b34700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3316:1875503589056768:0@lo:400:kworker.0 00000100:00100000:0.0:1788619908.707256:0:3319:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5ff5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:3319:1875503589056640:0@lo:400:kworker.0 00000100:00100000:3.0:1788619908.707395:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619908.707467:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589057024 00000100:00100000:3.0:1788619908.707476:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589057024 00000100:00100000:3.0:1788619908.707561:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 122 00000100:00100000:3.0:1788619908.707572:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d00a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589057024:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788619908.707637:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d00a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589057024:12345-0@lo:400:kworker.0 Request processed in 66us (242us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619908.707666:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 122 00000100:00100000:3.0:1788619908.707725:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b34380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589057024:0@lo:400:kworker.0 00000100:00100000:1.0:1788619908.709604:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619908.709668:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b36a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589057280:0@lo:400:kworker.0 00000100:00100000:1.0:1788619908.709714:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619908.709968:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589056768 00000100:00100000:3.0:1788619908.709978:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589056768 00000100:00100000:3.0:1788619908.710042:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 180 00000100:00100000:3.0:1788619908.710052:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5697100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3316:x1875503589056768:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788619908.710121:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5697100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3316:x1875503589056768:12345-0@lo:400:kworker.0 Request processed in 71us (528us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619908.710140:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 180 00000100:00100000:1.0:1788619908.710891:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b34700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3316:1875503589056768:0@lo:400:kworker.0 00000100:00100000:0.0:1788619908.711169:0:3319:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788619908.711213:0:3319:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b35880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3319:1875503589057152:0@lo:400:kworker.0 00000100:00100000:0.0:1788619908.711247:0:3319:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788619908.711316:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c31c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589056512:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619908.711360:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619908.711428:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8023450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571340288:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 10055us (10939us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619908.711439:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 243 00000100:00100000:3.0:1788619908.711985:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589057152 00000100:00100000:3.0:1788619908.711995:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589057152 00000100:00100000:3.0:1788619908.712052:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 181 00000100:00100000:3.0:1788619908.712061:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c31f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3319:x1875503589057152:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788619908.712143:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c31f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3319:x1875503589057152:12345-0@lo:400:kworker.0 Request processed in 82us (895us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619908.712150:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 181 00000100:00100000:0.0:1788619908.712766:0:3319:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b35880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3319:1875503589057152:0@lo:400:kworker.0 00000100:00100000:1.0:1788619908.713233:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589056640 00000100:00100000:1.0:1788619908.713241:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589056640 00000100:00100000:1.0:1788619908.713287:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 286 00000100:00100000:1.0:1788619908.713296:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5ce2050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+19:3319:x1875503589056640:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788619908.713403:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5ce2050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+19:3319:x1875503589056640:12345-0@lo:400:kworker.0 Request processed in 108us (2234us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619908.713410:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 286 00000100:00100000:0.0:1788619908.713799:0:3319:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5ff5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:3319:1875503589056640:0@lo:400:kworker.0 00000100:00080000:3.0:1788619908.714090:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1788619908.714099:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1788619908.714145:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589057280 00000100:00100000:3.0:1788619908.714151:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589057280 00000100:00100000:3.0:1788619908.714214:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 123 00000100:00100000:3.0:1788619908.714224:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a8aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503589057280:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788619908.714286:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a8aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503589057280:12345-0@lo:400:kworker.0 Request processed in 64us (4571us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619908.714294:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 123 00000100:00080000:0.0:1788619908.714478:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1788619908.714670:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b36a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589057280:0@lo:400:kworker.0 00000100:00100000:2.0:1788619908.714902:0:3317:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619908.715254:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589056896 00000100:00100000:1.0:1788619908.715260:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589056896 00000100:00100000:1.0:1788619908.715392:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 182 00000100:00100000:1.0:1788619908.715402:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b37b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3317:x1875503589056896:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788619908.715460:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b37b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3317:x1875503589056896:12345-0@lo:400:kworker.0 Request processed in 59us (557us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619908.715488:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 182 00000100:00100000:2.0:1788619908.716409:0:3317:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b34000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3317:1875503589056896:0@lo:400:kworker.0 00000100:00080000:0.0:1788619908.717231:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788619908.718455:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619908.718918:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571340416 00000100:00100000:0.0:1788619908.719062:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571340416 00002000:00100000:0.0:1788619908.719202:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1527808->1875967) req@ffff8e77c6d01f80 x1875503571340416/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:4/0 lens 504/0 e 0 to 0 dl 1788619919 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619908.719216:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6d01f80 with x1875503571340416 ext(1527808->1875967) 00002000:00010000:0.0:1788619908.719234:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6d01f80 00000100:00100000:0.0:1788619908.719274:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 308 00000100:00100000:0.0:1788619908.719287:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d01f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2353:x1875503571340416:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619908.727680:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d01f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2353:x1875503571340416:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 8377us (9219us total) trans 0 rc 245760/245760 00000100:00100000:1.0:1788619908.727698:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 308 00002000:00100000:1.0:1788619908.727714:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1527808->1875967) req@ffff8e77c6d01f80 x1875503571340416/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:4/0 lens 504/440 e 0 to 0 dl 1788619919 ref 1 fl Complete:/600/0 rc 245760/245760 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619908.727726:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6d01f80 with x1875503571340416 ext(1527808->1875967) 00002000:00010000:1.0:1788619908.727734:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6d01f80 00000100:00100000:1.0:1788619908.740119:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619908.740870:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571340544 00000100:00100000:0.0:1788619908.740884:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571340544 00002000:00100000:0.0:1788619908.741272:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1531904->1535999) req@ffff8e77c6a88e00 x1875503571340544/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:4/0 lens 488/0 e 0 to 0 dl 1788619919 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619908.741288:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a88e00 with x1875503571340544 ext(1531904->1535999) 00002000:00010000:0.0:1788619908.741296:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a88e00 00000100:00100000:1.0:1788619908.741441:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 309 00000100:00100000:1.0:1788619908.741457:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a88e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2353:x1875503571340544:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619908.742896:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a88e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2353:x1875503571340544:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 1432us (2783us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619908.742910:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 309 00002000:00100000:1.0:1788619908.742946:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1531904->1535999) req@ffff8e77c6a88e00 x1875503571340544/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:4/0 lens 488/4536 e 0 to 0 dl 1788619919 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619908.742957:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a88e00 with x1875503571340544 ext(1531904->1535999) 00002000:00010000:1.0:1788619908.742964:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a88e00 00000100:00100000:3.0:1788619908.748454:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619908.754638:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571340672 00000100:00100000:1.0:1788619908.754657:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571340672 00000100:00100000:1.0:1788619908.754799:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 726 00000100:00100000:1.0:1788619908.754812:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d03100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571340672:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619908.754838:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619908.754881:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f48e00/0x1cd7fe5368733e41 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b42e expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619908.754970:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368733e17 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b404 expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619908.755023:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a8ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589057408:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619908.755158:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd9639c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619908.762396:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a8ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589057408:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619908.762440:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:1.0:1788619908.762467:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f48e00/0x1cd7fe5368733e41 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b42e expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619908.762497:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619908.762513:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619908.762690:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f48e00/0x1cd7fe5368733e41 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b42e expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619908.762713:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f48e00, rc 0) 00000100:00100000:1.0:1788619908.762854:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d03100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571340672:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8042us (14414us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619908.762865:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 726 00000100:00100000:2.0:1788619908.772439:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619908.772708:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571340800 00000100:00100000:1.0:1788619908.772722:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571340800 00000100:00100000:1.0:1788619908.772882:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 727 00000100:00100000:1.0:1788619908.772898:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b34000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2355:x1875503571340800:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619908.772949:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619908.772986:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f48e00/0x1cd7fe5368733e48 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b435 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619908.773033:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368733e17 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b404 expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619908.773085:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a8ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589057536:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619908.773253:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd9639c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619908.792875:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a8ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589057536:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619908.793216:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:1.0:1788619908.793246:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f48e00/0x1cd7fe5368733e48 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b435 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619908.793297:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619908.793302:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619908.793311:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f48e00/0x1cd7fe5368733e48 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b435 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619908.793361:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f48e00, rc 0) 00000100:00100000:1.0:1788619908.793619:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b34000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2355:x1875503571340800:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 20724us (21192us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619908.793630:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 727 00000100:00100000:2.0:1788619908.801177:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619908.801641:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571340928 00000100:00100000:3.0:1788619908.801656:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571340928 00000100:00100000:2.0:1788619908.801864:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 728 00000100:00100000:2.0:1788619908.801879:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b36680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+12:14908:x1875503571340928:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619908.801907:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619908.801960:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7600/0x1cd7fe5368733e4f lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b43c expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619908.802002:0:9417:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368733e17 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b404 expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619908.802014:0:9417:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7600/0x1cd7fe5368733e4f lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b43c expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619908.802059:0:9417:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368733e17 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b404 expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619908.802075:0:9417:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368733e17 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b404 expref: 10 pid: 11397 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619908.802090:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b37480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589057664:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619908.802496:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc240 going to sleep for 11 seconds 00000100:00100000:0.0:1788619908.808161:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b37480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589057664:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619908.808224:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7600/0x1cd7fe5368733e4f lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b43c expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.808249:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619908.808252:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619908.808257:0:9417:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7600/0x1cd7fe5368733e4f lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b43c expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.808271:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7600, rc 0) 00000100:00100000:0.0:1788619908.808408:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b36680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571340928:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6533us (7247us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619908.808419:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 728 00000100:00100000:2.0:1788619908.808621:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619908.809271:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571341056 00000100:00100000:1.0:1788619908.809282:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571341056 00010000:00010000:1.0:1788619908.809360:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368733e17 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b404 expref: 11 pid: 11397 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619908.809401:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 244 00000100:00100000:1.0:1788619908.809433:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b37480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571341056:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619908.809451:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619908.809486:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368733e17 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b404 expref: 11 pid: 11397 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619908.809501:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368733e17 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b404 expref: 11 pid: 11397 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619908.809896:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368733e17 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b404 expref: 11 pid: 11397 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619908.810246:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619908.810256:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7600/0x1cd7fe5368733e4f lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b43c expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619908.810270:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7600/0x1cd7fe5368733e4f lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b43c expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619908.810282:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7600/0x1cd7fe5368733e4f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b43c expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619908.810387:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7600/0x1cd7fe5368733e4f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b43c expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619908.810404:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a8ad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589057792:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619908.811500:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd9639c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619908.814646:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a8ad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589057792:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619908.814702:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619908.814781:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b37480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571341056:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 5348us (6159us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619908.814796:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 244 00000100:00100000:2.0:1788619908.820055:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619908.820180:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571341184 00000100:00100000:3.0:1788619908.820197:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571341184 00000100:00100000:3.0:1788619908.820286:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 402 00000100:00100000:3.0:1788619908.820298:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b36a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571341184:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:3.0:1788619908.820586:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1048576 -> 1048576 00000004:00010000:3.0:1788619908.820590:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619908 -> 1788619908 00000004:00010000:3.0:1788619908.820593:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619908 -> 1788619908 00000004:00010000:3.0:1788619908.820596:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619908 -> 1788619908 00000004:00010000:3.0:1788619908.820598:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1030 -> 1030 00000100:00100000:3.0:1788619908.820684:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b36a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571341184:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 388us (646us total) trans 4294967579 rc 0/0 00000100:00100000:3.0:1788619908.820692:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 402 00000100:00100000:1.0:1788619908.833734:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619908.833873:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619908.834183:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571341312 00000100:00100000:2.0:1788619908.834204:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571341312 00000100:00100000:2.0:1788619908.834289:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 729 00000100:00100000:2.0:1788619908.834302:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71fbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571341312:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619908.834405:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619908.834433:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe5368733e56 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b443 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619908.834486:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7600/0x1cd7fe5368733e4f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b43c expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619908.834564:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b35880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589057920:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619908.834675:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccb40 going to sleep for 11 seconds 00000100:00100000:3.0:1788619908.837743:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571341440 00000100:00100000:3.0:1788619908.839692:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571341440 00000100:00100000:2.0:1788619908.839981:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 557 00000100:00100000:2.0:1788619908.839996:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a8ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571341440:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619908.840021:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619908.840045:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d2409dc/0x1cd7fe5368733e5d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b44a expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619908.840152:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b34700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589058048:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619908.842242:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc180 going to sleep for 11 seconds 00000100:00100000:2.0:1788619908.843275:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b35880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589057920:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619908.843307:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:2.0:1788619908.844153:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe5368733e56 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b443 expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619908.844188:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619908.844192:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619908.844199:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe5368733e56 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b443 expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619908.844244:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7000, rc 0) 00000100:00100000:2.0:1788619908.844447:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71fbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571341312:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 10145us (10572us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619908.844457:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 729 00000100:00100000:2.0:1788619908.849681:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b34700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589058048:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619908.849734:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000d2409dc/0x1cd7fe5368733e5d lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b44a expref: 12 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619908.849764:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d2409dc/0x1cd7fe5368733e5d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b44a expref: 12 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619908.849782:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6000, rc 0) 00000100:00100000:2.0:1788619908.849888:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a8ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571341440:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 9895us (16170us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619908.849898:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 557 00000100:00100000:1.0:1788619908.861100:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619908.861164:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571341568 00000100:00100000:2.0:1788619908.861176:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571341568 00000100:00100000:2.0:1788619908.861263:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 730 00000100:00100000:2.0:1788619908.861274:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571341568:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619908.861299:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1788619908.861394:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00010000:00010000:2.0:1788619908.861496:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe5368733e64 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b451 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619908.861567:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7600/0x1cd7fe5368733e4f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b43c expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619908.861646:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b34700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589058176:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619908.861744:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc180 going to sleep for 11 seconds 00000100:00100000:0.0:1788619908.862637:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571341696 00000100:00100000:0.0:1788619908.862661:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571341696 00000100:00100000:0.0:1788619908.862724:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 558 00000100:00100000:0.0:1788619908.862734:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:2352:x1875503571341696:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619908.862784:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619908.862802:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ff7ffc7/0x1cd7fe5368733e6b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b458 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619908.862899:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c31c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589058304:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619908.863360:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaab40 going to sleep for 11 seconds 00000100:00100000:2.0:1788619908.865006:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b34700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589058176:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619908.865045:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:2.0:1788619908.865069:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe5368733e64 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b451 expref: 11 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619908.865095:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619908.865099:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619908.865106:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe5368733e64 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b451 expref: 11 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619908.865124:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7c00, rc 0) 00000100:00100000:2.0:1788619908.865255:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571341568:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3981us (4340us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619908.865266:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 730 00000100:00100000:0.0:1788619908.872086:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c31c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589058304:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619908.872142:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003ff7ffc7/0x1cd7fe5368733e6b lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b458 expref: 12 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.872174:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ff7ffc7/0x1cd7fe5368733e6b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b458 expref: 12 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619908.872190:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6e200, rc 0) 00000100:00100000:0.0:1788619908.872371:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:2352:x1875503571341696:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 9636us (10975us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619908.872382:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 558 00000100:00100000:0.0:1788619908.882028:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619908.883557:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619908.883638:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571341952 00000100:00100000:3.0:1788619908.883653:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571341952 00000100:00100000:3.0:1788619908.883735:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 559 00000100:00100000:3.0:1788619908.883748:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c685aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571341952:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619908.883790:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619908.883814:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f980cda4/0x1cd7fe5368733e72 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b466 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619908.883894:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6d00380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589058432:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619908.884022:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5c76600 going to sleep for 11 seconds 00000100:00100000:2.0:1788619908.885075:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571341824 00000100:00100000:2.0:1788619908.885088:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571341824 00000100:00100000:2.0:1788619908.885197:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 731 00000100:00100000:2.0:1788619908.885208:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c31c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571341824:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619908.885229:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619908.885253:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe5368733e79 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b45f expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619908.885299:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7600/0x1cd7fe5368733e4f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b43c expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619908.885385:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b34700 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589058560:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619908.885471:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc180 going to sleep for 11 seconds 00000100:00100000:3.0:1788619908.887760:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6d00380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589058432:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619908.887823:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f980cda4/0x1cd7fe5368733e72 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b466 expref: 12 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619908.887856:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f980cda4/0x1cd7fe5368733e72 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b466 expref: 12 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619908.887873:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d200, rc 0) 00000100:00100000:3.0:1788619908.888124:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c685aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571341952:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4377us (4575us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619908.888210:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 559 00000100:00100000:2.0:1788619908.892125:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b34700 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589058560:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619908.892164:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:2.0:1788619908.892190:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe5368733e79 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b45f expref: 11 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619908.892221:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619908.892224:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619908.892232:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe5368733e79 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b45f expref: 11 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619908.892250:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7c00, rc 0) 00000100:00100000:2.0:1788619908.892395:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c31c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571341824:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7188us (10382us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619908.892406:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 731 00010000:00010000:3.0:1788619908.900372:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788619908.904850:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619908.904917:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571342080 00000100:00100000:2.0:1788619908.905066:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571342080 00000100:00100000:2.0:1788619908.905357:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 732 00000100:00100000:2.0:1788619908.905372:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d00380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:14908:x1875503571342080:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619908.905402:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619908.905431:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe5368733e80 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b46d expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619908.905470:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7600/0x1cd7fe5368733e4f lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b43c expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619908.905503:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe5368733e80 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b46d expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619908.905560:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7600/0x1cd7fe5368733e4f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b43c expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619908.905577:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7600/0x1cd7fe5368733e4f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b43c expref: 13 pid: 9417 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619908.905606:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b34700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589058688:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619908.905722:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc180 going to sleep for 11 seconds 00000100:00100000:1.0:1788619908.910244:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619908.910290:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b34700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589058688:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619908.910370:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe5368733e80 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b46d expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619908.910392:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619908.910394:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619908.910399:0:5846:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe5368733e80 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b46d expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619908.910409:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7c00, rc 0) 00000100:00100000:2.0:1788619908.910465:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d00380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571342080:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5097us (5631us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619908.910473:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 732 00000100:00100000:3.0:1788619908.910738:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571342208 00000100:00100000:3.0:1788619908.910747:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571342208 00010000:00010000:3.0:1788619908.910825:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7600/0x1cd7fe5368733e4f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b43c expref: 14 pid: 9417 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619908.910883:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 245 00000100:00100000:3.0:1788619908.910891:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6859180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571342208:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619908.910905:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619908.910951:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7600/0x1cd7fe5368733e4f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b43c expref: 14 pid: 9417 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619908.910966:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7600/0x1cd7fe5368733e4f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b43c expref: 14 pid: 9417 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619908.911022:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7600/0x1cd7fe5368733e4f lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b43c expref: 14 pid: 9417 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619908.911041:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619908.911047:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe5368733e80 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b46d expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619908.911059:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe5368733e80 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b46d expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619908.911071:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe5368733e80 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b46d expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619908.911140:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe5368733e80 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b46d expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619908.911157:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6d01180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589058816:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619908.911236:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5c76600 going to sleep for 11 seconds 00000100:00100000:3.0:1788619908.918298:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6d01180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589058816:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619908.918368:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619908.918603:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6859180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571342208:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 7709us (8374us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619908.918617:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 245 00000100:00100000:1.0:1788619908.927480:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619908.927659:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571342336 00000100:00100000:0.0:1788619908.927674:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571342336 00002000:00100000:0.0:1788619908.927746:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6859500 x1875503571342336/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:4/0 lens 488/0 e 0 to 0 dl 1788619919 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619908.927760:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6859500 with x1875503571342336 00000100:00100000:0.0:1788619908.927804:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 403 00000100:00100000:0.0:1788619908.927816:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6859500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571342336:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619908.943774:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619908.944251:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571342464 00000100:00100000:2.0:1788619908.944287:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571342464 00000100:00100000:2.0:1788619908.944459:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 287 00000100:00100000:2.0:1788619908.944472:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5ce1c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:597450a2-e90c-408c-b401-a0ce48eed204+13:2355:x1875503571342464:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:2.0:1788619908.944573:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5ce1c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:597450a2-e90c-408c-b401-a0ce48eed204+13:2355:x1875503571342464:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 104us (826us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619908.944582:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 287 00000100:00100000:3.0:1788619908.945110:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6859500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571342336:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 17294us (17644us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1788619908.945132:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 403 00002000:00100000:3.0:1788619908.945148:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6859500 x1875503571342336/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:4/0 lens 488/440 e 0 to 0 dl 1788619919 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619908.945159:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6859500 with x1875503571342336 00000100:00100000:1.0:1788619908.970091:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619908.970322:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571342592 00000100:00100000:3.0:1788619908.970386:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571342592 00002000:00100000:3.0:1788619908.970464:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c74f4a80 x1875503571342592/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:4/0 lens 488/0 e 0 to 0 dl 1788619919 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619908.970478:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c74f4a80 with x1875503571342592 00000100:00100000:3.0:1788619908.970528:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 404 00000100:00100000:3.0:1788619908.970541:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571342592:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:3.0:1788619908.990004:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1048576 -> 1048576 00000004:00010000:3.0:1788619908.990085:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619908 -> 1788619908 00000004:00010000:3.0:1788619908.990089:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619908 -> 1788619908 00000004:00010000:3.0:1788619908.990091:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619908 -> 1788619908 00000004:00010000:3.0:1788619908.990094:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1030 -> 1030 00000100:00100000:3.0:1788619908.992861:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+13:2355:x1875503571342592:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 22305us (22769us total) trans 4294967580 rc 0/0 00000100:00100000:3.0:1788619908.992879:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 404 00002000:00100000:3.0:1788619908.992898:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c74f4a80 x1875503571342592/t4294967580(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:4/0 lens 488/448 e 0 to 0 dl 1788619919 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619908.992910:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c74f4a80 with x1875503571342592 00000100:00100000:1.0:1788619908.997670:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619908.997905:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571342720 00000100:00100000:3.0:1788619908.997919:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571342720 00000100:00100000:3.0:1788619908.998087:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 733 00000100:00100000:3.0:1788619908.998101:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571342720:12345-192.168.201.5@tcp:16:fsx.0 00000100:00100000:0.0:1788619909.051111:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571342720:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 52995us (53437us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619909.051130:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 733 00000001:00080000:1.0:1788619909.052155:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967578 is committed 00000001:00080000:1.0:1788619909.052179:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967580 is committed 00000001:00080000:1.0:1788619909.052201:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967579 is committed 00000100:00100000:1.0:1788619909.057139:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619909.057494:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571342848 00000100:00100000:2.0:1788619909.057509:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571342848 00000100:00100000:2.0:1788619909.057584:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 734 00000100:00100000:2.0:1788619909.057606:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2352:x1875503571342848:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619909.057641:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619909.057668:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe5368733e87 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b474 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619909.057718:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe5368733e80 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b46d expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619909.057782:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b34700 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589058944:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619909.058073:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc600 going to sleep for 11 seconds 00000004:00010000:2.0:1788619909.064676:0:9417:0:(mdt_lvb.c:201:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks: 1030 -> 1288 00000100:00100000:2.0:1788619909.064722:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b34700 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589058944:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619909.064756:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:2.0:1788619909.064783:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe5368733e87 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b474 expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619909.064809:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619909.064813:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619909.064972:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe5368733e87 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b474 expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619909.064993:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6000, rc 0) 00000100:00100000:2.0:1788619909.065065:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2352:x1875503571342848:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7470us (7942us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619909.065076:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 734 00000100:00100000:1.0:1788619909.071277:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619909.071432:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571342976 00000100:00100000:2.0:1788619909.071446:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571342976 00000100:00100000:2.0:1788619909.071524:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 735 00000100:00100000:2.0:1788619909.071537:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2353:x1875503571342976:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619909.071563:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619909.071587:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe5368733e8e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b47b expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619909.071962:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe5368733e80 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b46d expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619909.072019:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b34700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589059072:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619909.072423:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc600 going to sleep for 11 seconds 00000100:00100000:2.0:1788619909.085802:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b34700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589059072:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619909.085850:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:2.0:1788619909.085881:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe5368733e8e lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b47b expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619909.085927:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619909.085932:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619909.085959:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe5368733e8e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b47b expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619909.085977:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6000, rc 0) 00000100:00100000:2.0:1788619909.086070:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2353:x1875503571342976:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 14535us (14808us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619909.086081:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 735 00000100:00100000:1.0:1788619909.092300:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619909.093133:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571343104 00000100:00100000:3.0:1788619909.093149:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571343104 00000100:00100000:3.0:1788619909.093245:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 736 00000100:00100000:3.0:1788619909.093258:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2353:x1875503571343104:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619909.093289:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619909.093320:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368733e95 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b482 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.093412:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe5368733e80 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b46d expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619909.093465:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6d01180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589059200:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619909.093593:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5c76600 going to sleep for 11 seconds 00000100:00100000:0.0:1788619909.099876:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6d01180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589059200:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619909.099931:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:0.0:1788619909.099959:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368733e95 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b482 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.099986:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619909.099990:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619909.099997:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368733e95 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b482 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.100017:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c400, rc 0) 00000100:00100000:0.0:1788619909.100110:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2353:x1875503571343104:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6854us (7824us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619909.100122:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 736 00000100:00100000:2.0:1788619909.109676:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619909.109804:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571343232 00000100:00100000:0.0:1788619909.109824:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571343232 00000100:00100000:0.0:1788619909.109915:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 737 00000100:00100000:0.0:1788619909.109938:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b34700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+12:14908:x1875503571343232:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619909.109967:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619909.109995:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fa00/0x1cd7fe5368733e9c lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b489 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.110037:0:9417:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe5368733e80 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b46d expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.110071:0:9417:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fa00/0x1cd7fe5368733e9c lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b489 expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.110132:0:9417:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe5368733e80 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b46d expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.110147:0:9417:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe5368733e80 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b46d expref: 9 pid: 5846 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619909.110164:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c32680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589059328:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619909.110289:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa180 going to sleep for 11 seconds 00000100:00100000:0.0:1788619909.116164:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c32680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589059328:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619909.116236:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fa00/0x1cd7fe5368733e9c lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b489 expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.116265:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619909.116268:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619909.116273:0:9417:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fa00/0x1cd7fe5368733e9c lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b489 expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.116289:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6fa00, rc 0) 00000100:00100000:0.0:1788619909.116414:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b34700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571343232:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6477us (6754us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619909.116426:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 737 00000100:00100000:1.0:1788619909.121890:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619909.122474:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571343360 00000100:00100000:2.0:1788619909.122491:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571343360 00010000:00010000:2.0:1788619909.122708:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe5368733e80 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b46d expref: 10 pid: 5846 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619909.123065:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 246 00000100:00100000:2.0:1788619909.123081:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f6300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571343360:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619909.123733:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619909.124300:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe5368733e80 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b46d expref: 10 pid: 5846 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619909.124322:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe5368733e80 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b46d expref: 10 pid: 5846 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619909.124387:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe5368733e80 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b46d expref: 10 pid: 5846 timeout: 501 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619909.124409:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619909.124416:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fa00/0x1cd7fe5368733e9c lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b489 expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619909.124458:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fa00/0x1cd7fe5368733e9c lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b489 expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619909.124468:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fa00/0x1cd7fe5368733e9c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b489 expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619909.124714:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fa00/0x1cd7fe5368733e9c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b489 expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619909.124734:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b35880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589059456:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619909.125093:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc600 going to sleep for 11 seconds 00000100:00100000:2.0:1788619909.133238:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619909.133431:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571343488 00000100:00100000:0.0:1788619909.133462:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571343488 00002000:00100000:0.0:1788619909.133544:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b34380 x1875503571343488/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:5/0 lens 488/0 e 0 to 0 dl 1788619920 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619909.133557:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b34380 with x1875503571343488 00000100:00100000:0.0:1788619909.133741:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 405 00000100:00100000:0.0:1788619909.133764:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b34380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571343488:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619909.134199:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b34380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571343488:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 429us (968us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619909.134208:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 405 00002000:00100000:0.0:1788619909.134217:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b34380 x1875503571343488/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:5/0 lens 488/4536 e 0 to 0 dl 1788619920 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619909.134226:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b34380 with x1875503571343488 00000100:00100000:3.0:1788619909.134812:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b35880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589059456:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619909.134975:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619909.135090:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f6300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571343360:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 12013us (13278us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619909.135210:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 246 00000100:00100000:2.0:1788619909.143611:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619909.144052:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571343616 00000100:00100000:3.0:1788619909.144068:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571343616 00002000:00100000:3.0:1788619909.144274:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b35880 x1875503571343616/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:5/0 lens 488/0 e 0 to 0 dl 1788619920 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619909.144287:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b35880 with x1875503571343616 00000100:00100000:3.0:1788619909.144411:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 406 00000100:00100000:3.0:1788619909.144444:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b35880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571343616:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619909.145070:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b35880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571343616:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 623us (1475us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619909.145080:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 406 00002000:00100000:3.0:1788619909.145089:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b35880 x1875503571343616/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:5/0 lens 488/4536 e 0 to 0 dl 1788619920 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619909.150982:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b35880 with x1875503571343616 00000100:00100000:2.0:1788619909.155261:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619909.155480:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571343744 00000100:00100000:1.0:1788619909.155494:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571343744 00002000:00100000:1.0:1788619909.155695:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be4e00 x1875503571343744/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:5/0 lens 488/0 e 0 to 0 dl 1788619920 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619909.155709:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be4e00 with x1875503571343744 00000100:00100000:1.0:1788619909.155908:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 407 00000100:00100000:1.0:1788619909.155933:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571343744:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619909.159620:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571343744:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 3682us (4355us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619909.159639:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 407 00002000:00100000:1.0:1788619909.161575:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be4e00 x1875503571343744/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:5/0 lens 488/4536 e 0 to 0 dl 1788619920 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619909.161598:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be4e00 with x1875503571343744 00000100:00100000:1.0:1788619909.172003:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619909.172391:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571343872 00000100:00100000:3.0:1788619909.172406:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571343872 00002000:00100000:3.0:1788619909.172479:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c74f4380 x1875503571343872/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:5/0 lens 488/0 e 0 to 0 dl 1788619920 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619909.172502:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c74f4380 with x1875503571343872 00000100:00100000:3.0:1788619909.172550:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 408 00000100:00100000:3.0:1788619909.172563:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571343872:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619909.175090:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571343872:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 2509us (3083us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619909.175118:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 408 00002000:00100000:0.0:1788619909.175132:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c74f4380 x1875503571343872/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:5/0 lens 488/4536 e 0 to 0 dl 1788619920 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619909.175144:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c74f4380 with x1875503571343872 00000100:00100000:3.0:1788619909.190422:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619909.190709:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571344000 00000100:00100000:1.0:1788619909.190723:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571344000 00002000:00100000:1.0:1788619909.190980:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6d01180 x1875503571344000/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:5/0 lens 488/0 e 0 to 0 dl 1788619920 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619909.190994:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6d01180 with x1875503571344000 00000100:00100000:1.0:1788619909.191043:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 409 00000100:00100000:1.0:1788619909.191054:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d01180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571344000:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619909.192319:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d01180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571344000:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 1257us (1904us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619909.192420:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 409 00002000:00100000:1.0:1788619909.192431:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6d01180 x1875503571344000/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:5/0 lens 488/4536 e 0 to 0 dl 1788619920 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619909.192442:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6d01180 with x1875503571344000 00000100:00100000:1.0:1788619909.202539:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619909.202856:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571344128 00000100:00100000:0.0:1788619909.202870:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571344128 00002000:00100000:0.0:1788619909.202944:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c74f6d80 x1875503571344128/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:5/0 lens 488/0 e 0 to 0 dl 1788619920 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619909.202957:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c74f6d80 with x1875503571344128 00000100:00100000:0.0:1788619909.203000:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 410 00000100:00100000:0.0:1788619909.203017:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571344128:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619909.203417:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571344128:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 406us (890us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619909.203426:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 410 00002000:00100000:0.0:1788619909.203433:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c74f6d80 x1875503571344128/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:5/0 lens 488/4536 e 0 to 0 dl 1788619920 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619909.203441:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c74f6d80 with x1875503571344128 00000100:00100000:2.0:1788619909.214485:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619909.215691:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571344256 00000100:00100000:1.0:1788619909.215711:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571344256 00002000:00100000:1.0:1788619909.215806:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be7100 x1875503571344256/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:5/0 lens 488/0 e 0 to 0 dl 1788619920 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619909.215856:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be7100 with x1875503571344256 00000100:00100000:1.0:1788619909.215917:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 411 00000100:00100000:1.0:1788619909.215942:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571344256:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619909.216967:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571344256:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 1017us (2486us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619909.216980:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 411 00002000:00100000:1.0:1788619909.216992:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be7100 x1875503571344256/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:5/0 lens 488/4536 e 0 to 0 dl 1788619920 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619909.217002:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be7100 with x1875503571344256 00000100:00100000:2.0:1788619909.218740:0:3317:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c32680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3317:1875503589059584:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1788619909.218841:0:3317:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619909.225082:0:6700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589059584 00000100:00100000:3.0:1788619909.225112:0:6700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589059584 00000100:00100000:1.0:1788619909.225473:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 84 00000100:00100000:1.0:1788619909.225491:0:6699:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3317:x1875503589059584:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1788619909.225563:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3856662528 avail 3854565376 00000100:00100000:1.0:1788619909.225705:0:6699:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3317:x1875503589059584:12345-0@lo:13:osp-pre-1-0.0 Request processed in 220us (6864us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619909.225713:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 84 00000100:00100000:0.0:1788619909.226203:0:3317:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c32680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3317:1875503589059584:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1788619909.233763:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619909.233811:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571344384 00000100:00100000:3.0:1788619909.233830:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571344384 00002000:00100000:3.0:1788619909.233893:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6d03480 x1875503571344384/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:5/0 lens 488/0 e 0 to 0 dl 1788619920 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619909.233907:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6d03480 with x1875503571344384 00000100:00100000:3.0:1788619909.233952:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 412 00000100:00100000:3.0:1788619909.233962:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d03480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571344384:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619909.234699:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d03480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571344384:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 733us (958us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619909.234710:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 412 00002000:00100000:3.0:1788619909.234719:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6d03480 x1875503571344384/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:5/0 lens 488/4536 e 0 to 0 dl 1788619920 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619909.234728:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6d03480 with x1875503571344384 00000100:00100000:2.0:1788619909.251811:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619909.252440:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571344512 00000100:00100000:1.0:1788619909.252456:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571344512 00002000:00100000:1.0:1788619909.252531:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be4a80 x1875503571344512/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:5/0 lens 488/0 e 0 to 0 dl 1788619920 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619909.252545:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be4a80 with x1875503571344512 00000100:00100000:1.0:1788619909.252594:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 413 00000100:00100000:1.0:1788619909.252606:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571344512:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619909.253213:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571344512:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 606us (1418us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619909.253244:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 413 00002000:00100000:1.0:1788619909.253252:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be4a80 x1875503571344512/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:5/0 lens 488/4536 e 0 to 0 dl 1788619920 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619909.253262:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be4a80 with x1875503571344512 00000100:00100000:3.0:1788619909.262510:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619909.262781:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571344640 00000100:00100000:0.0:1788619909.262795:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571344640 00002000:00100000:0.0:1788619909.262872:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fe002d80 x1875503571344640/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:5/0 lens 488/0 e 0 to 0 dl 1788619920 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619909.262885:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fe002d80 with x1875503571344640 00000100:00100000:0.0:1788619909.262955:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 414 00000100:00100000:0.0:1788619909.262968:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe002d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571344640:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619909.264137:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe002d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571344640:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 1160us (1816us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619909.264153:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 414 00002000:00100000:0.0:1788619909.264164:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fe002d80 x1875503571344640/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:5/0 lens 488/4536 e 0 to 0 dl 1788619920 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619909.264174:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fe002d80 with x1875503571344640 00000100:00100000:2.0:1788619909.272448:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619909.273129:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571344768 00000100:00100000:3.0:1788619909.273144:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571344768 00002000:00100000:3.0:1788619909.273228:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df19ce00 x1875503571344768/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:5/0 lens 488/0 e 0 to 0 dl 1788619920 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619909.273241:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df19ce00 with x1875503571344768 00000100:00100000:3.0:1788619909.273291:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 415 00000100:00100000:3.0:1788619909.273302:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571344768:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:0.0:1788619909.282316:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1048576 -> 1048576 00000004:00010000:0.0:1788619909.283126:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619908 -> 1788619909 00000004:00010000:0.0:1788619909.283131:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619908 -> 1788619908 00000004:00010000:0.0:1788619909.283134:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619908 -> 1788619909 00000004:00010000:0.0:1788619909.283154:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1288 -> 1288 00000100:00100000:0.0:1788619909.283930:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2353:x1875503571344768:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 10610us (11480us total) trans 4294967581 rc 0/0 00000100:00100000:0.0:1788619909.283948:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 415 00002000:00100000:0.0:1788619909.283967:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df19ce00 x1875503571344768/t4294967581(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:5/0 lens 488/448 e 0 to 0 dl 1788619920 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619909.283978:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df19ce00 with x1875503571344768 00000100:00100000:2.0:1788619909.288572:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619909.288702:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571344896 00000100:00100000:1.0:1788619909.288716:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571344896 00000100:00100000:1.0:1788619909.288937:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 738 00000100:00100000:1.0:1788619909.288951:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2352:x1875503571344896:12345-192.168.201.5@tcp:16:fsx.0 00000100:00100000:2.0:1788619909.322619:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2352:x1875503571344896:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 33650us (34043us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619909.322638:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 738 00000001:00080000:2.0:1788619909.322691:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967581 is committed 00000100:00100000:2.0:1788619909.329722:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619909.329889:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619909.330064:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571345152 00000100:00100000:3.0:1788619909.330078:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571345152 00000100:00100000:3.0:1788619909.330157:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 560 00000100:00100000:3.0:1788619909.330169:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571345152:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619909.330204:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619909.330233:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f980cda4/0x1cd7fe5368733ea3 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b497 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619909.330311:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589059712:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619909.330464:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5c76b40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619909.331993:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571345024 00000100:00100000:0.0:1788619909.332009:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571345024 00000100:00100000:0.0:1788619909.332079:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 739 00000100:00100000:0.0:1788619909.332090:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571345024:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619909.332114:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619909.332137:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368733eaa lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b490 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.332431:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fa00/0x1cd7fe5368733e9c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b489 expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619909.332486:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c32680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589059840:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619909.332588:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa0c0 going to sleep for 11 seconds 00002000:00010000:3.0:1788619909.336965:0:6696:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime: 1788619908 -> 1788619909 00002000:00010000:3.0:1788619909.337001:0:6696:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime: 1788619908 -> 1788619909 00000100:00100000:3.0:1788619909.337037:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589059712:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619909.337090:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f980cda4/0x1cd7fe5368733ea3 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b497 expref: 12 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.337122:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f980cda4/0x1cd7fe5368733ea3 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b497 expref: 12 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.337138:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d200, rc 0) 00000100:00100000:3.0:1788619909.337236:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571345152:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7067us (7352us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619909.337246:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 560 00000004:00010000:0.0:1788619909.338037:0:5844:0:(mdt_lvb.c:201:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks: 1288 -> 1416 00000100:00100000:0.0:1788619909.338073:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c32680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589059840:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619909.338099:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:0.0:1788619909.338118:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368733eaa lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b490 expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.338161:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619909.338165:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619909.338171:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368733eaa lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b490 expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.338296:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6f000, rc 0) 00000100:00100000:0.0:1788619909.338398:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571345024:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6309us (8690us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619909.338407:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 739 00000100:00100000:2.0:1788619909.342976:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619909.343034:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619909.343209:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571345408 00000100:00100000:1.0:1788619909.343224:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571345408 00000100:00100000:1.0:1788619909.343303:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 561 00000100:00100000:1.0:1788619909.343316:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571345408:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619909.343377:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619909.343403:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001cadc580/0x1cd7fe5368733eb1 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b4a5 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619909.343486:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589059968:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619909.343592:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963840 going to sleep for 11 seconds 00000100:00100000:0.0:1788619909.343778:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571345280 00000100:00100000:0.0:1788619909.343784:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571345280 00000100:00100000:0.0:1788619909.343836:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 740 00000100:00100000:0.0:1788619909.343846:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571345280:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619909.343859:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619909.343876:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ec00/0x1cd7fe5368733eb8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b49e expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.343915:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fa00/0x1cd7fe5368733e9c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b489 expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619909.343957:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c32680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589060096:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619909.343988:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619909.349358:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589059968:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619909.349415:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001cadc580/0x1cd7fe5368733eb1 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4a5 expref: 12 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.349450:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001cadc580/0x1cd7fe5368733eb1 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4a5 expref: 12 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.349495:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f4a000, rc 0) 00000100:00100000:1.0:1788619909.349575:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571345408:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6261us (6538us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619909.349586:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 561 00000100:00100000:1.0:1788619909.351571:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c32680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589060096:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619909.351606:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:1.0:1788619909.351627:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ec00/0x1cd7fe5368733eb8 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b49e expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.351650:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619909.351654:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619909.351659:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ec00/0x1cd7fe5368733eb8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b49e expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.351675:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6ec00, rc 0) 00000100:00100000:1.0:1788619909.351848:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571345280:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8002us (8886us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619909.351857:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 740 00000100:00100000:2.0:1788619909.355117:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619909.355692:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571345664 00000100:00100000:1.0:1788619909.355707:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571345664 00000100:00100000:1.0:1788619909.356163:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 741 00000100:00100000:1.0:1788619909.356177:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571345664:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:1.0:1788619909.356265:0:5846:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619909.356303:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe5368733d22 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b316 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.356401:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619909.356408:0:5846:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788619909.356767:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4b600/0x1cd7fe5368733ebf lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.356787:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368733d3e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b32b expref: 8 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.356798:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4b600/0x1cd7fe5368733ebf lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.357268:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368733d3e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b32b expref: 8 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.357287:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368733d3e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b32b expref: 8 pid: 5844 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619909.357303:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fc000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589060224:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619909.357516:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963840 going to sleep for 11 seconds 00000100:00100000:2.0:1788619909.361457:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619909.361561:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571345792 00000100:00100000:0.0:1788619909.361574:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571345792 00010000:00010000:0.0:1788619909.361629:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368733d3e lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b32b expref: 9 pid: 5844 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619909.361706:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 247 00000100:00100000:0.0:1788619909.361719:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571345792:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619909.361754:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619909.361764:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368733d3e lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b32b expref: 9 pid: 5844 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.361779:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368733d3e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b32b expref: 9 pid: 5844 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.361807:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619909.361969:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4b600/0x1cd7fe5368733ebf lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.361983:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4b600/0x1cd7fe5368733ebf lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.361993:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4b600/0x1cd7fe5368733ebf lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.362021:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619909.362099:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571345792:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 381us (655us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619909.362107:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 247 00010000:00010000:1.0:1788619909.364015:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368733d3e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b32b expref: 8 pid: 5844 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619909.364047:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fc000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589060224:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619909.364078:0:5846:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4b600/0x1cd7fe5368733ebf lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.364089:0:5846:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4b600/0x1cd7fe5368733ebf lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.364116:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4b600/0x1cd7fe5368733ebf lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788619909.365120:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967582 00000004:00010000:1.0:1788619909.365216:0:5846:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1048576 -> 1048576 00000004:00010000:1.0:1788619909.365221:0:5846:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619909 -> 1788619909 00000004:00010000:1.0:1788619909.365224:0:5846:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619908 -> 1788619908 00000004:00010000:1.0:1788619909.365226:0:5846:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619909 -> 1788619909 00000004:00010000:1.0:1788619909.365229:0:5846:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1416 -> 1416 00000004:00010000:1.0:1788619909.365239:0:5846:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000ceb9c64d reply state 000000006a0c6a62 transno 4294967583 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4b600/0x1cd7fe5368733ebf lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619909.365249:0:5846:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788619909.365626:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:14908:x1875503571345664:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 9450us (10520us total) trans 4294967583 rc 0/0 00000100:00100000:1.0:1788619909.365637:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 741 00000020:00080000:1.0:1788619909.365644:0:5846:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1788619909.369623:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619909.369658:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619909.369731:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571346048 00000100:00100000:1.0:1788619909.369756:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571346048 00002000:00010000:1.0:1788619909.369812:0:6701:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:1.0:1788619909.369825:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7433100 with x1875503571346048 ext(0->18446744073709551615) 00002000:00010000:1.0:1788619909.369841:0:6701:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c7433100. 00000100:00100000:1.0:1788619909.369885:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 310 00000100:00100000:1.0:1788619909.369898:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7433100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571346048:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:1.0:1788619909.370383:0:6701:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1873303 -> 0 00002000:00010000:1.0:1788619909.370388:0:6701:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619909 -> 1788619909 00002000:00010000:1.0:1788619909.370406:0:6701:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619908 -> 1788619907 00002000:00010000:1.0:1788619909.370409:0:6701:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619909 -> 1788619909 00002000:00010000:1.0:1788619909.370411:0:6701:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 391 -> 391 00000100:00100000:1.0:1788619909.370491:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7433100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571346048:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 594us (829us total) trans 4294967378 rc 0/0 00000100:00100000:1.0:1788619909.370501:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 310 00002000:00010000:1.0:1788619909.370505:0:6701:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:1.0:1788619909.370509:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7433100 with x1875503571346048 ext(0->18446744073709551615) 00002000:00010000:1.0:1788619909.370514:0:6701:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c7433100. 00000100:00100000:0.0:1788619909.371094:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571345920 00000100:00100000:0.0:1788619909.371104:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571345920 00002000:00010000:0.0:1788619909.371141:0:5858:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:0.0:1788619909.371148:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c7430e00 with x1875503571345920 00000100:00100000:0.0:1788619909.371194:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 416 00000100:00100000:0.0:1788619909.371202:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7430e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571345920:12345-192.168.201.5@tcp:10:fsx.0 00000004:00010000:0.0:1788619909.371567:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1048576 -> 192875 00000004:00010000:0.0:1788619909.371573:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619909 -> 1788619909 00000004:00010000:0.0:1788619909.371576:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619908 -> 1788619908 00000004:00010000:0.0:1788619909.371578:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619909 -> 1788619909 00000004:00010000:0.0:1788619909.371580:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1416 -> 1416 00000100:00100000:0.0:1788619909.371646:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7430e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571345920:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 444us (2040us total) trans 4294967584 rc 0/0 00000100:00100000:0.0:1788619909.371654:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 416 00002000:00010000:0.0:1788619909.371657:0:5858:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:0.0:1788619909.371660:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c7430e00 with x1875503571345920 00010000:00010000:3.0:1788619909.372467:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4b600/0x1cd7fe5368733ebf lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.372483:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4b600/0x1cd7fe5368733ebf lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.372491:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4b600/0x1cd7fe5368733ebf lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.372498:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c6f4b600 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:3.0:1788619909.372503:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4b600/0x1cd7fe5368733ebf lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.372512:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4b600/0x1cd7fe5368733ebf lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.372532:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619909.372549:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788619909.372556:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4b600/0x1cd7fe5368733ebf lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.372563:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4b600/0x1cd7fe5368733ebf lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619909.375730:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619909.375980:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571346176 00000100:00100000:0.0:1788619909.375996:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571346176 00000100:00100000:0.0:1788619909.376110:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 742 00000100:00100000:0.0:1788619909.376123:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dcaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:14908:x1875503571346176:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619909.376156:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619909.376195:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368733ec6 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b4ac expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.376314:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe5368733ecd lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.376377:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe5368733ecd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.376392:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe5368733ecd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619909.376446:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 192875 00000004:00200000:0.0:1788619909.376452:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619909.376478:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe5368733ecd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.376513:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368733ec6 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4ac expref: 17 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.376536:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe5368733ecd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4ac expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.376546:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619909.376550:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619909.376554:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6ee00, rc 0) 00000100:00100000:0.0:1788619909.376702:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dcaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:14908:x1875503571346176:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 572us (977us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619909.376710:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 742 00000100:00100000:2.0:1788619909.385977:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619909.388650:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571346304 00000100:00100000:1.0:1788619909.388673:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571346304 00000100:00100000:1.0:1788619909.388904:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 743 00000100:00100000:1.0:1788619909.388929:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dc9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+8:14908:x1875503571346304:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619909.388960:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619909.388994:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a000/0x1cd7fe5368733ed4 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b4b3 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.389038:0:9417:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fa00/0x1cd7fe5368733e9c lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b489 expref: 15 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.389051:0:9417:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a000/0x1cd7fe5368733ed4 lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4b3 expref: 9 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.389097:0:9417:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fa00/0x1cd7fe5368733e9c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b489 expref: 15 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.389114:0:9417:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fa00/0x1cd7fe5368733e9c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b489 expref: 15 pid: 9417 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619909.389151:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fc000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589060352:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619909.390463:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963840 going to sleep for 11 seconds 00000100:00100000:3.0:1788619909.394557:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619909.394661:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571346432 00000100:00100000:0.0:1788619909.394674:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571346432 00010000:00010000:0.0:1788619909.394729:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fa00/0x1cd7fe5368733e9c lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b489 expref: 16 pid: 9417 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619909.394778:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 248 00000100:00100000:0.0:1788619909.394790:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c5180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571346432:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619909.394810:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619909.394858:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fa00/0x1cd7fe5368733e9c lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b489 expref: 16 pid: 9417 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.394900:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fa00/0x1cd7fe5368733e9c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b489 expref: 16 pid: 9417 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619909.395368:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fc000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589060352:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619909.396410:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a000/0x1cd7fe5368733ed4 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4b3 expref: 9 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.396438:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619909.396442:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619909.396448:0:9417:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a000/0x1cd7fe5368733ed4 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4b3 expref: 9 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.396460:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f4a000, rc 0) 00010000:00010000:0.0:1788619909.396679:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fa00/0x1cd7fe5368733e9c lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b489 expref: 16 pid: 9417 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.396701:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619909.396706:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a000/0x1cd7fe5368733ed4 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4b3 expref: 9 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.396716:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a000/0x1cd7fe5368733ed4 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4b3 expref: 9 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.396726:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a000/0x1cd7fe5368733ed4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4b3 expref: 9 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.396789:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a000/0x1cd7fe5368733ed4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4b3 expref: 9 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619909.396806:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c31880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589060480:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619909.396898:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dc9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571346304:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7981us (10935us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619909.397147:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 743 00000100:00100000:0.0:1788619909.397629:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa240 going to sleep for 11 seconds 00000100:00100000:2.0:1788619909.404536:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619909.404632:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571346560 00000100:00100000:3.0:1788619909.404645:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571346560 00002000:00100000:3.0:1788619909.404711:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5dc8380 x1875503571346560/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:5/0 lens 488/0 e 0 to 0 dl 1788619920 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619909.404723:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5dc8380 with x1875503571346560 00000100:00100000:3.0:1788619909.404779:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 417 00000100:00100000:3.0:1788619909.404789:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dc8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571346560:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619909.412959:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c31880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589060480:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619909.412998:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619909.413138:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c5180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571346432:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 18348us (18599us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619909.413177:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 248 00000100:00100000:0.0:1788619909.418591:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dc8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571346560:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 13783us (14051us total) trans 0 rc 196608/196608 00000100:00100000:0.0:1788619909.418612:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 417 00002000:00100000:0.0:1788619909.418630:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5dc8380 x1875503571346560/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:5/0 lens 488/440 e 0 to 0 dl 1788619920 ref 1 fl Complete:/600/0 rc 196608/196608 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619909.418643:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5dc8380 with x1875503571346560 00000100:00100000:3.0:1788619909.426028:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619909.426257:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571346688 00000100:00100000:1.0:1788619909.426286:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571346688 00000100:00100000:1.0:1788619909.426603:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 744 00000100:00100000:1.0:1788619909.426619:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571346688:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619909.426651:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619909.426680:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4b800/0x1cd7fe5368733edb lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b4ba expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.426732:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4b800/0x1cd7fe5368733edb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4ba expref: 16 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788619909.426774:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 192875 00010000:00010000:1.0:1788619909.427071:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4b800/0x1cd7fe5368733edb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4ba expref: 16 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.427165:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619909.427170:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619909.427175:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f4b800, rc 0) 00000100:00100000:1.0:1788619909.427631:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571346688:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 1014us (1615us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619909.427640:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 744 00000100:00100000:2.0:1788619909.439242:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619909.439668:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571346816 00000100:00100000:1.0:1788619909.439682:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571346816 00000100:00100000:1.0:1788619909.439758:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 562 00000100:00100000:1.0:1788619909.439771:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dc9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571346816:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619909.439799:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619909.439834:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000054b6e35/0x1cd7fe5368733ee2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b4c1 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.439877:0:7654:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000244b3d37/0x1cd7fe5368733e3a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1527808->1535999) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b427 expref: 14 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.439888:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=1s v=(3 9) 00010000:00010000:1.0:1788619909.439893:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(4 9) 00010000:00010000:1.0:1788619909.439900:0:7654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000054b6e35/0x1cd7fe5368733ee2 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1380352->1441791] (req 1380352->1441791) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4c1 expref: 12 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.439982:0:7654:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000244b3d37/0x1cd7fe5368733e3a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1527808->1535999) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b427 expref: 14 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.439997:0:7654:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000244b3d37/0x1cd7fe5368733e3a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1527808->1535999) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b427 expref: 14 pid: 7654 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619909.440014:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589060608:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619909.440141:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963840 going to sleep for 11 seconds 00000100:00100000:2.0:1788619909.444432:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619909.444566:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571346944 00000100:00100000:3.0:1788619909.444579:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571346944 00010000:00010000:3.0:1788619909.444633:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000244b3d37/0x1cd7fe5368733e3a lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1527808->1535999) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b427 expref: 15 pid: 7654 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619909.444686:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 249 00000100:00100000:3.0:1788619909.444697:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79008ace00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571346944:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619909.444716:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619909.444742:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000244b3d37/0x1cd7fe5368733e3a lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1527808->1535999) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b427 expref: 15 pid: 7654 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.444782:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000244b3d37/0x1cd7fe5368733e3a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1527808->1535999) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b427 expref: 15 pid: 7654 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.444928:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(4 9) 00010000:00010000:3.0:1788619909.444934:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(4 9) 00010000:00010000:3.0:1788619909.444946:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000054b6e35/0x1cd7fe5368733ee2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1380352->1441791) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4c1 expref: 12 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.444958:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000054b6e35/0x1cd7fe5368733ee2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1380352->1441791) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4c1 expref: 12 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.445007:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000054b6e35/0x1cd7fe5368733ee2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1380352->1441791) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4c1 expref: 12 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619909.445025:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c4a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589060736:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619909.445571:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5c76b40 going to sleep for 11 seconds 00010000:00010000:1.0:1788619909.448384:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000244b3d37/0x1cd7fe5368733e3a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1527808->1535999) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b427 expref: 15 pid: 7654 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619909.448423:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589060608:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619909.448468:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000054b6e35/0x1cd7fe5368733ee2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1380352->1441791) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4c1 expref: 12 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.448489:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f4a600, rc 0) 00000100:00100000:1.0:1788619909.448572:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dc9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+12:14908:x1875503571346816:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8803us (9344us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619909.448582:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 562 00000100:00100000:3.0:1788619909.467924:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c4a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589060736:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619909.467962:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619909.468042:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79008ace00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571346944:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 23345us (23629us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619909.468056:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 249 00000100:00100000:2.0:1788619909.477733:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619909.478636:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571347072 00000100:00100000:3.0:1788619909.478651:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571347072 00000100:00100000:3.0:1788619909.478733:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 563 00000100:00100000:3.0:1788619909.478766:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79008ac000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571347072:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619909.478801:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619909.478833:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f980cda4/0x1cd7fe5368733ee9 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b4c8 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619909.478933:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c4a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589060864:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619909.479051:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5c76b40 going to sleep for 11 seconds 00002000:00010000:3.0:1788619909.482730:0:6698:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size: 0 -> 1440837 00002000:00010000:3.0:1788619909.482743:0:6698:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619907 -> 1788619909 00000100:00100000:3.0:1788619909.482946:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c4a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589060864:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619909.482998:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f980cda4/0x1cd7fe5368733ee9 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4c8 expref: 15 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.483029:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f980cda4/0x1cd7fe5368733ee9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4c8 expref: 15 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.483045:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d200, rc 0) 00000100:00100000:3.0:1788619909.483178:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79008ac000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571347072:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4415us (5460us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619909.483199:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 563 00000100:00100000:2.0:1788619909.495751:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619909.503865:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571347200 00000100:00100000:3.0:1788619909.503890:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571347200 00000100:00100000:0.0:1788619909.504417:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 564 00000100:00100000:0.0:1788619909.504432:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79008ad880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571347200:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619909.504459:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619909.504485:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000112ced4b/0x1cd7fe5368733ef0 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b4cf expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619909.504566:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c31880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589060992:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619909.504705:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa240 going to sleep for 11 seconds 00000100:00100000:0.0:1788619909.511126:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c31880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589060992:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619909.511183:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000112ced4b/0x1cd7fe5368733ef0 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4cf expref: 15 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.511215:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000112ced4b/0x1cd7fe5368733ef0 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4cf expref: 15 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.511231:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6fa00, rc 0) 00000100:00100000:0.0:1788619909.511439:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79008ad880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571347200:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7008us (15702us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619909.511449:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 564 00000100:00100000:2.0:1788619909.526541:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619909.526607:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619909.526840:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571347328 00000100:00100000:1.0:1788619909.526853:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571347328 00002000:00100000:1.0:1788619909.526932:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79008af800 x1875503571347328/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:5/0 lens 488/0 e 0 to 0 dl 1788619920 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619909.526971:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79008af800 with x1875503571347328 00000100:00100000:1.0:1788619909.527021:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 418 00000100:00100000:1.0:1788619909.527034:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79008af800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571347328:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619909.529056:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571347456 00000100:00100000:3.0:1788619909.529079:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571347456 00000100:00100000:3.0:1788619909.531717:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79008af800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571347328:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 4670us (5176us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619909.531768:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 418 00002000:00100000:3.0:1788619909.531895:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79008af800 x1875503571347328/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:5/0 lens 488/440 e 0 to 0 dl 1788619920 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619909.531908:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79008af800 with x1875503571347328 00002000:00100000:0.0:1788619909.533662:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1380351) req@ffff8e79008ae300 x1875503571347456/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:5/0 lens 488/0 e 0 to 0 dl 1788619920 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619909.533683:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e79008ae300 with x1875503571347456 ext(1048576->1380351) 00002000:00010000:0.0:1788619909.533691:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e79008ae300 00000100:00100000:0.0:1788619909.533806:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 311 00000100:00100000:0.0:1788619909.533826:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79008ae300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2353:x1875503571347456:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619909.538158:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619909.540019:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79008ae300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2353:x1875503571347456:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 6183us (13397us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619909.540037:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 311 00002000:00100000:3.0:1788619909.540054:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1380351) req@ffff8e79008ae300 x1875503571347456/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:5/0 lens 488/440 e 0 to 0 dl 1788619920 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619909.540065:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e79008ae300 with x1875503571347456 ext(1048576->1380351) 00002000:00010000:3.0:1788619909.540073:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e79008ae300 00000100:00100000:0.0:1788619909.540166:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571347584 00000100:00100000:0.0:1788619909.540178:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571347584 00000100:00100000:0.0:1788619909.540242:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 745 00000100:00100000:0.0:1788619909.540251:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79008ad180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571347584:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619909.540640:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619909.540666:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fa00/0x1cd7fe5368733ef7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b4d6 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.540790:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368733efe lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.540817:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368733efe lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.540833:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368733efe lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619909.540867:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 192875 00000004:00200000:0.0:1788619909.540891:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619909.540929:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368733efe lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.540963:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fa00/0x1cd7fe5368733ef7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4d6 expref: 11 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.540983:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368733efe lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4d6 expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.540993:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619909.540997:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619909.541002:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6f000, rc 0) 00000100:00100000:0.0:1788619909.541049:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79008ad180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:14908:x1875503571347584:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 799us (2908us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619909.541056:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 745 00000100:00100000:2.0:1788619909.553170:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619909.553635:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571347712 00000100:00100000:1.0:1788619909.553649:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571347712 00000100:00100000:1.0:1788619909.553723:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 746 00000100:00100000:1.0:1788619909.553734:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79008af100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:14908:x1875503571347712:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619909.553759:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619909.553763:0:9417:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:1.0:1788619909.554106:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a000/0x1cd7fe5368733ed4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4b3 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.554131:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619909.554137:0:9417:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788619909.554155:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4aa00/0x1cd7fe5368733f05 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b4dd expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.554187:0:9417:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4b800/0x1cd7fe5368733edb lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4ba expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.554199:0:9417:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4aa00/0x1cd7fe5368733f05 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4dd expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.554240:0:9417:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4b800/0x1cd7fe5368733edb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4ba expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.554282:0:9417:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4b800/0x1cd7fe5368733edb lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4ba expref: 15 pid: 5846 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619909.554298:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fc000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589061120:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619909.557226:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963840 going to sleep for 11 seconds 00000100:00100000:2.0:1788619909.560675:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619909.560814:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fc000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589061120:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619909.560862:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4aa00/0x1cd7fe5368733f05 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4dd expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.560888:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619909.560891:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619909.560896:0:9417:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4aa00/0x1cd7fe5368733f05 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4dd expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619909.560903:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571347840 00010000:00010000:1.0:1788619909.560909:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f4aa00, rc 0) 00000100:00100000:3.0:1788619909.560924:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571347840 00010000:00010000:3.0:1788619909.560968:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4b800/0x1cd7fe5368733edb lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4ba expref: 16 pid: 5846 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619909.560997:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79008af100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:14908:x1875503571347712:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7264us (7840us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619909.561006:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 746 00000100:00100000:3.0:1788619909.561011:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 250 00000100:00100000:3.0:1788619909.561020:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8022450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571347840:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619909.561036:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619909.561071:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4b800/0x1cd7fe5368733edb lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4ba expref: 16 pid: 5846 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.561085:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4b800/0x1cd7fe5368733edb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4ba expref: 16 pid: 5846 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.561156:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4b800/0x1cd7fe5368733edb lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4ba expref: 16 pid: 5846 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.561176:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619909.561191:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4aa00/0x1cd7fe5368733f05 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4dd expref: 9 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.561202:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4aa00/0x1cd7fe5368733f05 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4dd expref: 9 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.561213:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4aa00/0x1cd7fe5368733f05 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4dd expref: 9 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.561259:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4aa00/0x1cd7fe5368733f05 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4dd expref: 9 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619909.561288:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c4a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589061248:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619909.561398:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5c76b40 going to sleep for 11 seconds 00000100:00100000:2.0:1788619909.566737:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619909.566788:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c4a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589061248:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619909.566822:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619909.566899:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8022450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571347840:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 5878us (6239us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619909.566948:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 250 00000100:00100000:0.0:1788619909.567615:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571347968 00000100:00100000:0.0:1788619909.567628:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571347968 00000100:00100000:0.0:1788619909.567712:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 419 00000100:00100000:0.0:1788619909.567723:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79008ad500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571347968:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:0.0:1788619909.568017:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 192875 -> 192875 00000004:00010000:0.0:1788619909.568022:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619909 -> 1788619909 00000004:00010000:0.0:1788619909.568025:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619908 -> 1788619908 00000004:00010000:0.0:1788619909.568028:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619909 -> 1788619909 00000004:00010000:0.0:1788619909.568030:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1416 -> 1416 00000100:00100000:0.0:1788619909.568103:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79008ad500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571347968:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 381us (1380us total) trans 4294967585 rc 0/0 00000100:00100000:0.0:1788619909.568136:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 419 00000100:00100000:2.0:1788619909.573501:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619909.573564:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619909.573674:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571348096 00000100:00100000:1.0:1788619909.573688:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571348096 00000100:00100000:1.0:1788619909.573767:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 747 00000100:00100000:1.0:1788619909.573779:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79008ace00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2353:x1875503571348096:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619909.573807:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619909.573835:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a000/0x1cd7fe5368733f0c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b4e4 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.573930:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4aa00/0x1cd7fe5368733f05 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4dd expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619909.573985:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fc000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589061376:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619909.574092:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963840 going to sleep for 11 seconds 00000100:00100000:0.0:1788619909.574411:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571348224 00000100:00100000:0.0:1788619909.574417:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571348224 00000100:00100000:0.0:1788619909.574464:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 565 00000100:00100000:0.0:1788619909.574474:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2353:x1875503571348224:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619909.574484:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619909.574499:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000112ced4b/0x1cd7fe5368733f13 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b4eb expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619909.574550:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c31880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589061504:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619909.574590:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa240 going to sleep for 11 seconds 00000100:00100000:0.0:1788619909.577842:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c31880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589061504:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619909.577896:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000112ced4b/0x1cd7fe5368733f13 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4eb expref: 15 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.577948:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000112ced4b/0x1cd7fe5368733f13 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4eb expref: 15 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.577965:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6fa00, rc 0) 00000100:00100000:0.0:1788619909.578048:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2353:x1875503571348224:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3575us (4481us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619909.578057:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 565 00000004:00010000:1.0:1788619909.579757:0:5846:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619908 -> 1788619909 00000100:00100000:1.0:1788619909.579796:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fc000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589061376:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619909.579824:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 192875 00010000:00010000:1.0:1788619909.579848:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a000/0x1cd7fe5368733f0c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4e4 expref: 16 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.579873:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619909.579876:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619909.579883:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a000/0x1cd7fe5368733f0c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4e4 expref: 16 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.579900:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f4a000, rc 0) 00000100:00100000:1.0:1788619909.580002:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79008ace00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2353:x1875503571348096:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6224us (6514us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619909.580011:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 747 00000100:00100000:2.0:1788619909.585880:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619909.586879:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619909.586978:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571348480 00000100:00100000:1.0:1788619909.586992:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571348480 00000100:00100000:1.0:1788619909.587073:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 566 00000100:00100000:1.0:1788619909.587086:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe138c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2352:x1875503571348480:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619909.587111:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619909.587153:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f5d2a141/0x1cd7fe5368733f1a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b4f9 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619909.587246:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589061632:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619909.587406:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963840 going to sleep for 11 seconds 00000100:00100000:1.0:1788619909.591390:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589061632:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619909.591444:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f5d2a141/0x1cd7fe5368733f1a lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4f9 expref: 15 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.591477:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f5d2a141/0x1cd7fe5368733f1a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4f9 expref: 15 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.591493:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f4ba00, rc 0) 00000100:00100000:1.0:1788619909.591562:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe138c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2352:x1875503571348480:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4479us (4689us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619909.591573:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 566 00000100:00100000:0.0:1788619909.594708:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571348352 00000100:00100000:0.0:1788619909.594730:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571348352 00000100:00100000:0.0:1788619909.594861:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 748 00000100:00100000:0.0:1788619909.594873:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2352:x1875503571348352:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619909.594920:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619909.594949:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fa00/0x1cd7fe5368733f21 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b4f2 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.595001:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4aa00/0x1cd7fe5368733f05 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4dd expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619909.595057:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c31880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589061760:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619909.595170:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa240 going to sleep for 11 seconds 00000100:00100000:3.0:1788619909.599760:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c31880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589061760:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619909.599800:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 192875 00010000:00010000:3.0:1788619909.599849:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fa00/0x1cd7fe5368733f21 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4f2 expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.599877:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619909.599880:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619909.599887:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fa00/0x1cd7fe5368733f21 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4f2 expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.599923:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6fa00, rc 0) 00000100:00100000:3.0:1788619909.600007:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2352:x1875503571348352:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5135us (14140us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619909.600017:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 748 00000100:00100000:3.0:1788619909.607053:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619909.609154:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619909.609667:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571348736 00000100:00100000:3.0:1788619909.609682:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571348736 00000100:00100000:3.0:1788619909.609766:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 567 00000100:00100000:3.0:1788619909.609780:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c33b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571348736:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619909.609804:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619909.609829:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f980cda4/0x1cd7fe5368733f28 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b507 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619909.609909:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c4a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589061888:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619909.610047:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5c76300 going to sleep for 11 seconds 00000100:00100000:2.0:1788619909.613713:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c4a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589061888:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619909.613780:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f980cda4/0x1cd7fe5368733f28 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b507 expref: 15 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619909.613842:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f980cda4/0x1cd7fe5368733f28 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b507 expref: 15 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619909.613859:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d200, rc 0) 00000100:00100000:2.0:1788619909.614018:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c33b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571348736:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4239us (4874us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619909.614029:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 567 00000100:00100000:1.0:1788619909.614304:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571348608 00000100:00100000:1.0:1788619909.614314:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571348608 00000100:00100000:1.0:1788619909.614403:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 749 00000100:00100000:1.0:1788619909.614414:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571348608:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619909.614436:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619909.614462:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a000/0x1cd7fe5368733f2f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b500 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.614509:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4aa00/0x1cd7fe5368733f05 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4dd expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619909.614576:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fc000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589062016:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619909.614719:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963840 going to sleep for 11 seconds 00000100:00100000:1.0:1788619909.620816:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fc000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589062016:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619909.620852:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 192875 00010000:00010000:1.0:1788619909.620878:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a000/0x1cd7fe5368733f2f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b500 expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.620976:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619909.620981:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619909.620988:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a000/0x1cd7fe5368733f2f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b500 expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.621008:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f4a000, rc 0) 00000100:00100000:1.0:1788619909.621141:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571348608:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6726us (14099us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619909.621152:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 749 00000100:00100000:0.0:1788619909.626487:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619909.626535:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619909.626603:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571348864 00000100:00100000:1.0:1788619909.626618:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571348864 00000100:00100000:1.0:1788619909.626709:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 750 00000100:00100000:1.0:1788619909.626722:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c31880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571348864:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619909.626751:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619909.626781:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4ba00/0x1cd7fe5368733f36 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b50e expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.626835:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4aa00/0x1cd7fe5368733f05 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4dd expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619909.626909:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fc000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589062144:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619909.627033:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963840 going to sleep for 11 seconds 00000100:00100000:2.0:1788619909.627480:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571348992 00000100:00100000:2.0:1788619909.627487:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571348992 00000100:00100000:2.0:1788619909.627530:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 568 00000100:00100000:2.0:1788619909.627538:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571348992:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619909.627549:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619909.627564:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d2409dc/0x1cd7fe5368733f3d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b515 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619909.627610:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f3100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589062272:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619909.627677:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc300 going to sleep for 11 seconds 00000100:00100000:1.0:1788619909.633689:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fc000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589062144:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619909.633728:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 192875 00010000:00010000:1.0:1788619909.633767:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4ba00/0x1cd7fe5368733f36 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b50e expref: 16 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.633793:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619909.633797:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619909.633804:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4ba00/0x1cd7fe5368733f36 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b50e expref: 16 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.633825:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f4ba00, rc 0) 00000100:00100000:1.0:1788619909.633903:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c31880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571348864:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7183us (7429us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619909.633925:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 750 00000100:00100000:2.0:1788619909.634201:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f3100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589062272:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619909.634232:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000d2409dc/0x1cd7fe5368733f3d lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b515 expref: 15 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619909.634250:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d2409dc/0x1cd7fe5368733f3d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b515 expref: 15 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619909.634302:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6000, rc 0) 00000100:00100000:2.0:1788619909.634399:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571348992:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6861us (7862us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619909.634407:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 568 00000100:00100000:2.0:1788619909.643595:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619909.643671:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571349248 00000100:00100000:0.0:1788619909.643682:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571349248 00000100:00100000:0.0:1788619909.643750:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 751 00000100:00100000:0.0:1788619909.643759:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:14908:x1875503571349248:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:0.0:1788619909.643832:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619909.643867:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe5368733ecd lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4ac expref: 15 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.643890:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619909.643896:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788619909.644200:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368733f44 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.644214:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368733efe lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4d6 expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.644239:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368733f44 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.644281:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368733efe lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4d6 expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.644293:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368733efe lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4d6 expref: 10 pid: 5844 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619909.644306:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5ff5c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589062400:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619909.644614:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa240 going to sleep for 11 seconds 00000100:00100000:0.0:1788619909.648471:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619909.648595:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5ff5c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589062400:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619909.648626:0:5845:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368733f44 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619909.648723:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571349376 00000100:00100000:1.0:1788619909.648731:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571349376 00010000:00010000:1.0:1788619909.648770:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368733efe lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4d6 expref: 11 pid: 5844 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619909.648844:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 251 00000100:00100000:1.0:1788619909.648857:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff6680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571349376:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619909.648877:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619909.648885:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368733efe lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4d6 expref: 11 pid: 5844 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.648940:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368733efe lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4d6 expref: 11 pid: 5844 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.648971:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368733efe lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4d6 expref: 11 pid: 5844 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.648991:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619909.648997:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368733f44 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.649009:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368733f44 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.649018:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368733f44 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.649055:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619909.649133:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff6680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571349376:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 278us (673us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619909.649141:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 251 00010000:00010000:0.0:1788619909.649198:0:5845:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368733f44 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.649206:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368733f44 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788619909.649380:0:5845:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967586 00000004:00010000:0.0:1788619909.649432:0:5845:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 192875 -> 192875 00000004:00010000:0.0:1788619909.649435:0:5845:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619909 -> 1788619909 00000004:00010000:0.0:1788619909.649437:0:5845:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619909 -> 1788619908 00000004:00010000:0.0:1788619909.649439:0:5845:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619909 -> 1788619909 00000004:00010000:0.0:1788619909.649440:0:5845:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1416 -> 1416 00000004:00010000:0.0:1788619909.649446:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000491f9ab9 reply state 00000000de7bfe7e transno 4294967587 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368733f44 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619909.649463:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788619909.649534:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+17:14908:x1875503571349248:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 5777us (5952us total) trans 4294967587 rc 0/0 00000100:00100000:0.0:1788619909.649540:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 751 00000020:00080000:0.0:1788619909.649542:0:5845:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:2.0:1788619909.653364:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368733f44 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619909.653382:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368733f44 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619909.653388:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368733f44 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619909.653392:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c6b6e000 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:2.0:1788619909.653396:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368733f44 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619909.653402:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368733f44 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619909.653420:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619909.653422:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788619909.653426:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368733f44 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619909.653430:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368733f44 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619909.653551:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619909.654410:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571349504 00000100:00100000:1.0:1788619909.654424:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571349504 00000100:00100000:1.0:1788619909.654495:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 752 00000100:00100000:1.0:1788619909.654507:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:14908:x1875503571349504:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619909.654535:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619909.654557:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4ba00/0x1cd7fe5368733f4b lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b51c expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.654606:0:9417:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4aa00/0x1cd7fe5368733f05 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4dd expref: 9 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.654614:0:9417:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4ba00/0x1cd7fe5368733f4b lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b51c expref: 17 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.654657:0:9417:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4aa00/0x1cd7fe5368733f05 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4dd expref: 9 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.654667:0:9417:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4aa00/0x1cd7fe5368733f05 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4dd expref: 9 pid: 9417 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619909.654679:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fc000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589062528:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619909.654888:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963840 going to sleep for 11 seconds 00000100:00100000:1.0:1788619909.657746:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fc000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589062528:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619909.657792:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4ba00/0x1cd7fe5368733f4b lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b51c expref: 17 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.657822:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619909.657826:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619909.657831:0:9417:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4ba00/0x1cd7fe5368733f4b lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b51c expref: 17 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.657843:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f4ba00, rc 0) 00000100:00100000:1.0:1788619909.657927:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+17:14908:x1875503571349504:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3420us (4377us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619909.657957:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 752 00000100:00100000:0.0:1788619909.659712:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619909.659788:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571349632 00000100:00100000:1.0:1788619909.659798:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571349632 00010000:00010000:1.0:1788619909.659846:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4aa00/0x1cd7fe5368733f05 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4dd expref: 10 pid: 9417 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619909.659888:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 252 00000100:00100000:1.0:1788619909.659897:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff4e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571349632:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619909.659930:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619909.659965:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4aa00/0x1cd7fe5368733f05 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4dd expref: 10 pid: 9417 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.659979:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4aa00/0x1cd7fe5368733f05 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4dd expref: 10 pid: 9417 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.660006:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4aa00/0x1cd7fe5368733f05 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4dd expref: 10 pid: 9417 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.660025:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619909.660031:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4ba00/0x1cd7fe5368733f4b lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b51c expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.660039:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4ba00/0x1cd7fe5368733f4b lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b51c expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.660046:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4ba00/0x1cd7fe5368733f4b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b51c expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.660102:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4ba00/0x1cd7fe5368733f4b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b51c expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619909.660118:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fc000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589062656:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619909.660215:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963840 going to sleep for 11 seconds 00000100:00100000:0.0:1788619909.662568:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619909.662885:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571349760 00000100:00100000:3.0:1788619909.662926:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571349760 00000100:00100000:3.0:1788619909.663001:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 569 00000100:00100000:3.0:1788619909.663013:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571349760:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619909.663036:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619909.663057:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001fc792ba/0x1cd7fe5368733f52 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b523 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.663093:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000054b6e35/0x1cd7fe5368733ee2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1380352->1441791) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4c1 expref: 11 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.663104:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(4 9) 00010000:00010000:3.0:1788619909.663108:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(5 9) 00010000:00010000:3.0:1788619909.663115:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001fc792ba/0x1cd7fe5368733f52 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b523 expref: 15 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.663161:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000054b6e35/0x1cd7fe5368733ee2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1380352->1441791) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4c1 expref: 11 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.663188:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000054b6e35/0x1cd7fe5368733ee2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1380352->1441791) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4c1 expref: 11 pid: 7654 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619909.663203:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c6d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589062784:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619909.663308:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5c76d80 going to sleep for 11 seconds 00000100:00100000:1.0:1788619909.664139:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fc000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589062656:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619909.664166:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619909.664228:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff4e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571349632:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 4332us (4527us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619909.664238:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 252 00000100:00100000:3.0:1788619909.666376:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c6d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589062784:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619909.666724:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001fc792ba/0x1cd7fe5368733f52 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b523 expref: 15 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.666779:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d800, rc 0) 00000100:00100000:3.0:1788619909.666861:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:14908:x1875503571349760:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3849us (4301us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619909.666870:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 569 00000100:00100000:3.0:1788619909.667769:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619909.668045:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571349888 00000100:00100000:0.0:1788619909.668054:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571349888 00002000:00100000:0.0:1788619909.668111:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1380352->1440836) req@ffff8e77c50c6d80 x1875503571349888/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:5/0 lens 488/0 e 0 to 0 dl 1788619920 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619909.668123:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c50c6d80 with x1875503571349888 ext(1380352->1440836) 00010000:00010000:0.0:1788619909.668133:0:6702:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000054b6e35/0x1cd7fe5368733ee2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1380352->1441791) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4c1 expref: 12 pid: 7654 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.668152:0:6702:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000054b6e35/0x1cd7fe5368733ee2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1380352->1441791) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4c1 expref: 12 pid: 7654 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788619909.668158:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff8e77c50c6d80 00000100:00100000:0.0:1788619909.668219:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 11 00000100:00100000:0.0:1788619909.668227:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571349888:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:0.0:1788619909.675769:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+14:2355:x1875503571349888:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 7526us (7990us total) trans 4294967379 rc 0/0 00000100:00100000:0.0:1788619909.675787:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 11 00002000:00100000:0.0:1788619909.675804:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1380352->1440836) req@ffff8e77c50c6d80 x1875503571349888/t4294967379(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:5/0 lens 488/448 e 0 to 0 dl 1788619920 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619909.675829:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c50c6d80 with x1875503571349888 ext(1380352->1440836) 00010000:00010000:0.0:1788619909.675842:0:6702:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000054b6e35/0x1cd7fe5368733ee2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1380352->1441791) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4c1 expref: 14 pid: 7654 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.675864:0:6702:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000054b6e35/0x1cd7fe5368733ee2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1380352->1441791) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4c1 expref: 14 pid: 7654 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788619909.675873:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff8e77c50c6d80 00000100:00100000:3.0:1788619909.681880:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619909.684624:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571350016 00000100:00100000:1.0:1788619909.684725:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571350016 00010000:00010000:2.0:1788619909.685019:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000054b6e35/0x1cd7fe5368733ee2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1380352->1441791) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4c1 expref: 14 pid: 7654 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619909.685127:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 253 00000100:00100000:2.0:1788619909.685141:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c6300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+14:2354:x1875503571350016:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619909.685162:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619909.685204:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000054b6e35/0x1cd7fe5368733ee2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1380352->1441791) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4c1 expref: 14 pid: 7654 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619909.685220:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000054b6e35/0x1cd7fe5368733ee2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1380352->1441791) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4c1 expref: 14 pid: 7654 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619909.685251:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000054b6e35/0x1cd7fe5368733ee2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1380352->1441791) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b4c1 expref: 14 pid: 7654 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619909.685273:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(5 9) 00010000:00010000:2.0:1788619909.685278:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(5 9) 00010000:00010000:2.0:1788619909.685288:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000001fc792ba/0x1cd7fe5368733f52 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b523 expref: 14 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619909.685299:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001fc792ba/0x1cd7fe5368733f52 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b523 expref: 14 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619909.685430:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000001fc792ba/0x1cd7fe5368733f52 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b523 expref: 14 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619909.685446:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f3480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589062912:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619909.685614:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc300 going to sleep for 11 seconds 00000100:00100000:1.0:1788619909.695713:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619909.695971:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571350144 00000100:00100000:0.0:1788619909.695985:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571350144 00002000:00010000:0.0:1788619909.696057:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1927109->18446744073709551615) 00002000:00100000:0.0:1788619909.696063:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e79010fc000 with x1875503571350144 ext(1927109->18446744073709551615) 00002000:00010000:0.0:1788619909.696079:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e79010fc000. 00000100:00100000:0.0:1788619909.696120:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 312 00000100:00100000:0.0:1788619909.696133:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010fc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571350144:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:0.0:1788619909.696487:0:6703:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1440837 -> 1927109 00002000:00010000:0.0:1788619909.696493:0:6703:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619909 -> 1788619909 00002000:00010000:0.0:1788619909.696496:0:6703:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619909 -> 1788619907 00002000:00010000:0.0:1788619909.696498:0:6703:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619909 -> 1788619909 00002000:00010000:0.0:1788619909.696501:0:6703:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 391 -> 391 00000100:00100000:0.0:1788619909.696590:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010fc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571350144:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 457us (895us total) trans 4294967380 rc 0/0 00000100:00100000:0.0:1788619909.696598:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 312 00002000:00010000:0.0:1788619909.696602:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1927109->18446744073709551615) 00002000:00100000:0.0:1788619909.696624:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e79010fc000 with x1875503571350144 ext(1927109->18446744073709551615) 00002000:00010000:0.0:1788619909.696629:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e79010fc000. 00000100:00100000:3.0:1788619909.697426:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f3480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589062912:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619909.697457:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619909.697502:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c6300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571350016:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 12364us (15639us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619909.697514:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 253 00000100:00100000:2.0:1788619909.705939:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619909.706095:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571350272 00000100:00100000:1.0:1788619909.706110:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571350272 00000100:00100000:3.0:1788619909.706253:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 753 00000100:00100000:3.0:1788619909.706525:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+17:14908:x1875503571350272:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619909.706555:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619909.706582:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368733f59 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b52a expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.706978:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e600/0x1cd7fe5368733f60 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.707008:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e600/0x1cd7fe5368733f60 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.707025:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e600/0x1cd7fe5368733f60 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619909.707066:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1927109 00000004:00200000:0.0:1788619909.707072:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619909.707100:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e600/0x1cd7fe5368733f60 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.707133:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368733f59 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b52a expref: 19 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.707155:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e600/0x1cd7fe5368733f60 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b52a expref: 18 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.707165:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619909.707168:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619909.707172:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6e600, rc 0) 00000100:00100000:0.0:1788619909.707272:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+18:14908:x1875503571350272:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 755us (1348us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619909.707281:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 753 00000100:00100000:2.0:1788619909.712549:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619909.712686:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571350528 00000100:00100000:0.0:1788619909.712700:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571350528 00000100:00100000:0.0:1788619909.712813:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 754 00000100:00100000:0.0:1788619909.712827:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+18:14908:x1875503571350528:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:0.0:1788619909.712903:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619909.712951:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e600/0x1cd7fe5368733f60 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b52a expref: 18 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.712973:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619909.712979:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788619909.713462:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe5368733f67 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.713482:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe5368733f67 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.713495:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe5368733f67 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788619909.713557:0:5845:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967588 00000004:00010000:0.0:1788619909.713584:0:5845:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 192875 -> 192875 00000004:00010000:0.0:1788619909.713588:0:5845:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619909 -> 1788619909 00000004:00010000:0.0:1788619909.713590:0:5845:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619908 -> 1788619908 00000004:00010000:0.0:1788619909.713593:0:5845:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619909 -> 1788619909 00000004:00010000:0.0:1788619909.713595:0:5845:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1416 -> 1416 00000004:00010000:0.0:1788619909.713603:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000009ee0ddf5 reply state 00000000a4dfd6ef transno 4294967589 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe5368733f67 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619909.713613:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788619909.713744:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+20:14908:x1875503571350528:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 919us (1208us total) trans 4294967589 rc 0/0 00000100:00100000:0.0:1788619909.713767:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 754 00000020:00080000:0.0:1788619909.713773:0:5845:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:3.0:1788619909.716818:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe5368733f67 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.716841:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe5368733f67 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.716850:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe5368733f67 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.716881:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c6b6ea00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:3.0:1788619909.716887:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe5368733f67 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.716896:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe5368733f67 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.716927:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619909.716930:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788619909.716936:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe5368733f67 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.716943:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe5368733f67 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619909.720021:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619909.720148:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619909.720529:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571350656 00000100:00100000:3.0:1788619909.720543:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571350656 00002000:00010000:3.0:1788619909.720601:0:5859:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:3.0:1788619909.720607:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e67f0e00 with x1875503571350656 00000100:00100000:3.0:1788619909.720653:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 420 00000100:00100000:3.0:1788619909.720664:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2352:x1875503571350656:12345-192.168.201.5@tcp:10:fsx.0 00000004:00010000:3.0:1788619909.720959:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 192875 -> 816620 00000004:00010000:3.0:1788619909.720962:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619909 -> 1788619909 00000004:00010000:3.0:1788619909.720965:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619908 -> 1788619908 00000004:00010000:3.0:1788619909.720968:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619909 -> 1788619909 00000004:00010000:3.0:1788619909.720970:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1416 -> 1416 00000100:00100000:3.0:1788619909.721287:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2352:x1875503571350656:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 622us (1287us total) trans 4294967590 rc 0/0 00000100:00100000:3.0:1788619909.721318:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 420 00002000:00010000:3.0:1788619909.721322:0:5859:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:3.0:1788619909.721410:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e67f0e00 with x1875503571350656 00000100:00100000:0.0:1788619909.723633:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571350784 00000100:00100000:0.0:1788619909.723654:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571350784 00002000:00010000:1.0:1788619909.723873:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:1.0:1788619909.723880:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df969880 with x1875503571350784 ext(0->18446744073709551615) 00002000:00010000:1.0:1788619909.723896:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df969880. 00000100:00100000:1.0:1788619909.724140:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 313 00000100:00100000:1.0:1788619909.724154:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df969880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2352:x1875503571350784:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:1.0:1788619909.724593:0:6702:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1927109 -> 0 00002000:00010000:1.0:1788619909.724598:0:6702:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619909 -> 1788619909 00002000:00010000:1.0:1788619909.724601:0:6702:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619907 -> 1788619907 00002000:00010000:1.0:1788619909.724603:0:6702:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619909 -> 1788619909 00002000:00010000:1.0:1788619909.724606:0:6702:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 391 -> 391 00000100:00100000:1.0:1788619909.724721:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df969880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2352:x1875503571350784:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 568us (4570us total) trans 4294967381 rc 0/0 00000100:00100000:1.0:1788619909.724730:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 313 00002000:00010000:1.0:1788619909.724811:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:1.0:1788619909.724815:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df969880 with x1875503571350784 ext(0->18446744073709551615) 00002000:00010000:1.0:1788619909.724821:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df969880. 00000100:00100000:2.0:1788619909.729546:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619909.730392:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571350912 00000100:00100000:1.0:1788619909.730408:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571350912 00000100:00100000:1.0:1788619909.730495:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 755 00000100:00100000:1.0:1788619909.730507:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df96ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571350912:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619909.730540:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619909.730569:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4ac00/0x1cd7fe5368733f6e lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b531 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.730686:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4ae00/0x1cd7fe5368733f75 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.730705:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4ae00/0x1cd7fe5368733f75 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.730719:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4ae00/0x1cd7fe5368733f75 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788619909.730767:0:9417:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 816620 00000004:00200000:1.0:1788619909.730772:0:9417:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788619909.730798:0:9417:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4ae00/0x1cd7fe5368733f75 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.730832:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4ac00/0x1cd7fe5368733f6e lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b531 expref: 11 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.730878:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4ae00/0x1cd7fe5368733f75 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b531 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.730887:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619909.730890:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788619909.730894:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f4ae00, rc 0) 00000100:00100000:1.0:1788619909.730991:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df96ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:14908:x1875503571350912:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 485us (1458us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619909.730999:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 755 00000100:00100000:2.0:1788619909.735965:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619909.736029:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619909.736222:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571351040 00000100:00100000:0.0:1788619909.736235:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571351040 00000100:00100000:0.0:1788619909.736307:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 756 00000100:00100000:0.0:1788619909.736321:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cd3c9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571351040:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619909.736382:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619909.736410:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e600/0x1cd7fe5368733f7c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b538 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.736460:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4ba00/0x1cd7fe5368733f4b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b51c expref: 19 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619909.736509:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5ff4000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589063040:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619909.736611:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619909.737901:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571351168 00000100:00100000:1.0:1788619909.738121:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571351168 00000100:00100000:1.0:1788619909.738304:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 570 00000100:00100000:1.0:1788619909.738316:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cd3c9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+13:2353:x1875503571351168:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619909.738359:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619909.738379:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd0fba3b/0x1cd7fe5368733f83 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b53f expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619909.738453:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010ffb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589063168:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619909.738554:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963840 going to sleep for 11 seconds 00000004:00010000:1.0:1788619909.740823:0:5844:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619908 -> 1788619909 00000100:00100000:1.0:1788619909.740869:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5ff4000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589063040:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619909.740902:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 816620 00010000:00010000:1.0:1788619909.740939:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e600/0x1cd7fe5368733f7c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b538 expref: 11 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.740993:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619909.740997:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619909.741017:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e600/0x1cd7fe5368733f7c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b538 expref: 11 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.741036:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6e600, rc 0) 00000100:00100000:1.0:1788619909.741110:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cd3c9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571351040:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4792us (5169us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619909.741120:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 756 00002000:00010000:1.0:1788619909.745892:0:7654:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619907 -> 1788619909 00000100:00100000:1.0:1788619909.745945:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010ffb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589063168:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619909.745998:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000cd0fba3b/0x1cd7fe5368733f83 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b53f expref: 14 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.746042:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd0fba3b/0x1cd7fe5368733f83 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b53f expref: 14 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.746061:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f4ac00, rc 0) 00000100:00100000:1.0:1788619909.746145:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cd3c9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+13:2353:x1875503571351168:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7830us (10114us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619909.746157:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 570 00000100:00100000:0.0:1788619909.749954:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619909.750570:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571351296 00000100:00100000:3.0:1788619909.750584:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571351296 00000100:00100000:3.0:1788619909.750656:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 757 00000100:00100000:3.0:1788619909.750668:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571351296:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619909.750694:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619909.750718:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368733f8a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b546 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.750775:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4ba00/0x1cd7fe5368733f4b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b51c expref: 19 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619909.750830:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c5f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589063296:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619909.750955:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5c76d80 going to sleep for 11 seconds 00000100:00100000:3.0:1788619909.753461:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619909.753844:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571351424 00000100:00100000:2.0:1788619909.753858:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571351424 00000100:00100000:2.0:1788619909.753948:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 571 00000100:00100000:2.0:1788619909.753960:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+13:2352:x1875503571351424:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619909.753984:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619909.754016:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1c87370/0x1cd7fe5368733f91 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b54d expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619909.754091:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78cd3ca300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589063424:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619909.754202:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cca80 going to sleep for 11 seconds 00000100:00100000:2.0:1788619909.759866:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78cd3ca300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589063424:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619909.759930:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b1c87370/0x1cd7fe5368733f91 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b54d expref: 14 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619909.759986:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1c87370/0x1cd7fe5368733f91 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b54d expref: 14 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619909.760073:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6200, rc 0) 00000100:00100000:2.0:1788619909.760212:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+13:2352:x1875503571351424:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6253us (6765us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619909.760222:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 571 00000100:00100000:3.0:1788619909.760476:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c5f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589063296:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619909.760495:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 816620 00010000:00010000:3.0:1788619909.760511:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368733f8a lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b546 expref: 11 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.760527:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619909.760530:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619909.760535:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368733f8a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b546 expref: 11 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.760562:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683dc00, rc 0) 00000100:00100000:3.0:1788619909.760589:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571351296:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 9923us (10649us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619909.760597:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 757 00000100:00100000:2.0:1788619909.765169:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619909.765496:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571351552 00000100:00100000:1.0:1788619909.765510:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571351552 00000100:00100000:1.0:1788619909.765627:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 758 00000100:00100000:1.0:1788619909.765666:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cd3ca300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+20:14908:x1875503571351552:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619909.765710:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619909.765753:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4ac00/0x1cd7fe5368733f98 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b554 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.765888:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a400/0x1cd7fe5368733f9f lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.765926:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a400/0x1cd7fe5368733f9f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.765942:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a400/0x1cd7fe5368733f9f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788619909.765982:0:9417:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 816620 00000004:00200000:1.0:1788619909.765988:0:9417:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788619909.766021:0:9417:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a400/0x1cd7fe5368733f9f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.766057:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4ac00/0x1cd7fe5368733f98 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b554 expref: 22 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.766076:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a400/0x1cd7fe5368733f9f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b554 expref: 21 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.766086:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619909.766089:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788619909.766094:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f4a400, rc 0) 00000100:00100000:1.0:1788619909.766185:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cd3ca300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+21:14908:x1875503571351552:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 516us (1024us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619909.766205:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 758 00000100:00100000:0.0:1788619909.771492:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619909.772115:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571351808 00000100:00100000:2.0:1788619909.772130:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571351808 00000100:00100000:3.0:1788619909.772285:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 759 00000100:00100000:3.0:1788619909.772302:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:14908:x1875503571351808:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:3.0:1788619909.772409:0:5844:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619909.772444:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4ae00/0x1cd7fe5368733f75 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b531 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.772471:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619909.772478:0:5844:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788619909.772779:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368733fa6 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.772800:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a400/0x1cd7fe5368733f9f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b554 expref: 20 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.772810:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368733fa6 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.772859:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a400/0x1cd7fe5368733f9f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b554 expref: 20 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.772877:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a400/0x1cd7fe5368733f9f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b554 expref: 20 pid: 9417 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619909.773128:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c5f80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589063552:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619909.773262:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5c76d80 going to sleep for 11 seconds 00000100:00100000:0.0:1788619909.777714:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619909.777838:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c5f80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589063552:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619909.777878:0:5844:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368733fa6 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619909.779305:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571351936 00000100:00100000:1.0:1788619909.779321:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571351936 00010000:00010000:1.0:1788619909.779393:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a400/0x1cd7fe5368733f9f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b554 expref: 21 pid: 9417 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619909.779433:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 254 00000100:00100000:1.0:1788619909.779445:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2355:x1875503571351936:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619909.779463:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619909.779471:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a400/0x1cd7fe5368733f9f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b554 expref: 21 pid: 9417 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.779487:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a400/0x1cd7fe5368733f9f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b554 expref: 21 pid: 9417 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.779516:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a400/0x1cd7fe5368733f9f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b554 expref: 21 pid: 9417 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.779536:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619909.779542:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368733fa6 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.779554:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368733fa6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.779563:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368733fa6 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.779623:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619909.779701:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2355:x1875503571351936:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 259us (2002us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619909.779709:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 254 00010000:00010000:1.0:1788619909.779754:0:5844:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368733fa6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.779763:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368733fa6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788619909.779951:0:5844:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967591 00000004:00010000:1.0:1788619909.780553:0:5844:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 816620 -> 816620 00000004:00010000:1.0:1788619909.780558:0:5844:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619909 -> 1788619909 00000004:00010000:1.0:1788619909.780561:0:5844:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619909 -> 1788619908 00000004:00010000:1.0:1788619909.780563:0:5844:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619909 -> 1788619909 00000004:00010000:1.0:1788619909.780565:0:5844:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1416 -> 1416 00000004:00010000:1.0:1788619909.780576:0:5844:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000688bfbac reply state 00000000f2846c73 transno 4294967592 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368733fa6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619909.780587:0:5844:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788619909.780694:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:14908:x1875503571351808:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 8396us (9218us total) trans 4294967592 rc 0/0 00000100:00100000:1.0:1788619909.780702:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 759 00000020:00080000:1.0:1788619909.780707:0:5844:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1788619909.784989:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619909.785048:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571352064 00000100:00100000:0.0:1788619909.785062:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571352064 00000100:00100000:0.0:1788619909.785143:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 760 00000100:00100000:0.0:1788619909.785157:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5695c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:14908:x1875503571352064:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619909.785194:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619909.785224:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368733fad lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b55b expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.785282:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4ba00/0x1cd7fe5368733f4b lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b51c expref: 19 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.785295:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368733fad lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b55b expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.785383:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4ba00/0x1cd7fe5368733f4b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b51c expref: 19 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.785401:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4ba00/0x1cd7fe5368733f4b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b51c expref: 19 pid: 9417 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619909.785417:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5696a00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589063680:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619909.785524:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa180 going to sleep for 11 seconds 00010000:00010000:3.0:1788619909.785734:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368733fa6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.785745:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368733fa6 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.785753:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368733fa6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.785758:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c683ca00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:3.0:1788619909.785764:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368733fa6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.785774:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368733fa6 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.785794:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619909.785797:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788619909.785803:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368733fa6 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.785810:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368733fa6 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619909.789831:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619909.789968:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5696a00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589063680:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619909.790025:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368733fad lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b55b expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.790050:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619909.790053:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619909.790059:0:5845:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368733fad lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b55b expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.790071:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6f000, rc 0) 00000100:00100000:3.0:1788619909.790151:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5695c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:14908:x1875503571352064:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4997us (5177us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619909.790162:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 760 00000100:00100000:1.0:1788619909.794757:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571352192 00000100:00100000:1.0:1788619909.794781:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571352192 00010000:00010000:1.0:1788619909.794855:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4ba00/0x1cd7fe5368733f4b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b51c expref: 20 pid: 9417 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619909.794953:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 255 00000100:00100000:1.0:1788619909.794965:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5694000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2355:x1875503571352192:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619909.794989:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619909.795033:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4ba00/0x1cd7fe5368733f4b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b51c expref: 20 pid: 9417 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.795049:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4ba00/0x1cd7fe5368733f4b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b51c expref: 20 pid: 9417 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.795148:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4ba00/0x1cd7fe5368733f4b lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b51c expref: 20 pid: 9417 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.795705:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619909.795716:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368733fad lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b55b expref: 11 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.795736:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368733fad lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b55b expref: 11 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.795747:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368733fad lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b55b expref: 11 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.795846:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368733fad lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b55b expref: 11 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619909.795961:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010ffb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589063808:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619909.796481:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963840 going to sleep for 11 seconds 00000100:00100000:1.0:1788619909.799730:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010ffb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589063808:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619909.799775:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619909.799851:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5694000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2355:x1875503571352192:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 4886us (10035us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619909.799863:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 255 00000100:00100000:2.0:1788619909.803236:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619909.803422:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571352320 00000100:00100000:0.0:1788619909.803435:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571352320 00000100:00100000:0.0:1788619909.803511:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 572 00000100:00100000:0.0:1788619909.803523:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c0b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+13:14908:x1875503571352320:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619909.803547:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619909.803569:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000112ced4b/0x1cd7fe5368733fb4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b562 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.803612:0:6696:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000001fc792ba/0x1cd7fe5368733f52 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b523 expref: 16 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.803647:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(5 9) 00010000:00010000:0.0:1788619909.803651:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(6 9) 00010000:00010000:0.0:1788619909.803658:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000112ced4b/0x1cd7fe5368733fb4 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b562 expref: 14 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.803704:0:6696:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000001fc792ba/0x1cd7fe5368733f52 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b523 expref: 16 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.803720:0:6696:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000001fc792ba/0x1cd7fe5368733f52 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b523 expref: 16 pid: 6697 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619909.803736:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5696a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589063936:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619909.803852:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619909.808103:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619909.808276:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571352448 00000100:00100000:1.0:1788619909.808288:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571352448 00010000:00010000:1.0:1788619909.808375:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000001fc792ba/0x1cd7fe5368733f52 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b523 expref: 17 pid: 6697 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619909.808425:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 256 00000100:00100000:1.0:1788619909.808436:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2355:x1875503571352448:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619909.808456:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619909.808485:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000001fc792ba/0x1cd7fe5368733f52 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b523 expref: 17 pid: 6697 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.808499:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000001fc792ba/0x1cd7fe5368733f52 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b523 expref: 17 pid: 6697 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.808531:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(6 9) 00010000:00010000:1.0:1788619909.808536:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(6 9) 00010000:00010000:1.0:1788619909.808547:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000112ced4b/0x1cd7fe5368733fb4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b562 expref: 14 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.808577:0:5831:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000112ced4b/0x1cd7fe5368733fb4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b562 expref: 14 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.808626:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000112ced4b/0x1cd7fe5368733fb4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b562 expref: 14 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619909.808643:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010ffb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5831:1875503589064064:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619909.808820:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963840 going to sleep for 11 seconds 00010000:00010000:2.0:1788619909.809258:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001fc792ba/0x1cd7fe5368733f52 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b523 expref: 17 pid: 6697 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619909.809281:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5696a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589063936:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619909.809313:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000112ced4b/0x1cd7fe5368733fb4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b562 expref: 14 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619909.809362:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6fa00, rc 0) 00000100:00100000:2.0:1788619909.809407:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c0b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+14:14908:x1875503571352320:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5886us (6187us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619909.809415:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 572 00000100:00100000:1.0:1788619909.812948:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010ffb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5831:1875503589064064:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619909.812986:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619909.813068:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571352448:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 4633us (4979us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619909.813083:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 256 00000100:00100000:2.0:1788619909.815830:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619909.816229:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571352576 00000100:00100000:1.0:1788619909.816242:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571352576 00002000:00010000:1.0:1788619909.816300:0:6701:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1644160->18446744073709551615) 00002000:00100000:1.0:1788619909.816306:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5c09500 with x1875503571352576 ext(1644160->18446744073709551615) 00002000:00010000:1.0:1788619909.816322:0:6701:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5c09500. 00000100:00100000:1.0:1788619909.816394:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 314 00000100:00100000:1.0:1788619909.816407:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c09500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+14:2355:x1875503571352576:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:1.0:1788619909.816721:0:6701:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 0 -> 1644160 00002000:00010000:1.0:1788619909.816726:0:6701:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619909 -> 1788619909 00002000:00010000:1.0:1788619909.816729:0:6701:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619909 -> 1788619907 00002000:00010000:1.0:1788619909.817538:0:6701:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619909 -> 1788619909 00002000:00010000:1.0:1788619909.817541:0:6701:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 391 -> 391 00000100:00100000:1.0:1788619909.817745:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c09500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+15:2355:x1875503571352576:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 1327us (1918us total) trans 4294967382 rc 0/0 00000100:00100000:1.0:1788619909.817757:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 314 00002000:00010000:1.0:1788619909.817763:0:6701:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1644160->18446744073709551615) 00002000:00100000:1.0:1788619909.817766:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5c09500 with x1875503571352576 ext(1644160->18446744073709551615) 00002000:00010000:1.0:1788619909.817774:0:6701:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5c09500. 00000100:00100000:0.0:1788619909.822262:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619909.822459:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571352704 00000100:00100000:3.0:1788619909.822474:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571352704 00000100:00100000:3.0:1788619909.822556:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 761 00000100:00100000:3.0:1788619909.822568:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:14908:x1875503571352704:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619909.822597:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619909.822627:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368733fbb lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b569 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.822753:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368733fc2 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.822774:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368733fc2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.822814:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368733fc2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788619909.822853:0:5846:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1644160 00000004:00200000:3.0:1788619909.822858:0:5846:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788619909.822883:0:5846:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368733fc2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.822931:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368733fbb lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b569 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.822954:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368733fc2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b569 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.822964:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619909.822967:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788619909.822971:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683dc00, rc 0) 00000100:00100000:3.0:1788619909.823256:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+13:14908:x1875503571352704:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 688us (1009us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619909.823265:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 761 00000100:00100000:0.0:1788619909.832031:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619909.832169:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571352832 00000100:00100000:3.0:1788619909.832196:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571352832 00000100:00100000:3.0:1788619909.832276:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 762 00000100:00100000:3.0:1788619909.832290:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+19:14908:x1875503571352832:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619909.832318:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619909.832394:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368733fc9 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b570 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.832459:0:11397:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368733fad lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b55b expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.832471:0:11397:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368733fc9 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b570 expref: 20 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.832515:0:11397:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368733fad lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b55b expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.832531:0:11397:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368733fad lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b55b expref: 12 pid: 5845 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619909.832547:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c5f80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589064192:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619909.832654:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5c76d80 going to sleep for 11 seconds 00000100:00100000:3.0:1788619909.838520:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c5f80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589064192:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619909.838575:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368733fc9 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b570 expref: 20 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.838602:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619909.838606:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619909.838612:0:11397:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368733fc9 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b570 expref: 20 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.838625:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683ca00, rc 0) 00000100:00100000:3.0:1788619909.838710:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+20:14908:x1875503571352832:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6423us (6693us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619909.838738:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 762 00000100:00100000:1.0:1788619909.845289:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619909.845706:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571352960 00000100:00100000:3.0:1788619909.845721:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571352960 00010000:00010000:3.0:1788619909.845786:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368733fad lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b55b expref: 13 pid: 5845 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619909.845845:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 257 00000100:00100000:3.0:1788619909.845857:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8020850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+13:2355:x1875503571352960:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619909.845876:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619909.846560:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368733fad lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b55b expref: 13 pid: 5845 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.846578:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368733fad lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b55b expref: 13 pid: 5845 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.846605:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368733fad lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b55b expref: 13 pid: 5845 timeout: 502 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.846628:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619909.846635:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368733fc9 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b570 expref: 19 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.846647:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368733fc9 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b570 expref: 19 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.846657:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368733fc9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b570 expref: 19 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.846727:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368733fc9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b570 expref: 19 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619909.846763:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c5f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589064320:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619909.846879:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5c76d80 going to sleep for 11 seconds 00000100:00100000:0.0:1788619909.857890:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619909.859317:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c5f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589064320:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619909.859413:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619909.859503:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8020850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571352960:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 13646us (14227us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619909.859518:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 257 00000100:00100000:1.0:1788619909.859696:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571353088 00000100:00100000:1.0:1788619909.859705:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571353088 00000100:00100000:1.0:1788619909.859809:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 421 00000100:00100000:1.0:1788619909.859820:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2354:x1875503571353088:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:1.0:1788619909.860268:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 816620 -> 816620 00000004:00010000:1.0:1788619909.860273:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619909 -> 1788619909 00000004:00010000:1.0:1788619909.860276:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619908 -> 1788619908 00000004:00010000:1.0:1788619909.860278:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619909 -> 1788619909 00000004:00010000:1.0:1788619909.860280:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1416 -> 1416 00000100:00100000:1.0:1788619909.860360:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2354:x1875503571353088:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 541us (2484us total) trans 4294967593 rc 0/0 00000100:00100000:1.0:1788619909.860368:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 421 00000100:00100000:0.0:1788619909.868070:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619909.868467:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571353216 00000100:00100000:1.0:1788619909.868481:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571353216 00000100:00100000:1.0:1788619909.868559:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 763 00000100:00100000:1.0:1788619909.868570:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+21:14908:x1875503571353216:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619909.868600:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619909.868646:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a400/0x1cd7fe5368733fd0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b577 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.868769:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4ac00/0x1cd7fe5368733fd7 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.868788:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4ac00/0x1cd7fe5368733fd7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.868811:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4ac00/0x1cd7fe5368733fd7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788619909.868846:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1644160 00000004:00200000:1.0:1788619909.868852:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788619909.868874:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4ac00/0x1cd7fe5368733fd7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.868919:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a400/0x1cd7fe5368733fd0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b577 expref: 23 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.868947:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4ac00/0x1cd7fe5368733fd7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b577 expref: 22 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.868955:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619909.868958:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788619909.868963:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f4ac00, rc 0) 00000100:00100000:1.0:1788619909.869057:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+22:14908:x1875503571353216:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 488us (1002us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619909.869065:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 763 00000100:00100000:0.0:1788619909.875464:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619909.875569:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571353344 00000100:00100000:3.0:1788619909.875583:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571353344 00000100:00100000:3.0:1788619909.875664:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 573 00000100:00100000:3.0:1788619909.875678:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571353344:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619909.875704:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619909.875729:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001391204f/0x1cd7fe5368733fde lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b57e expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619909.875838:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589064448:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619909.875989:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5c76e40 going to sleep for 11 seconds 00002000:00010000:3.0:1788619909.879664:0:6697:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619907 -> 1788619909 00000100:00100000:3.0:1788619909.879711:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589064448:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619909.879771:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001391204f/0x1cd7fe5368733fde lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b57e expref: 17 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.879925:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001391204f/0x1cd7fe5368733fde lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b57e expref: 17 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619909.879943:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683da00, rc 0) 00000100:00100000:3.0:1788619909.880090:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571353344:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4414us (4641us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619909.880100:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 573 00000100:00100000:0.0:1788619909.886119:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619909.886267:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571353472 00000100:00100000:1.0:1788619909.886298:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571353472 00000100:00100000:1.0:1788619909.886403:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 574 00000100:00100000:1.0:1788619909.886415:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2352:x1875503571353472:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619909.886438:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619909.886462:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d124dcf7/0x1cd7fe5368733fe5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b585 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619909.886539:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010ffb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589064576:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619909.886636:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963840 going to sleep for 11 seconds 00000100:00100000:0.0:1788619909.890650:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010ffb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589064576:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619909.890702:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d124dcf7/0x1cd7fe5368733fe5 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b585 expref: 17 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.890763:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d124dcf7/0x1cd7fe5368733fe5 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b585 expref: 17 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.890781:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f4a200, rc 0) 00000100:00100000:0.0:1788619909.891053:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2352:x1875503571353472:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4638us (4949us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619909.891064:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 574 00000100:00100000:0.0:1788619909.898489:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619909.898688:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571353600 00000100:00100000:2.0:1788619909.898701:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571353600 00000100:00100000:2.0:1788619909.898834:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 575 00000100:00100000:2.0:1788619909.898847:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6531c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2353:x1875503571353600:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619909.898873:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619909.898899:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3b293bd/0x1cd7fe5368733fec lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b58c expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619909.898993:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c0ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589064704:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619909.899106:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc780 going to sleep for 11 seconds 00000100:00100000:0.0:1788619909.904271:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c0ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589064704:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619909.904426:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f3b293bd/0x1cd7fe5368733fec lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b58c expref: 17 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.904461:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3b293bd/0x1cd7fe5368733fec lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b58c expref: 17 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.904480:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7e00, rc 0) 00000100:00100000:1.0:1788619909.904676:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6531c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2353:x1875503571353600:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5828us (6200us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619909.904688:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 575 00000100:00100000:1.0:1788619909.914492:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619909.914596:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571353728 00000100:00100000:2.0:1788619909.914610:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571353728 00000100:00100000:2.0:1788619909.914692:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 576 00000100:00100000:2.0:1788619909.914706:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010ff100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2353:x1875503571353728:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619909.914731:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619909.914754:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5ec8983/0x1cd7fe5368733ff3 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b593 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619909.914951:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c0ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589064832:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619909.915101:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619909.921273:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c0ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589064832:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619909.921497:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a5ec8983/0x1cd7fe5368733ff3 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b593 expref: 17 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619909.921558:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5ec8983/0x1cd7fe5368733ff3 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b593 expref: 17 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619909.921576:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6600, rc 0) 00000100:00100000:2.0:1788619909.921724:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010ff100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2353:x1875503571353728:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7019us (7245us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619909.921734:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 576 00010000:00010000:3.0:1788619909.922736:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788619909.927018:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619909.927153:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571353856 00000100:00100000:3.0:1788619909.927167:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571353856 00002000:00100000:3.0:1788619909.927243:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6532680 x1875503571353856/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:5/0 lens 488/0 e 0 to 0 dl 1788619920 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619909.927265:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6532680 with x1875503571353856 00000100:00100000:3.0:1788619909.927364:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 422 00000100:00100000:3.0:1788619909.927375:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6532680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2353:x1875503571353856:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619909.927747:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6532680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2353:x1875503571353856:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 372us (752us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619909.927757:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 422 00002000:00100000:3.0:1788619909.927765:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6532680 x1875503571353856/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:5/0 lens 488/440 e 0 to 0 dl 1788619920 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619909.927773:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6532680 with x1875503571353856 00000100:00100000:1.0:1788619909.934461:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619909.935104:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571353984 00000100:00100000:3.0:1788619909.935118:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571353984 00000100:00100000:3.0:1788619909.935193:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 577 00000100:00100000:3.0:1788619909.935205:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010ff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2352:x1875503571353984:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619909.935229:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619909.935538:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001391204f/0x1cd7fe5368733ffa lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b59a expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619909.937196:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c0ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589064960:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619909.938233:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619909.942635:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c0ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589064960:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619909.942691:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001391204f/0x1cd7fe5368733ffa lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b59a expref: 17 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.942724:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001391204f/0x1cd7fe5368733ffa lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b59a expref: 17 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.942957:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683da00, rc 0) 00000100:00100000:0.0:1788619909.943161:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010ff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2352:x1875503571353984:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7956us (8710us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619909.943172:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 577 00000100:00100000:0.0:1788619909.948711:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619909.948994:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571354112 00000100:00100000:2.0:1788619909.949066:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571354112 00002000:00100000:2.0:1788619909.949139:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c501aa00 x1875503571354112/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:5/0 lens 488/0 e 0 to 0 dl 1788619920 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619909.949152:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c501aa00 with x1875503571354112 00000100:00100000:2.0:1788619909.949234:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 423 00000100:00100000:2.0:1788619909.949246:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2352:x1875503571354112:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619909.949641:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2352:x1875503571354112:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 395us (944us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619909.949650:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 423 00002000:00100000:2.0:1788619909.949657:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c501aa00 x1875503571354112/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:5/0 lens 488/440 e 0 to 0 dl 1788619920 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619909.949665:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c501aa00 with x1875503571354112 00000100:00100000:1.0:1788619909.952572:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619909.952742:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571354240 00000100:00100000:3.0:1788619909.952778:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571354240 00000100:00100000:3.0:1788619909.952858:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 578 00000100:00100000:3.0:1788619909.952871:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010ff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571354240:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619909.952897:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619909.952930:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003b84bbc9/0x1cd7fe5368734001 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b5a1 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619909.953012:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589065088:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619909.953256:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5c76e40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619909.958674:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589065088:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619909.958724:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003b84bbc9/0x1cd7fe5368734001 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b5a1 expref: 17 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.959065:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b84bbc9/0x1cd7fe5368734001 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b5a1 expref: 17 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619909.959086:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683de00, rc 0) 00000100:00100000:0.0:1788619909.959232:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010ff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571354240:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6362us (6673us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619909.959242:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 578 00000100:00100000:0.0:1788619909.963577:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619909.964113:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571354368 00000100:00100000:3.0:1788619909.964127:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571354368 00002000:00100000:3.0:1788619909.964260:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c501a300 x1875503571354368/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:5/0 lens 488/0 e 0 to 0 dl 1788619920 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619909.964272:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c501a300 with x1875503571354368 00000100:00100000:3.0:1788619909.964385:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 424 00000100:00100000:3.0:1788619909.964396:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2354:x1875503571354368:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619909.964754:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2354:x1875503571354368:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 358us (1190us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619909.964762:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 424 00002000:00100000:3.0:1788619909.964770:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c501a300 x1875503571354368/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:5/0 lens 488/440 e 0 to 0 dl 1788619920 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619909.964789:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c501a300 with x1875503571354368 00000100:00100000:1.0:1788619909.974312:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619909.974469:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571354496 00000100:00100000:2.0:1788619909.974496:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571354496 00000100:00100000:2.0:1788619909.974595:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 579 00000100:00100000:2.0:1788619909.974604:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010fce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571354496:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619909.974626:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619909.974647:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000073be6ea/0x1cd7fe5368734008 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b5a8 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619909.974717:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c0ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589065216:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619909.974834:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccc00 going to sleep for 11 seconds 00000100:00100000:1.0:1788619909.988389:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c0ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589065216:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619909.988443:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000073be6ea/0x1cd7fe5368734008 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b5a8 expref: 17 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.988475:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000073be6ea/0x1cd7fe5368734008 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b5a8 expref: 17 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619909.988491:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7c00, rc 0) 00000100:00100000:1.0:1788619909.988655:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010fce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571354496:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 14046us (14352us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619909.988687:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 579 00000100:00100000:3.0:1788619909.996267:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619909.996457:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571354624 00000100:00100000:2.0:1788619909.996472:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571354624 00002000:00100000:2.0:1788619909.996543:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c50c4000 x1875503571354624/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:5/0 lens 488/0 e 0 to 0 dl 1788619920 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619909.996556:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c50c4000 with x1875503571354624 00000100:00100000:2.0:1788619909.996664:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 425 00000100:00100000:2.0:1788619909.996677:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2354:x1875503571354624:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619909.997187:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2354:x1875503571354624:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 510us (933us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619909.997196:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 425 00002000:00100000:2.0:1788619909.997204:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c50c4000 x1875503571354624/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:5/0 lens 488/440 e 0 to 0 dl 1788619920 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619909.997214:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c50c4000 with x1875503571354624 00000100:00100000:3.0:1788619910.001290:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619910.001711:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571354752 00000100:00100000:0.0:1788619910.001724:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571354752 00000100:00100000:0.0:1788619910.001833:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 580 00000100:00100000:0.0:1788619910.001847:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2352:x1875503571354752:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619910.001872:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619910.001896:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d058bad0/0x1cd7fe536873400f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b5af expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619910.002009:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5019180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589065344:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619910.002104:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00080000:3.0:1788619910.002668:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1788619910.004458:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5019180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589065344:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619910.004499:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d058bad0/0x1cd7fe536873400f lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b5af expref: 17 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.004525:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d058bad0/0x1cd7fe536873400f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b5af expref: 17 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.004537:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6e800, rc 0) 00000100:00100000:0.0:1788619910.004597:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2352:x1875503571354752:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2754us (3324us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619910.004605:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 580 00000100:00100000:2.0:1788619910.010119:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619910.010259:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571354880 00000100:00100000:1.0:1788619910.010273:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571354880 00002000:00100000:1.0:1788619910.010466:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5c0ad80 x1875503571354880/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:6/0 lens 488/0 e 0 to 0 dl 1788619921 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619910.010479:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5c0ad80 with x1875503571354880 00000100:00100000:1.0:1788619910.010523:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 426 00000100:00100000:1.0:1788619910.010534:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c0ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2354:x1875503571354880:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619910.010943:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c0ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2354:x1875503571354880:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 408us (838us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619910.010951:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 426 00002000:00100000:1.0:1788619910.010959:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5c0ad80 x1875503571354880/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:6/0 lens 488/440 e 0 to 0 dl 1788619921 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619910.010967:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5c0ad80 with x1875503571354880 00000100:00080000:0.0:1788619910.012974:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788619910.021456:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619910.021613:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571355008 00000100:00100000:0.0:1788619910.021628:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571355008 00000100:00100000:0.0:1788619910.021767:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 581 00000100:00100000:0.0:1788619910.021780:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571355008:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619910.021814:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619910.021838:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d058bad0/0x1cd7fe5368734016 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b5b6 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619910.021936:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5019180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589065472:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619910.022077:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:0.0:1788619910.026986:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5019180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589065472:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619910.027051:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d058bad0/0x1cd7fe5368734016 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b5b6 expref: 17 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.027084:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d058bad0/0x1cd7fe5368734016 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b5b6 expref: 17 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.027112:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6e800, rc 0) 00000100:00100000:0.0:1788619910.027202:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571355008:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5424us (5756us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619910.027213:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 581 00000100:00100000:2.0:1788619910.030678:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619910.030899:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571355136 00000100:00100000:1.0:1788619910.030925:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571355136 00002000:00100000:1.0:1788619910.030994:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5ff6680 x1875503571355136/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:6/0 lens 488/0 e 0 to 0 dl 1788619921 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619910.031007:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5ff6680 with x1875503571355136 00000100:00100000:1.0:1788619910.031116:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 427 00000100:00100000:1.0:1788619910.031256:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2355:x1875503571355136:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619910.034261:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2355:x1875503571355136:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 2848us (3440us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619910.034283:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 427 00002000:00100000:1.0:1788619910.034319:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5ff6680 x1875503571355136/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:6/0 lens 488/440 e 0 to 0 dl 1788619921 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619910.034501:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5ff6680 with x1875503571355136 00000100:00100000:3.0:1788619910.040472:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619910.040625:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571355264 00000100:00100000:1.0:1788619910.040638:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571355264 00000100:00100000:1.0:1788619910.040702:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 582 00000100:00100000:1.0:1788619910.040714:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571355264:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619910.040858:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619910.040886:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002119e2f0/0x1cd7fe536873401d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b5bd expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619910.040979:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fe300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589065600:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619910.041536:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963f00 going to sleep for 11 seconds 00000100:00100000:1.0:1788619910.044921:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fe300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589065600:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619910.044974:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002119e2f0/0x1cd7fe536873401d lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b5bd expref: 17 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619910.045023:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002119e2f0/0x1cd7fe536873401d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b5bd expref: 17 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619910.045041:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f4b400, rc 0) 00000100:00100000:1.0:1788619910.045126:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571355264:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4412us (4668us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619910.045137:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 582 00000100:00100000:2.0:1788619910.050440:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619910.056098:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571355392 00000100:00100000:0.0:1788619910.056133:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571355392 00002000:00100000:0.0:1788619910.056226:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5ff4e00 x1875503571355392/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:6/0 lens 488/0 e 0 to 0 dl 1788619921 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619910.056240:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5ff4e00 with x1875503571355392 00000100:00100000:0.0:1788619910.056298:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 428 00000100:00100000:0.0:1788619910.056310:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2355:x1875503571355392:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619910.056719:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2355:x1875503571355392:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 409us (6288us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619910.056728:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 428 00002000:00100000:0.0:1788619910.056735:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5ff4e00 x1875503571355392/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:6/0 lens 488/440 e 0 to 0 dl 1788619921 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619910.056756:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5ff4e00 with x1875503571355392 00000100:00100000:3.0:1788619910.063374:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619910.064194:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571355520 00000100:00100000:1.0:1788619910.064229:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571355520 00000100:00100000:1.0:1788619910.064321:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 583 00000100:00100000:1.0:1788619910.064391:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571355520:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619910.064416:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619910.064440:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002119e2f0/0x1cd7fe5368734024 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b5c4 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619910.064520:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fe300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589065728:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619910.064633:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963f00 going to sleep for 11 seconds 00000100:00100000:1.0:1788619910.072406:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fe300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589065728:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619910.072459:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002119e2f0/0x1cd7fe5368734024 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b5c4 expref: 17 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619910.072490:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002119e2f0/0x1cd7fe5368734024 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b5c4 expref: 17 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619910.072508:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f4b400, rc 0) 00000100:00100000:1.0:1788619910.072596:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571355520:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8230us (9285us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619910.072606:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 583 00000100:00100000:2.0:1788619910.090087:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619910.090548:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571355648 00000100:00100000:1.0:1788619910.090562:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571355648 00002000:00100000:1.0:1788619910.090627:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5ff5c00 x1875503571355648/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:6/0 lens 488/0 e 0 to 0 dl 1788619921 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619910.090640:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5ff5c00 with x1875503571355648 00000100:00100000:1.0:1788619910.090698:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 429 00000100:00100000:1.0:1788619910.090709:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2355:x1875503571355648:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619910.092534:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2355:x1875503571355648:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 1815us (2451us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619910.092545:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 429 00002000:00100000:1.0:1788619910.092557:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5ff5c00 x1875503571355648/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:6/0 lens 488/440 e 0 to 0 dl 1788619921 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619910.092566:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5ff5c00 with x1875503571355648 00000100:00100000:3.0:1788619910.097632:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619910.097945:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571355776 00000100:00100000:0.0:1788619910.097959:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571355776 00000100:00100000:0.0:1788619910.098060:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 584 00000100:00100000:0.0:1788619910.098072:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571355776:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619910.098095:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619910.098131:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d058bad0/0x1cd7fe536873402b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b5cb expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619910.098216:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5019180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589065856:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619910.098377:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:0.0:1788619910.108600:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5019180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589065856:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619910.108659:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d058bad0/0x1cd7fe536873402b lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b5cb expref: 17 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.108692:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d058bad0/0x1cd7fe536873402b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b5cb expref: 17 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.108707:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6e800, rc 0) 00000100:00100000:0.0:1788619910.108950:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571355776:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 10878us (11331us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619910.108962:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 584 00000100:00100000:2.0:1788619910.124491:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619910.124596:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571355904 00000100:00100000:0.0:1788619910.124608:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571355904 00002000:00100000:0.0:1788619910.124684:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900d5f800 x1875503571355904/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:6/0 lens 488/0 e 0 to 0 dl 1788619921 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619910.124694:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900d5f800 with x1875503571355904 00000100:00100000:0.0:1788619910.124733:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 430 00000100:00100000:0.0:1788619910.124744:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2354:x1875503571355904:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619910.125111:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2354:x1875503571355904:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 366us (633us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619910.125121:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 430 00002000:00100000:0.0:1788619910.125129:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900d5f800 x1875503571355904/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:6/0 lens 488/440 e 0 to 0 dl 1788619921 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619910.125137:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900d5f800 with x1875503571355904 00000100:00100000:3.0:1788619910.132709:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619910.133022:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571356032 00000100:00100000:0.0:1788619910.133037:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571356032 00000100:00100000:0.0:1788619910.133122:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 585 00000100:00100000:0.0:1788619910.133135:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571356032:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619910.133160:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619910.133195:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d058bad0/0x1cd7fe5368734032 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b5d2 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619910.133399:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5019180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589065984:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619910.133489:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:0.0:1788619910.137638:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5019180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589065984:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619910.137694:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d058bad0/0x1cd7fe5368734032 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b5d2 expref: 17 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.137756:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d058bad0/0x1cd7fe5368734032 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b5d2 expref: 17 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.137773:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6e800, rc 0) 00000100:00100000:0.0:1788619910.138176:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571356032:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5033us (5473us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619910.138188:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 585 00000100:00100000:2.0:1788619910.172965:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619910.173191:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571356160 00000100:00100000:0.0:1788619910.173206:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571356160 00002000:00100000:0.0:1788619910.173278:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5694000 x1875503571356160/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:6/0 lens 488/0 e 0 to 0 dl 1788619921 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619910.173292:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5694000 with x1875503571356160 00000100:00100000:0.0:1788619910.173372:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 431 00000100:00100000:0.0:1788619910.173384:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5694000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2353:x1875503571356160:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619910.173738:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5694000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2353:x1875503571356160:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 355us (797us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619910.173747:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 431 00002000:00100000:0.0:1788619910.173754:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5694000 x1875503571356160/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:6/0 lens 488/440 e 0 to 0 dl 1788619921 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619910.173763:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5694000 with x1875503571356160 00000100:00100000:2.0:1788619910.189515:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619910.189640:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571356288 00000100:00100000:1.0:1788619910.189654:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571356288 00000100:00100000:1.0:1788619910.189731:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 586 00000100:00100000:1.0:1788619910.189743:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5696a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571356288:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619910.189768:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619910.189792:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002119e2f0/0x1cd7fe5368734039 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b5d9 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619910.189876:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fe300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589066112:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619910.190035:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963f00 going to sleep for 11 seconds 00000100:00100000:1.0:1788619910.200763:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fe300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589066112:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619910.200819:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002119e2f0/0x1cd7fe5368734039 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b5d9 expref: 17 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619910.200851:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002119e2f0/0x1cd7fe5368734039 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b5d9 expref: 17 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619910.200895:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f4b400, rc 0) 00000100:00100000:1.0:1788619910.200984:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5696a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571356288:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 11243us (11482us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619910.203634:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 586 00000100:00100000:2.0:1788619910.208687:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619910.208986:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571356416 00000100:00100000:3.0:1788619910.209008:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571356416 00002000:00100000:3.0:1788619910.209083:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e67f7100 x1875503571356416/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:6/0 lens 488/0 e 0 to 0 dl 1788619921 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619910.209098:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e67f7100 with x1875503571356416 00000100:00100000:3.0:1788619910.209145:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 432 00000100:00100000:3.0:1788619910.209157:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2352:x1875503571356416:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619910.209884:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2352:x1875503571356416:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 724us (1206us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619910.209894:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 432 00002000:00100000:3.0:1788619910.209913:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e67f7100 x1875503571356416/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:6/0 lens 488/440 e 0 to 0 dl 1788619921 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619910.209923:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e67f7100 with x1875503571356416 00000100:00100000:2.0:1788619910.214053:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619910.214370:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571356544 00000100:00100000:1.0:1788619910.214384:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571356544 00000100:00100000:1.0:1788619910.214450:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 587 00000100:00100000:1.0:1788619910.214462:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5770700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2353:x1875503571356544:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619910.214485:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619910.214509:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002119e2f0/0x1cd7fe5368734040 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b5e0 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619910.214587:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fe300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589066240:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619910.214703:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963f00 going to sleep for 11 seconds 00000100:00100000:1.0:1788619910.218673:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fe300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589066240:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619910.218725:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002119e2f0/0x1cd7fe5368734040 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b5e0 expref: 17 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619910.218772:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002119e2f0/0x1cd7fe5368734040 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b5e0 expref: 17 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619910.218791:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f4b400, rc 0) 00000100:00100000:1.0:1788619910.218874:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5770700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2353:x1875503571356544:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4413us (4842us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619910.218915:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 587 00000100:00100000:2.0:1788619910.226396:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619910.226491:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571356672 00000100:00100000:0.0:1788619910.226505:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571356672 00002000:00100000:0.0:1788619910.226570:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c75f4380 x1875503571356672/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:6/0 lens 488/0 e 0 to 0 dl 1788619921 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619910.226601:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c75f4380 with x1875503571356672 00000100:00100000:0.0:1788619910.226640:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 433 00000100:00100000:0.0:1788619910.226649:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2354:x1875503571356672:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619910.230046:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2354:x1875503571356672:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 3378us (3646us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619910.230066:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 433 00002000:00100000:0.0:1788619910.230086:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c75f4380 x1875503571356672/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:6/0 lens 488/440 e 0 to 0 dl 1788619921 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619910.230098:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c75f4380 with x1875503571356672 00000100:00100000:2.0:1788619910.240848:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619910.240966:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571356800 00000100:00100000:0.0:1788619910.240979:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571356800 00000100:00100000:0.0:1788619910.241052:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 588 00000100:00100000:0.0:1788619910.241065:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b10a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571356800:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619910.241088:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619910.241112:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d058bad0/0x1cd7fe5368734047 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b5e7 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619910.241207:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5019180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589066368:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619910.241315:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:3.0:1788619910.242646:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589066496:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1788619910.242728:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619910.242843:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589066496 00000100:00100000:1.0:1788619910.242853:0:6699:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589066496 00000100:00100000:1.0:1788619910.242940:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 85 00000100:00100000:1.0:1788619910.242951:0:6699:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589066496:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1788619910.242980:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 103874560 pending 10223616 free 3859808256 avail 3857711104 00000100:00100000:1.0:1788619910.243037:0:6699:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589066496:12345-0@lo:13:osp-pre-0-0.0 Request processed in 88us (309us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619910.243044:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 85 00000100:00080000:3.0:1788619910.243162:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1788619910.243212:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589066496:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1788619910.251591:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5019180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589066368:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619910.251648:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d058bad0/0x1cd7fe5368734047 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b5e7 expref: 17 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.251708:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d058bad0/0x1cd7fe5368734047 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b5e7 expref: 17 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.251726:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6e800, rc 0) 00000100:00100000:0.0:1788619910.251805:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b10a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571356800:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 10740us (10970us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619910.251815:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 588 00000100:00100000:2.0:1788619910.255280:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619910.255475:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571356928 00000100:00100000:3.0:1788619910.255488:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571356928 00002000:00100000:3.0:1788619910.255556:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b12300 x1875503571356928/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:6/0 lens 488/0 e 0 to 0 dl 1788619921 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619910.255568:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b12300 with x1875503571356928 00000100:00100000:3.0:1788619910.255638:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 434 00000100:00100000:3.0:1788619910.255649:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b12300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2354:x1875503571356928:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619910.256051:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b12300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2354:x1875503571356928:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 402us (784us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619910.256060:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 434 00002000:00100000:3.0:1788619910.256068:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b12300 x1875503571356928/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:6/0 lens 488/440 e 0 to 0 dl 1788619921 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619910.256076:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b12300 with x1875503571356928 00000100:00100000:2.0:1788619910.266389:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619910.266490:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571357056 00000100:00100000:0.0:1788619910.266504:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571357056 00000100:00100000:0.0:1788619910.266582:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 589 00000100:00100000:0.0:1788619910.266596:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b13b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571357056:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619910.266622:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619910.266647:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d058bad0/0x1cd7fe536873404e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b5ee expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619910.266742:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5019180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589066624:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619910.266853:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:0.0:1788619910.269573:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5019180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589066624:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619910.269627:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d058bad0/0x1cd7fe536873404e lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b5ee expref: 17 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.269657:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d058bad0/0x1cd7fe536873404e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b5ee expref: 17 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.269702:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6e800, rc 0) 00000100:00100000:0.0:1788619910.269855:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b13b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571357056:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3260us (3480us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619910.269865:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 589 00000100:00100000:2.0:1788619910.277253:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619910.277678:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571357184 00000100:00100000:0.0:1788619910.277692:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571357184 00002000:00100000:0.0:1788619910.277759:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b11c00 x1875503571357184/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:6/0 lens 488/0 e 0 to 0 dl 1788619921 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619910.277772:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b11c00 with x1875503571357184 00000100:00100000:0.0:1788619910.277827:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 435 00000100:00100000:0.0:1788619910.277838:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b11c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2354:x1875503571357184:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619910.278239:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b11c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2354:x1875503571357184:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 401us (1001us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619910.278249:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 435 00002000:00100000:0.0:1788619910.278256:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b11c00 x1875503571357184/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:6/0 lens 488/440 e 0 to 0 dl 1788619921 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619910.278265:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b11c00 with x1875503571357184 00000100:00100000:2.0:1788619910.286076:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619910.286198:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571357312 00000100:00100000:1.0:1788619910.286213:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571357312 00000100:00100000:1.0:1788619910.286289:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 590 00000100:00100000:1.0:1788619910.286302:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b11500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571357312:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619910.286367:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619910.286391:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002119e2f0/0x1cd7fe5368734055 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b5f5 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619910.286470:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fe300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589066752:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619910.286570:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963f00 going to sleep for 11 seconds 00000100:00100000:1.0:1788619910.289870:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fe300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589066752:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619910.289958:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002119e2f0/0x1cd7fe5368734055 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b5f5 expref: 17 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619910.289992:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002119e2f0/0x1cd7fe5368734055 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b5f5 expref: 17 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619910.290035:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f4b400, rc 0) 00000100:00100000:1.0:1788619910.290161:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b11500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571357312:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3861us (4101us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619910.290180:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 590 00000100:00100000:2.0:1788619910.295525:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619910.295640:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571357440 00000100:00100000:0.0:1788619910.295654:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571357440 00002000:00100000:0.0:1788619910.295722:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b10380 x1875503571357440/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:6/0 lens 488/0 e 0 to 0 dl 1788619921 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619910.295809:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b10380 with x1875503571357440 00000100:00100000:0.0:1788619910.295892:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 436 00000100:00100000:0.0:1788619910.295921:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b10380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2353:x1875503571357440:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619910.296607:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b10380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2353:x1875503571357440:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 699us (1095us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619910.296618:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 436 00002000:00100000:0.0:1788619910.296627:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b10380 x1875503571357440/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:6/0 lens 488/440 e 0 to 0 dl 1788619921 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619910.296637:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b10380 with x1875503571357440 00000100:00100000:2.0:1788619910.300646:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619910.300765:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571357568 00000100:00100000:1.0:1788619910.300778:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571357568 00000100:00100000:1.0:1788619910.300856:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 591 00000100:00100000:1.0:1788619910.300869:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571357568:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619910.300894:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619910.300927:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002119e2f0/0x1cd7fe536873405c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b5fc expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619910.301011:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fe300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589066880:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619910.301123:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963f00 going to sleep for 11 seconds 00000100:00100000:1.0:1788619910.304096:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fe300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589066880:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619910.304152:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002119e2f0/0x1cd7fe536873405c lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b5fc expref: 17 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619910.304201:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002119e2f0/0x1cd7fe536873405c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b5fc expref: 17 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619910.304220:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f4b400, rc 0) 00000100:00100000:1.0:1788619910.304315:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571357568:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3449us (3685us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619910.304416:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 591 00000100:00100000:2.0:1788619910.308625:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619910.308774:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571357696 00000100:00100000:3.0:1788619910.308788:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571357696 00002000:00100000:3.0:1788619910.308858:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fefd7800 x1875503571357696/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:6/0 lens 488/0 e 0 to 0 dl 1788619921 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619910.308869:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fefd7800 with x1875503571357696 00000100:00100000:3.0:1788619910.308926:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 437 00000100:00100000:3.0:1788619910.308938:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2352:x1875503571357696:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619910.309563:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2352:x1875503571357696:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 623us (950us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619910.309574:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 437 00002000:00100000:3.0:1788619910.309584:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fefd7800 x1875503571357696/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:6/0 lens 488/440 e 0 to 0 dl 1788619921 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619910.309593:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fefd7800 with x1875503571357696 00000100:00100000:2.0:1788619910.318022:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619910.318294:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571357824 00000100:00100000:0.0:1788619910.318309:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571357824 00000100:00100000:0.0:1788619910.318428:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 592 00000100:00100000:0.0:1788619910.318441:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e647aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571357824:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619910.319240:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619910.319273:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000084d1816c/0x1cd7fe5368734063 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b603 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619910.319998:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5019180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589067008:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619910.320146:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:2.0:1788619910.324514:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5019180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589067008:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619910.324569:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000084d1816c/0x1cd7fe5368734063 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b603 expref: 17 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619910.324603:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000084d1816c/0x1cd7fe5368734063 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b603 expref: 17 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619910.324620:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6e000, rc 0) 00000100:00100000:2.0:1788619910.324701:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e647aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571357824:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6262us (6695us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619910.324712:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 592 00000100:00100000:2.0:1788619910.332522:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619910.333144:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571357952 00000100:00100000:0.0:1788619910.333161:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571357952 00002000:00100000:0.0:1788619910.333252:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78dfa42d80 x1875503571357952/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:6/0 lens 488/0 e 0 to 0 dl 1788619921 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619910.333266:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78dfa42d80 with x1875503571357952 00000100:00100000:0.0:1788619910.333318:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 438 00000100:00100000:0.0:1788619910.333370:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa42d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2354:x1875503571357952:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619910.333745:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa42d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2354:x1875503571357952:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 377us (1237us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619910.333955:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 438 00002000:00100000:0.0:1788619910.333966:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78dfa42d80 x1875503571357952/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:6/0 lens 488/440 e 0 to 0 dl 1788619921 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619910.333975:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78dfa42d80 with x1875503571357952 00000100:00100000:1.0:1788619910.344658:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619910.345020:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571358080 00000100:00100000:2.0:1788619910.345034:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571358080 00000100:00100000:2.0:1788619910.345109:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 593 00000100:00100000:2.0:1788619910.345122:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010fe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2353:x1875503571358080:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619910.345144:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619910.345179:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007527590a/0x1cd7fe536873406a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b60a expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619910.345262:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78dfa42300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589067136:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619910.345405:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc240 going to sleep for 11 seconds 00000100:00100000:2.0:1788619910.348640:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78dfa42300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589067136:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619910.348687:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007527590a/0x1cd7fe536873406a lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b60a expref: 17 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619910.348715:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007527590a/0x1cd7fe536873406a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b60a expref: 17 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619910.348739:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6800, rc 0) 00000100:00100000:2.0:1788619910.348935:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010fe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2353:x1875503571358080:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3814us (4291us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619910.348945:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 593 00000100:00100000:3.0:1788619910.351961:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619910.352133:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571358208 00000100:00100000:1.0:1788619910.352146:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571358208 00000100:00100000:1.0:1788619910.352230:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 594 00000100:00100000:1.0:1788619910.352242:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2352:x1875503571358208:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619910.352267:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619910.352291:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000822e5532/0x1cd7fe5368734071 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b611 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619910.352412:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010ffb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589067264:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619910.352511:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963f00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619910.355388:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010ffb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589067264:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619910.355439:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000822e5532/0x1cd7fe5368734071 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b611 expref: 17 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619910.355497:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000822e5532/0x1cd7fe5368734071 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b611 expref: 17 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619910.355514:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f4bc00, rc 0) 00000100:00100000:2.0:1788619910.355581:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2352:x1875503571358208:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3342us (3633us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619910.355592:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 594 00000100:00100000:0.0:1788619910.360104:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619910.360497:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571358464 00000100:00100000:3.0:1788619910.360512:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571358464 00000100:00080000:0.0:1788619910.360670:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788619910.360722:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 764 00000100:00100000:3.0:1788619910.360755:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:14908:x1875503571358464:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:3.0:1788619910.360841:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619910.360879:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368733fc2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b569 expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619910.360921:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619910.360928:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788619910.361229:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368734078 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619910.361248:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4ac00/0x1cd7fe5368733fd7 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b577 expref: 21 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619910.361258:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368734078 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619910.361302:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4ac00/0x1cd7fe5368733fd7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b577 expref: 21 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619910.361318:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4ac00/0x1cd7fe5368733fd7 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b577 expref: 21 pid: 5844 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619910.361628:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c4700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589067392:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619910.362608:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5c76480 going to sleep for 11 seconds 00000100:00100000:2.0:1788619910.368637:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619910.368743:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c4700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589067392:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619910.368791:0:5845:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368734078 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619910.369016:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571358592 00000100:00100000:1.0:1788619910.369026:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571358592 00010000:00010000:1.0:1788619910.369076:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4ac00/0x1cd7fe5368733fd7 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b577 expref: 22 pid: 5844 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619910.369125:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 258 00000100:00100000:1.0:1788619910.369136:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa42300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2354:x1875503571358592:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619910.369155:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619910.369162:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4ac00/0x1cd7fe5368733fd7 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b577 expref: 22 pid: 5844 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619910.369187:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4ac00/0x1cd7fe5368733fd7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b577 expref: 22 pid: 5844 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619910.369253:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4ac00/0x1cd7fe5368733fd7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b577 expref: 22 pid: 5844 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619910.369272:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619910.369278:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368734078 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619910.369289:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368734078 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619910.369298:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368734078 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619910.369355:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619910.369420:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa42300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2354:x1875503571358592:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 285us (795us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619910.369427:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 258 00010000:00010000:2.0:1788619910.369505:0:5845:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368734078 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619910.369514:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368734078 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788619910.369687:0:5845:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967594 00000004:00010000:2.0:1788619910.369752:0:5845:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 816620 -> 816620 00000004:00010000:2.0:1788619910.369757:0:5845:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619909 -> 1788619910 00000004:00010000:2.0:1788619910.369760:0:5845:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619908 -> 1788619908 00000004:00010000:2.0:1788619910.369762:0:5845:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619909 -> 1788619910 00000004:00010000:2.0:1788619910.369765:0:5845:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1416 -> 1416 00000004:00010000:2.0:1788619910.369773:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000dbdbbbe8 reply state 000000004c8f4660 transno 4294967595 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368734078 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619910.369800:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788619910.369893:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+14:14908:x1875503571358464:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 9142us (9804us total) trans 4294967595 rc 0/0 00000100:00100000:2.0:1788619910.369927:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 764 00000020:00080000:2.0:1788619910.369931:0:5845:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:1.0:1788619910.374133:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619910.374211:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571358720 00000100:00100000:2.0:1788619910.374223:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571358720 00000100:00100000:2.0:1788619910.374295:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 765 00000100:00100000:2.0:1788619910.374305:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010ffb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+14:14908:x1875503571358720:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619910.374368:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619910.374397:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6800/0x1cd7fe536873407f lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b618 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619910.374434:0:11397:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368733fc9 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b570 expref: 20 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619910.374445:0:11397:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6800/0x1cd7fe536873407f lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b618 expref: 15 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619910.374487:0:11397:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368733fc9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b570 expref: 20 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619910.374502:0:11397:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368733fc9 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b570 expref: 20 pid: 11397 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619910.374518:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78dfa43100 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589067520:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619910.374613:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc240 going to sleep for 11 seconds 00010000:00010000:3.0:1788619910.379231:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368734078 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619910.379256:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368734078 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619910.379294:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368734078 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619910.379302:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c683c800 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:3.0:1788619910.379308:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368734078 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619910.379318:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368734078 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619910.379828:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619910.379832:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788619910.379840:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368734078 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619910.379848:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368734078 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619910.382122:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619910.382241:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571358848 00000100:00100000:1.0:1788619910.382255:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571358848 00010000:00010000:1.0:1788619910.382303:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368733fc9 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b570 expref: 21 pid: 11397 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619910.382354:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 259 00000100:00100000:1.0:1788619910.382362:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5019180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2355:x1875503571358848:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619910.382376:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619910.382408:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368733fc9 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b570 expref: 21 pid: 11397 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619910.382432:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368733fc9 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b570 expref: 21 pid: 11397 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619910.382455:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619910.382460:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6800/0x1cd7fe536873407f lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b618 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619910.382470:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6800/0x1cd7fe536873407f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b618 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619910.382480:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6800/0x1cd7fe536873407f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b618 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619910.382529:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6800/0x1cd7fe536873407f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b618 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619910.382544:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c4a4ed80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589067648:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619910.382633:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963d80 going to sleep for 11 seconds 00010000:00010000:2.0:1788619910.383141:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368733fc9 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b570 expref: 21 pid: 11397 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619910.383178:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78dfa43100 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589067520:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619910.383210:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6800/0x1cd7fe536873407f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b618 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619910.383222:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619910.383225:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619910.383229:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6800, rc 0) 00000100:00100000:2.0:1788619910.383263:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010ffb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+14:14908:x1875503571358720:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8960us (9145us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619910.383270:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 765 00000100:00100000:0.0:1788619910.388271:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619910.388305:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619910.388381:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571359104 00000100:00100000:3.0:1788619910.388394:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571359104 00000100:00100000:1.0:1788619910.388428:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c4a4ed80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589067648:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619910.388458:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00002000:00010000:3.0:1788619910.388475:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:3.0:1788619910.388481:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c8595880 with x1875503571359104 ext(0->18446744073709551615) 00002000:00010000:3.0:1788619910.388494:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c8595880. 00000100:00100000:1.0:1788619910.388519:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5019180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2355:x1875503571358848:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 6156us (6407us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619910.388526:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 315 00000100:00100000:1.0:1788619910.388530:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 259 00000100:00100000:3.0:1788619910.388535:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8595880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+15:2355:x1875503571359104:12345-192.168.201.5@tcp:10:fsx.0 00000100:00100000:2.0:1788619910.388608:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571358976 00000100:00100000:2.0:1788619910.388613:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571358976 00002000:00010000:2.0:1788619910.388638:0:5857:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:2.0:1788619910.388644:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c8594a80 with x1875503571358976 00000100:00100000:2.0:1788619910.388673:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 439 00000100:00100000:2.0:1788619910.388680:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8594a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+14:2355:x1875503571358976:12345-192.168.201.5@tcp:10:fsx.0 00000100:00080000:0.0:1788619910.388865:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:3.0:1788619910.388942:0:6702:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1644160 -> 0 00002000:00010000:3.0:1788619910.388946:0:6702:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619909 -> 1788619910 00002000:00010000:3.0:1788619910.388949:0:6702:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619909 -> 1788619907 00002000:00010000:3.0:1788619910.388970:0:6702:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619909 -> 1788619910 00002000:00010000:3.0:1788619910.388972:0:6702:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 391 -> 391 00000100:00100000:3.0:1788619910.389004:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8595880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571359104:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 470us (697us total) trans 4294967383 rc 0/0 00000100:00100000:3.0:1788619910.389012:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 315 00002000:00010000:3.0:1788619910.389016:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:3.0:1788619910.389019:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c8595880 with x1875503571359104 ext(0->18446744073709551615) 00002000:00010000:3.0:1788619910.389024:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c8595880. 00000004:00010000:2.0:1788619910.390408:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 816620 -> 608213 00000004:00010000:2.0:1788619910.390419:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619910 -> 1788619910 00000004:00010000:2.0:1788619910.390422:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619908 -> 1788619908 00000004:00010000:2.0:1788619910.390425:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619910 -> 1788619910 00000004:00010000:2.0:1788619910.390427:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1416 -> 1416 00000100:00100000:2.0:1788619910.390495:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8594a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+15:2355:x1875503571358976:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 1810us (2231us total) trans 4294967596 rc 0/0 00000100:00100000:2.0:1788619910.390504:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 439 00002000:00010000:2.0:1788619910.390526:0:5857:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:2.0:1788619910.390529:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c8594a80 with x1875503571358976 00000100:00100000:2.0:1788619910.394627:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619910.394717:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571359232 00000100:00100000:3.0:1788619910.394730:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571359232 00000100:00100000:3.0:1788619910.394867:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 766 00000100:00100000:3.0:1788619910.394880:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa43100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+15:14908:x1875503571359232:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619910.394921:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619910.394948:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368734086 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b61f expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619910.395066:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe536873408d lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619910.395085:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe536873408d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619910.395099:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe536873408d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788619910.395132:0:5846:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 608213 00000004:00200000:3.0:1788619910.395137:0:5846:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788619910.395161:0:5846:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe536873408d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619910.395291:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368734086 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b61f expref: 17 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619910.395312:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe536873408d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b61f expref: 16 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619910.395322:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619910.395379:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788619910.395384:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683dc00, rc 0) 00000100:00100000:3.0:1788619910.395457:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa43100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+16:14908:x1875503571359232:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 580us (840us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619910.395465:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 766 00000100:00100000:0.0:1788619910.403596:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619910.403980:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571359360 00000100:00100000:3.0:1788619910.404009:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571359360 00002000:00100000:3.0:1788619910.404117:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be0380 x1875503571359360/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:6/0 lens 488/0 e 0 to 0 dl 1788619921 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619910.404131:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be0380 with x1875503571359360 00000100:00100000:3.0:1788619910.404190:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 440 00000100:00100000:3.0:1788619910.404204:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+16:2353:x1875503571359360:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619910.405709:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+16:2353:x1875503571359360:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 1495us (2117us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619910.405725:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 440 00002000:00100000:2.0:1788619910.405757:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be0380 x1875503571359360/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:6/0 lens 488/4536 e 0 to 0 dl 1788619921 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619910.405768:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be0380 with x1875503571359360 00000100:00100000:0.0:1788619910.410831:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619910.412263:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571359488 00000100:00100000:2.0:1788619910.412283:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571359488 00002000:00100000:2.0:1788619910.412378:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be2300 x1875503571359488/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:6/0 lens 488/0 e 0 to 0 dl 1788619921 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619910.412393:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be2300 with x1875503571359488 00000100:00100000:2.0:1788619910.412505:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 441 00000100:00100000:2.0:1788619910.412516:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571359488:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619910.412913:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571359488:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 382us (2081us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619910.412922:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 441 00002000:00100000:2.0:1788619910.412931:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be2300 x1875503571359488/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:6/0 lens 488/4536 e 0 to 0 dl 1788619921 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619910.412939:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be2300 with x1875503571359488 00000100:00100000:0.0:1788619910.418598:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619910.418727:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571359616 00000100:00100000:2.0:1788619910.418741:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571359616 00002000:00100000:2.0:1788619910.418821:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b17800 x1875503571359616/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:6/0 lens 488/0 e 0 to 0 dl 1788619921 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619910.418859:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b17800 with x1875503571359616 00000100:00100000:1.0:1788619910.419073:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 442 00000100:00100000:1.0:1788619910.419087:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b17800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+16:2354:x1875503571359616:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619910.420175:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b17800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+16:2354:x1875503571359616:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 1082us (1585us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619910.420838:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 442 00002000:00100000:1.0:1788619910.420855:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b17800 x1875503571359616/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:6/0 lens 488/4536 e 0 to 0 dl 1788619921 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619910.420868:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b17800 with x1875503571359616 00000100:00100000:0.0:1788619910.430130:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619910.430305:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571359744 00000100:00100000:2.0:1788619910.430318:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571359744 00002000:00100000:2.0:1788619910.430474:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b15c00 x1875503571359744/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:6/0 lens 488/0 e 0 to 0 dl 1788619921 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619910.430488:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b15c00 with x1875503571359744 00000100:00100000:2.0:1788619910.430536:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 443 00000100:00100000:2.0:1788619910.430548:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b15c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571359744:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619910.433926:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b15c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571359744:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 1608us (2040us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619910.433956:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 443 00002000:00100000:2.0:1788619910.433979:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b15c00 x1875503571359744/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:6/0 lens 488/4536 e 0 to 0 dl 1788619921 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619910.433991:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b15c00 with x1875503571359744 00000100:00100000:0.0:1788619910.439415:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619910.439969:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571359872 00000100:00100000:2.0:1788619910.439983:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571359872 00002000:00100000:2.0:1788619910.440043:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b14380 x1875503571359872/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:6/0 lens 488/0 e 0 to 0 dl 1788619921 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619910.440053:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b14380 with x1875503571359872 00000100:00100000:2.0:1788619910.440094:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 444 00000100:00100000:2.0:1788619910.440105:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b14380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+16:2354:x1875503571359872:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619910.440873:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b14380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+16:2354:x1875503571359872:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 764us (1467us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619910.440884:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 444 00002000:00100000:2.0:1788619910.440924:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b14380 x1875503571359872/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:6/0 lens 488/4536 e 0 to 0 dl 1788619921 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619910.440933:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b14380 with x1875503571359872 00000100:00100000:0.0:1788619910.459667:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619910.460037:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571360000 00000100:00100000:2.0:1788619910.460052:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571360000 00002000:00100000:2.0:1788619910.460136:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b16d80 x1875503571360000/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:6/0 lens 488/0 e 0 to 0 dl 1788619921 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619910.460150:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b16d80 with x1875503571360000 00000100:00100000:2.0:1788619910.460299:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 445 00000100:00100000:2.0:1788619910.460312:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b16d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571360000:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619910.461093:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b16d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571360000:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 779us (1434us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619910.461104:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 445 00002000:00100000:2.0:1788619910.461112:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b16d80 x1875503571360000/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:6/0 lens 488/4536 e 0 to 0 dl 1788619921 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619910.461121:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b16d80 with x1875503571360000 00000100:00100000:2.0:1788619910.466025:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619910.466126:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571360128 00000100:00100000:1.0:1788619910.466138:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571360128 00002000:00100000:1.0:1788619910.466214:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78dfa40700 x1875503571360128/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:6/0 lens 488/0 e 0 to 0 dl 1788619921 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619910.466226:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78dfa40700 with x1875503571360128 00000100:00100000:1.0:1788619910.466267:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 446 00000100:00100000:1.0:1788619910.466280:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa40700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+16:2352:x1875503571360128:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619910.466663:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa40700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+16:2352:x1875503571360128:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 385us (654us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619910.466670:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 446 00002000:00100000:1.0:1788619910.466677:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78dfa40700 x1875503571360128/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:6/0 lens 488/4536 e 0 to 0 dl 1788619921 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619910.466685:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78dfa40700 with x1875503571360128 00000100:00100000:0.0:1788619910.472033:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619910.472107:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571360256 00000100:00100000:2.0:1788619910.472121:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571360256 00002000:00100000:2.0:1788619910.472267:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df967b80 x1875503571360256/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:6/0 lens 488/0 e 0 to 0 dl 1788619921 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619910.472289:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df967b80 with x1875503571360256 00000100:00100000:1.0:1788619910.472377:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 447 00000100:00100000:1.0:1788619910.472443:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df967b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+16:2353:x1875503571360256:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619910.472816:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df967b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+16:2353:x1875503571360256:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 377us (794us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619910.474517:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 447 00002000:00100000:1.0:1788619910.474561:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df967b80 x1875503571360256/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:6/0 lens 488/4536 e 0 to 0 dl 1788619921 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619910.474575:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df967b80 with x1875503571360256 00000100:00100000:0.0:1788619910.478698:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619910.478998:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571360384 00000100:00100000:1.0:1788619910.479015:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571360384 00002000:00100000:1.0:1788619910.479086:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df387480 x1875503571360384/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:6/0 lens 488/0 e 0 to 0 dl 1788619921 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619910.479100:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df387480 with x1875503571360384 00000100:00100000:1.0:1788619910.479147:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 448 00000100:00100000:1.0:1788619910.479158:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df387480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+16:2352:x1875503571360384:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619910.479629:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df387480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+16:2352:x1875503571360384:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 466us (940us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619910.479638:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 448 00002000:00100000:1.0:1788619910.479646:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df387480 x1875503571360384/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:6/0 lens 488/4536 e 0 to 0 dl 1788619921 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619910.479654:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df387480 with x1875503571360384 00000100:00100000:0.0:1788619910.485631:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619910.486006:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571360512 00000100:00100000:1.0:1788619910.486021:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571360512 00002000:00100000:1.0:1788619910.486102:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6002680 x1875503571360512/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:6/0 lens 488/0 e 0 to 0 dl 1788619921 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619910.486116:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6002680 with x1875503571360512 00000100:00100000:1.0:1788619910.486225:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 449 00000100:00100000:1.0:1788619910.486237:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6002680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+16:2353:x1875503571360512:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:1.0:1788619910.490647:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 608213 -> 608213 00000004:00010000:1.0:1788619910.490660:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619910 -> 1788619910 00000004:00010000:1.0:1788619910.490738:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619908 -> 1788619908 00000004:00010000:1.0:1788619910.490741:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619910 -> 1788619910 00000004:00010000:1.0:1788619910.490743:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1416 -> 1416 00000100:00100000:1.0:1788619910.491069:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6002680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+18:2353:x1875503571360512:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 4816us (5437us total) trans 4294967597 rc 0/0 00000100:00100000:1.0:1788619910.491085:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 449 00002000:00100000:1.0:1788619910.491102:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6002680 x1875503571360512/t4294967597(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:6/0 lens 488/448 e 0 to 0 dl 1788619921 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619910.491113:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6002680 with x1875503571360512 00000100:00100000:0.0:1788619910.499596:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619910.499700:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571360640 00000100:00100000:2.0:1788619910.499718:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571360640 00000100:00100000:2.0:1788619910.499932:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 767 00000100:00100000:2.0:1788619910.499954:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed61880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+18:2352:x1875503571360640:12345-192.168.201.5@tcp:16:fsx.0 00000001:00080000:1.0:1788619910.577602:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967589 is committed 00000001:00080000:1.0:1788619910.577634:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967585 is committed 00000001:00080000:1.0:1788619910.577644:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967597 is committed 00000001:00080000:1.0:1788619910.577654:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967593 is committed 00000100:00100000:3.0:1788619910.577888:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed61880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571360640:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 77937us (78293us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619910.577917:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 767 00000100:00100000:2.0:1788619910.589566:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619910.593714:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571360896 00000100:00100000:3.0:1788619910.593736:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571360896 00000100:00100000:2.0:1788619910.595107:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 768 00000100:00100000:2.0:1788619910.595133:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa40380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+11:14908:x1875503571360896:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:2.0:1788619910.595609:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe5368734094 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619910.595634:0:11397:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe536873408d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b61f expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619910.595645:0:11397:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe5368734094 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619910.595693:0:11397:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe536873408d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b61f expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619910.595708:0:11397:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe536873408d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b61f expref: 9 pid: 5846 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619910.595721:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78dfa41c00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589067776:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619910.595855:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619910.602613:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619910.603264:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571361024 00000100:00100000:1.0:1788619910.603276:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571361024 00010000:00010000:1.0:1788619910.603357:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe536873408d lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b61f expref: 10 pid: 5846 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619910.603392:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 260 00000100:00100000:1.0:1788619910.603401:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa40e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571361024:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619910.603414:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619910.603420:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe536873408d lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b61f expref: 10 pid: 5846 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619910.603431:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe536873408d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b61f expref: 10 pid: 5846 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619910.603475:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619910.603479:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe5368734094 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619910.603489:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe5368734094 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619910.603496:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe5368734094 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619910.603515:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619910.603572:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa40e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571361024:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 172us (970us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619910.603577:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 260 00010000:00010000:0.0:1788619910.604385:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe536873408d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b61f expref: 9 pid: 5846 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619910.604423:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78dfa41c00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589067776:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619910.604451:0:11397:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe5368734094 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.604461:0:11397:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe5368734094 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.604466:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe5368734094 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788619910.604605:0:11397:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967598 00000004:00010000:0.0:1788619910.604657:0:11397:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 608213 -> 608213 00000004:00010000:0.0:1788619910.604661:0:11397:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619910 -> 1788619910 00000004:00010000:0.0:1788619910.604664:0:11397:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619908 -> 1788619908 00000004:00010000:0.0:1788619910.604666:0:11397:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619910 -> 1788619910 00000004:00010000:0.0:1788619910.604668:0:11397:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1416 -> 518 00000004:00010000:0.0:1788619910.604676:0:11397:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000000fff6532 reply state 00000000136e1f5a transno 4294967599 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe5368734094 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619910.604683:0:11397:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788619910.604781:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa40380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571360896:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 9659us (15232us total) trans 4294967599 rc 0/0 00000100:00100000:0.0:1788619910.604789:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 768 00000020:00080000:0.0:1788619910.604794:0:11397:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1788619910.618207:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619910.618302:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571361152 00000100:00100000:0.0:1788619910.618316:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571361152 00000100:00100000:0.0:1788619910.618427:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 769 00000100:00100000:0.0:1788619910.618440:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa41c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571361152:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619910.618490:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619910.618519:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe536873409b lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b626 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.618557:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6800/0x1cd7fe536873407f lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b618 expref: 8 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.618569:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe536873409b lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b626 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.618611:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6800/0x1cd7fe536873407f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b618 expref: 8 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.618627:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6800/0x1cd7fe536873407f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b618 expref: 8 pid: 11397 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619910.618642:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed62300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589067904:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619910.618735:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa9c0 going to sleep for 11 seconds 00010000:00010000:3.0:1788619910.619967:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe5368734094 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619910.620003:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe5368734094 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619910.620012:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe5368734094 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619910.620018:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78cdcc7c00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:3.0:1788619910.620024:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe5368734094 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619910.620034:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe5368734094 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619910.620054:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619910.620057:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788619910.620063:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe5368734094 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619910.620070:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe5368734094 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619910.622687:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed62300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589067904:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619910.622749:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe536873409b lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b626 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.622788:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619910.623068:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619910.623079:0:5846:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe536873409b lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b626 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.623118:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6f000, rc 0) 00000100:00100000:0.0:1788619910.623204:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa41c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571361152:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4765us (5011us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619910.623215:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 769 00000100:00100000:2.0:1788619910.625243:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619910.625426:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571361280 00000100:00100000:3.0:1788619910.625440:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571361280 00010000:00010000:3.0:1788619910.625496:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6800/0x1cd7fe536873407f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b618 expref: 9 pid: 11397 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619910.625549:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 261 00000100:00100000:3.0:1788619910.625561:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571361280:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619910.625580:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619910.625633:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6800/0x1cd7fe536873407f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b618 expref: 9 pid: 11397 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619910.625651:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6800/0x1cd7fe536873407f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b618 expref: 9 pid: 11397 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619910.625681:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6800/0x1cd7fe536873407f lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b618 expref: 9 pid: 11397 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619910.625700:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619910.625708:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe536873409b lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b626 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619910.625732:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe536873409b lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b626 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619910.625743:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe536873409b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b626 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619910.625917:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe536873409b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b626 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619910.625935:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c6300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589068032:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619910.626050:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5c76600 going to sleep for 11 seconds 00000100:00100000:3.0:1788619910.631572:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c6300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589068032:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619910.631611:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619910.631812:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+8:2354:x1875503571361280:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 6250us (6578us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619910.631847:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 261 00000100:00100000:0.0:1788619910.633176:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619910.633360:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571361408 00000100:00100000:1.0:1788619910.633373:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571361408 00000100:00100000:1.0:1788619910.633447:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 595 00000100:00100000:1.0:1788619910.633460:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed62300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:14908:x1875503571361408:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619910.633485:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619910.633511:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd0fba3b/0x1cd7fe53687340a2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b62d expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619910.633552:0:6698:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000112ced4b/0x1cd7fe5368733fb4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b562 expref: 15 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619910.633563:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=1s v=(6 9) 00010000:00010000:1.0:1788619910.633568:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(7 9) 00010000:00010000:1.0:1788619910.633590:0:6698:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd0fba3b/0x1cd7fe53687340a2 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b62d expref: 17 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619910.633634:0:6698:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000112ced4b/0x1cd7fe5368733fb4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b562 expref: 15 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619910.633650:0:6698:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000112ced4b/0x1cd7fe5368733fb4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b562 expref: 15 pid: 6696 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619910.633666:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c4a4ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589068160:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619910.633772:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963840 going to sleep for 11 seconds 00000100:00100000:2.0:1788619910.640915:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619910.641118:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c4a4ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589068160:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619910.641186:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000cd0fba3b/0x1cd7fe53687340a2 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b62d expref: 17 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619910.641215:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f4ac00, rc 0) 00000100:00100000:1.0:1788619910.641297:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed62300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+17:14908:x1875503571361408:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7838us (8139us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619910.641308:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 595 00000100:00100000:3.0:1788619910.641919:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571361536 00000100:00100000:3.0:1788619910.641930:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571361536 00010000:00010000:3.0:1788619910.641976:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000112ced4b/0x1cd7fe5368733fb4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b562 expref: 16 pid: 6696 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619910.642078:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 262 00000100:00100000:3.0:1788619910.642089:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+16:2354:x1875503571361536:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619910.642110:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619910.642142:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000112ced4b/0x1cd7fe5368733fb4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b562 expref: 16 pid: 6696 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619910.642278:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000112ced4b/0x1cd7fe5368733fb4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b562 expref: 16 pid: 6696 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619910.642316:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000112ced4b/0x1cd7fe5368733fb4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b562 expref: 16 pid: 6696 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619910.642368:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(7 9) 00010000:00010000:3.0:1788619910.642374:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(7 9) 00010000:00010000:3.0:1788619910.642384:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000cd0fba3b/0x1cd7fe53687340a2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b62d expref: 16 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619910.642406:0:5829:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd0fba3b/0x1cd7fe53687340a2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b62d expref: 16 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619910.642478:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000cd0fba3b/0x1cd7fe53687340a2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b62d expref: 16 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619910.642495:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c6300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1875503589068288:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619910.642583:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5c76600 going to sleep for 11 seconds 00000100:00100000:3.0:1788619910.650085:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c6300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1875503589068288:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619910.650126:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619910.650599:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+15:2354:x1875503571361536:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 8508us (9709us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619910.650613:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 262 00000100:00100000:2.0:1788619910.652038:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619910.653641:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571361664 00000100:00100000:1.0:1788619910.653658:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571361664 00002000:00010000:1.0:1788619910.653732:0:6701:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (2097152->18446744073709551615) 00002000:00100000:1.0:1788619910.653738:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c71f9880 with x1875503571361664 ext(2097152->18446744073709551615) 00002000:00010000:1.0:1788619910.653755:0:6701:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c71f9880. 00000100:00100000:3.0:1788619910.653973:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 316 00000100:00100000:3.0:1788619910.653989:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2354:x1875503571361664:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:3.0:1788619910.654475:0:6702:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 0 -> 2097152 00002000:00010000:3.0:1788619910.654481:0:6702:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619910 -> 1788619910 00002000:00010000:3.0:1788619910.654483:0:6702:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619907 -> 1788619907 00002000:00010000:3.0:1788619910.654486:0:6702:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619910 -> 1788619910 00002000:00010000:3.0:1788619910.654488:0:6702:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 391 -> 391 00000100:00100000:3.0:1788619910.654571:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2354:x1875503571361664:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 585us (2541us total) trans 4294967384 rc 0/0 00000100:00100000:3.0:1788619910.654580:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 316 00002000:00010000:3.0:1788619910.654585:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (2097152->18446744073709551615) 00002000:00100000:3.0:1788619910.654588:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c71f9880 with x1875503571361664 ext(2097152->18446744073709551615) 00002000:00010000:3.0:1788619910.654594:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c71f9880. 00000100:00100000:0.0:1788619910.661089:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619910.661525:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571361792 00000100:00100000:3.0:1788619910.661540:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571361792 00002000:00100000:3.0:1788619910.661611:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1875968->2097151) req@ffff8e78fed60a80 x1875503571361792/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:6/0 lens 488/0 e 0 to 0 dl 1788619921 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619910.661625:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fed60a80 with x1875503571361792 ext(1875968->2097151) 00002000:00010000:3.0:1788619910.661632:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fed60a80 00000100:00100000:3.0:1788619910.661675:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 317 00000100:00100000:3.0:1788619910.661686:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed60a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2352:x1875503571361792:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619910.688528:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed60a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2352:x1875503571361792:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 26830us (27441us total) trans 0 rc 221184/221184 00000100:00100000:0.0:1788619910.688541:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 317 00002000:00100000:0.0:1788619910.688564:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1875968->2097151) req@ffff8e78fed60a80 x1875503571361792/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:6/0 lens 488/440 e 0 to 0 dl 1788619921 ref 1 fl Complete:/600/0 rc 221184/221184 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619910.688573:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fed60a80 with x1875503571361792 ext(1875968->2097151) 00002000:00010000:0.0:1788619910.688579:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fed60a80 00000100:00100000:2.0:1788619910.692384:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619910.692682:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571361920 00000100:00100000:1.0:1788619910.692727:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571361920 00002000:00100000:1.0:1788619910.693095:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (2093056->2097151) req@ffff8e77c6a8aa00 x1875503571361920/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:6/0 lens 4584/0 e 0 to 0 dl 1788619921 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619910.693521:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a8aa00 with x1875503571361920 ext(2093056->2097151) 00002000:00010000:1.0:1788619910.693530:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a8aa00 00000100:00100000:1.0:1788619910.693584:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 318 00000100:00100000:1.0:1788619910.693599:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a8aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2354:x1875503571361920:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:1.0:1788619910.696287:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a8aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2354:x1875503571361920:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 2676us (3898us total) trans 4294967385 rc 0/0 00000100:00100000:1.0:1788619910.696306:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 318 00002000:00100000:1.0:1788619910.696321:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (2093056->2097151) req@ffff8e77c6a8aa00 x1875503571361920/t4294967385(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:6/0 lens 4584/448 e 0 to 0 dl 1788619921 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619910.696367:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a8aa00 with x1875503571361920 ext(2093056->2097151) 00002000:00010000:1.0:1788619910.696375:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a8aa00 00000100:00100000:2.0:1788619910.699842:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619910.699943:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571362048 00000100:00100000:0.0:1788619910.699966:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571362048 00000100:00100000:0.0:1788619910.700037:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 596 00000100:00100000:0.0:1788619910.700049:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a8ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2354:x1875503571362048:12345-192.168.201.5@tcp:16:fsx.0 00000001:00080000:2.0:1788619910.758113:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967380 is committed 00000001:00080000:2.0:1788619910.758137:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967379 is committed 00000001:00080000:2.0:1788619910.758175:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967385 is committed 00000001:00080000:2.0:1788619910.758295:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967382 is committed 00000001:00080000:2.0:1788619910.758355:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967383 is committed 00000100:00100000:0.0:1788619910.758634:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a8ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571362048:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 58579us (58798us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619910.758648:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 596 00000100:00100000:2.0:1788619910.780156:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619910.780388:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571362176 00000100:00100000:3.0:1788619910.780402:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571362176 00000100:00100000:3.0:1788619910.780485:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 770 00000100:00100000:3.0:1788619910.780499:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d02680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571362176:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619910.780539:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619910.780567:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe53687340a9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b634 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619910.780688:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe53687340b0 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619910.780708:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe53687340b0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619910.780735:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe53687340b0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788619910.780772:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 2097152 00000004:00200000:3.0:1788619910.781051:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788619910.781081:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe53687340b0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619910.781135:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe53687340a9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b634 expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619910.781171:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe53687340b0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b634 expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619910.781250:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619910.781254:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788619910.781258:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683ca00, rc 0) 00000100:00100000:3.0:1788619910.781557:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d02680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:14908:x1875503571362176:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 1059us (1418us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619910.781568:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 770 00000100:00100000:2.0:1788619910.790312:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619910.790545:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571362304 00000100:00100000:0.0:1788619910.790613:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571362304 00000100:00100000:0.0:1788619910.790726:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 771 00000100:00100000:0.0:1788619910.790738:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d00a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+8:14908:x1875503571362304:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619910.790795:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619910.790823:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ec00/0x1cd7fe53687340b7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b63b expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.790946:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e200/0x1cd7fe53687340be lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.790966:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e200/0x1cd7fe53687340be lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.790981:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e200/0x1cd7fe53687340be lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619910.791015:0:11397:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 2097152 00000004:00200000:0.0:1788619910.791020:0:11397:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619910.791044:0:11397:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e200/0x1cd7fe53687340be lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.791086:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ec00/0x1cd7fe53687340b7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b63b expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.791106:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e200/0x1cd7fe53687340be lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b63b expref: 9 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.791116:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619910.791118:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619910.791123:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6e200, rc 0) 00000100:00100000:0.0:1788619910.791445:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d00a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571362304:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 707us (1145us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619910.791454:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 771 00000100:00100000:2.0:1788619910.807762:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619910.807818:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619910.807974:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571362560 00000100:00100000:3.0:1788619910.807988:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571362560 00000100:00100000:3.0:1788619910.808064:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 597 00000100:00100000:3.0:1788619910.808087:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79008ac000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571362560:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619910.808110:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619910.808134:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062c34eb6/0x1cd7fe53687340c5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b649 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619910.808229:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589068416:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619910.808394:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5c76480 going to sleep for 11 seconds 00000100:00100000:0.0:1788619910.809423:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571362432 00000100:00100000:0.0:1788619910.809435:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571362432 00000100:00100000:0.0:1788619910.809501:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 772 00000100:00100000:0.0:1788619910.809511:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dc9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571362432:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619910.809529:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619910.809551:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ec00/0x1cd7fe53687340cc lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b642 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.809593:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe536873409b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b626 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619910.809635:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed62a00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589068544:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619910.809720:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78eab85e40 going to sleep for 11 seconds 00002000:00010000:1.0:1788619910.829736:0:6697:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619907 -> 1788619910 00000100:00100000:1.0:1788619910.829790:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589068416:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619910.829840:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000062c34eb6/0x1cd7fe53687340c5 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b649 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619910.829870:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062c34eb6/0x1cd7fe53687340c5 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b649 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619910.829885:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c800, rc 0) 00000100:00100000:1.0:1788619910.829977:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79008ac000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571362560:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 21892us (22156us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619910.830006:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 597 00000004:00010000:0.0:1788619910.831036:0:5846:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619908 -> 1788619910 00000100:00100000:0.0:1788619910.831069:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed62a00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589068544:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619910.831096:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 608213 00010000:00010000:0.0:1788619910.831116:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ec00/0x1cd7fe53687340cc lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b642 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.831135:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619910.831138:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619910.831143:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ec00/0x1cd7fe53687340cc lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b642 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.831168:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6ec00, rc 0) 00000100:00100000:0.0:1788619910.831386:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dc9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571362432:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 21875us (23644us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619910.831409:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 772 00000100:00100000:2.0:1788619910.843301:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619910.843430:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619910.843587:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571362816 00000100:00100000:1.0:1788619910.843600:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571362816 00000100:00100000:1.0:1788619910.843663:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 598 00000100:00100000:1.0:1788619910.843672:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571362816:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619910.843692:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619910.843714:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000043a20f27/0x1cd7fe53687340d3 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b657 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619910.843820:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c4a4ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589068672:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619910.843942:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd9633c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619910.844568:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571362688 00000100:00100000:0.0:1788619910.844574:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571362688 00000100:00100000:0.0:1788619910.844620:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 773 00000100:00100000:0.0:1788619910.844626:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79008ad880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571362688:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619910.844639:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619910.844653:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ec00/0x1cd7fe53687340da lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b650 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.844687:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe536873409b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b626 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619910.844711:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed62a00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589068800:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619910.844766:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78eab85e40 going to sleep for 11 seconds 00000100:00100000:1.0:1788619910.850000:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c4a4ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589068672:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619910.850055:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000043a20f27/0x1cd7fe53687340d3 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b657 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619910.850099:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000043a20f27/0x1cd7fe53687340d3 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b657 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619910.850118:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f4b200, rc 0) 00000100:00100000:1.0:1788619910.850210:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571362816:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6536us (6779us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619910.850220:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 598 00000100:00100000:0.0:1788619910.866757:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed62a00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589068800:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619910.866800:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 608213 00010000:00010000:0.0:1788619910.866826:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ec00/0x1cd7fe53687340da lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b650 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.866854:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619910.866858:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619910.866864:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ec00/0x1cd7fe53687340da lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b650 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.866881:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6ec00, rc 0) 00000100:00100000:0.0:1788619910.867011:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79008ad880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571362688:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 22382us (23723us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619910.867040:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 773 00000100:00100000:2.0:1788619910.907302:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619910.907508:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571362944 00000100:00100000:0.0:1788619910.907521:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571362944 00000100:00100000:0.0:1788619910.907618:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 319 00000100:00100000:0.0:1788619910.907630:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571362944:12345-192.168.201.5@tcp:22:fsx.0 00002000:00010000:0.0:1788619910.908082:0:6703:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 2097152 -> 2097152 00002000:00010000:0.0:1788619910.908087:0:6703:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619910 -> 1788619910 00002000:00010000:0.0:1788619910.908089:0:6703:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619910 -> 1788619907 00002000:00010000:0.0:1788619910.908092:0:6703:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619910 -> 1788619910 00002000:00010000:0.0:1788619910.908094:0:6703:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 391 -> 135 00000100:00100000:0.0:1788619910.908184:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571362944:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 556us (897us total) trans 4294967386 rc 0/0 00000100:00100000:0.0:1788619910.908194:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 319 00000100:00100000:1.0:1788619910.915077:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619910.915255:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571363072 00000100:00100000:0.0:1788619910.915268:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571363072 00000100:00100000:0.0:1788619910.915361:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 774 00000100:00100000:0.0:1788619910.915374:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571363072:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619910.915401:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619910.915429:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ec00/0x1cd7fe53687340e1 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b65e expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619910.915433:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00010000:00010000:0.0:1788619910.915480:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe536873409b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b626 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619910.915533:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed62a00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589068928:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619910.915704:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78eab85e40 going to sleep for 11 seconds 00000100:00100000:3.0:1788619910.916656:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571363200 00000100:00100000:3.0:1788619910.916670:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571363200 00000100:00100000:3.0:1788619910.917035:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 599 00000100:00100000:3.0:1788619910.917046:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa14a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571363200:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619910.917079:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619910.917099:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe53687340e8 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b665 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619910.917183:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c5f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589069056:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619910.917283:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5c76e40 going to sleep for 11 seconds 00002000:00010000:3.0:1788619910.922661:0:6698:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619907 -> 1788619910 00000100:00100000:3.0:1788619910.922705:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c5f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589069056:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619910.922771:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe53687340e8 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b665 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619910.923051:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe53687340e8 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b665 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619910.923070:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683dc00, rc 0) 00000100:00100000:3.0:1788619910.923152:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa14a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571363200:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6107us (7717us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619910.923164:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 599 00000100:00100000:0.0:1788619910.923430:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed62a00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589068928:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619910.923453:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 608213 00010000:00010000:0.0:1788619910.923471:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ec00/0x1cd7fe53687340e1 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b65e expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.923487:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619910.923490:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619910.923495:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ec00/0x1cd7fe53687340e1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b65e expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.923508:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6ec00, rc 0) 00000100:00100000:0.0:1788619910.923541:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571363072:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8171us (8480us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619910.923548:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 774 00000100:00100000:1.0:1788619910.931097:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619910.931480:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619910.931546:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571363456 00000100:00100000:2.0:1788619910.931558:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571363456 00000100:00100000:2.0:1788619910.931623:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 600 00000100:00100000:2.0:1788619910.931634:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571363456:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619910.931655:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619910.931678:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3b293bd/0x1cd7fe53687340ef lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b673 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619910.931761:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cd6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589069184:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619910.931877:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccd80 going to sleep for 11 seconds 00000100:00100000:0.0:1788619910.933227:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571363328 00000100:00100000:0.0:1788619910.933241:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571363328 00000100:00100000:0.0:1788619910.933302:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 775 00000100:00100000:0.0:1788619910.933313:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571363328:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619910.933362:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619910.933385:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f200/0x1cd7fe53687340f6 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b66c expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.933431:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe536873409b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b626 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619910.933476:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed62a00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589069312:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619910.933590:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78eab85e40 going to sleep for 11 seconds 00000100:00100000:2.0:1788619910.945448:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cd6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589069184:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619910.945506:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f3b293bd/0x1cd7fe53687340ef lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b673 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619910.945540:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3b293bd/0x1cd7fe53687340ef lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b673 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619910.945578:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7e00, rc 0) 00000100:00100000:3.0:1788619910.945650:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed62a00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589069312:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619910.945672:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 608213 00000100:00100000:2.0:1788619910.945680:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571363456:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 14047us (14200us total) trans 0 rc 301/301 00010000:00010000:3.0:1788619910.945687:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f200/0x1cd7fe53687340f6 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b66c expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619910.945690:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 600 00010000:00010000:3.0:1788619910.945703:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619910.945706:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619910.945718:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f200/0x1cd7fe53687340f6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b66c expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619910.945730:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6f200, rc 0) 00000100:00100000:3.0:1788619910.945765:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571363328:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 12455us (14685us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619910.945772:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 775 00010000:00010000:3.0:1788619910.948668:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788619910.951461:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619910.951570:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571363712 00000100:00100000:0.0:1788619910.951582:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571363712 00000100:00100000:0.0:1788619910.951646:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 776 00000100:00100000:0.0:1788619910.951654:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:14908:x1875503571363712:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:0.0:1788619910.951715:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619910.952216:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe53687340b0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b634 expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.952245:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619910.952253:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788619910.952550:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fe00/0x1cd7fe53687340fd lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.952565:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e200/0x1cd7fe53687340be lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b63b expref: 8 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.952572:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fe00/0x1cd7fe53687340fd lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.952607:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e200/0x1cd7fe53687340be lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b63b expref: 8 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.952619:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e200/0x1cd7fe53687340be lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b63b expref: 8 pid: 11397 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619910.952632:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed62d80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589069440:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619910.952737:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78eab85780 going to sleep for 11 seconds 00000100:00100000:2.0:1788619910.957163:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed62d80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589069440:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619910.957213:0:5845:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fe00/0x1cd7fe53687340fd lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619910.957548:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619910.957717:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571363840 00000100:00100000:1.0:1788619910.957727:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571363840 00010000:00010000:1.0:1788619910.957859:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e200/0x1cd7fe53687340be lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b63b expref: 9 pid: 11397 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619910.958045:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 263 00000100:00100000:2.0:1788619910.958060:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571363840:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619910.958079:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619910.958088:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e200/0x1cd7fe53687340be lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b63b expref: 9 pid: 11397 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619910.958103:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e200/0x1cd7fe53687340be lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b63b expref: 9 pid: 11397 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619910.958133:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e200/0x1cd7fe53687340be lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b63b expref: 9 pid: 11397 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619910.958163:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619910.958169:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fe00/0x1cd7fe53687340fd lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619910.958182:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fe00/0x1cd7fe53687340fd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619910.958210:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fe00/0x1cd7fe53687340fd lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619910.958245:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619910.958321:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+8:2355:x1875503571363840:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 266us (775us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619910.958344:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 263 00010000:00010000:0.0:1788619910.958624:0:5845:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fe00/0x1cd7fe53687340fd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.958634:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fe00/0x1cd7fe53687340fd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788619910.959280:0:5845:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967600 00000004:00010000:0.0:1788619910.959592:0:5845:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 608213 -> 608213 00000004:00010000:0.0:1788619910.959596:0:5845:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619910 -> 1788619910 00000004:00010000:0.0:1788619910.959598:0:5845:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619910 -> 1788619908 00000004:00010000:0.0:1788619910.959600:0:5845:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619910 -> 1788619910 00000004:00010000:0.0:1788619910.959601:0:5845:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 518 -> 518 00000004:00010000:0.0:1788619910.959610:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000000165cc62 reply state 00000000f44345b4 transno 4294967601 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fe00/0x1cd7fe53687340fd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619910.959618:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788619910.959742:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+17:14908:x1875503571363712:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 8085us (8290us total) trans 4294967601 rc 0/0 00000100:00100000:0.0:1788619910.959752:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 776 00000020:00080000:0.0:1788619910.959757:0:5845:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788619910.963571:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fe00/0x1cd7fe53687340fd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.963589:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fe00/0x1cd7fe53687340fd lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.963595:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fe00/0x1cd7fe53687340fd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.963600:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c6b6fe00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788619910.963604:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fe00/0x1cd7fe53687340fd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.963629:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fe00/0x1cd7fe53687340fd lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.963645:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619910.963647:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788619910.963652:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fe00/0x1cd7fe53687340fd lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.963657:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fe00/0x1cd7fe53687340fd lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619910.964748:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619910.965015:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571363968 00000100:00100000:1.0:1788619910.965029:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571363968 00002000:00010000:1.0:1788619910.965083:0:6701:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1906088->18446744073709551615) 00002000:00100000:1.0:1788619910.965089:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c50c6300 with x1875503571363968 ext(1906088->18446744073709551615) 00002000:00010000:1.0:1788619910.965103:0:6701:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c50c6300. 00000100:00100000:1.0:1788619910.965175:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 320 00000100:00100000:1.0:1788619910.965186:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2353:x1875503571363968:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:1.0:1788619910.965620:0:6701:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 2097152 -> 1906088 00002000:00010000:1.0:1788619910.965625:0:6701:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619910 -> 1788619910 00002000:00010000:1.0:1788619910.965627:0:6701:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619910 -> 1788619907 00002000:00010000:1.0:1788619910.965630:0:6701:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619910 -> 1788619910 00002000:00010000:1.0:1788619910.965632:0:6701:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 135 -> 135 00000100:00100000:1.0:1788619910.965719:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571363968:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 528us (976us total) trans 4294967387 rc 0/0 00000100:00100000:1.0:1788619910.965729:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 320 00002000:00010000:1.0:1788619910.965734:0:6701:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1906088->18446744073709551615) 00002000:00100000:1.0:1788619910.965737:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c50c6300 with x1875503571363968 ext(1906088->18446744073709551615) 00002000:00010000:1.0:1788619910.965742:0:6701:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c50c6300. 00000100:00100000:2.0:1788619910.969013:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619910.969127:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571364096 00000100:00100000:0.0:1788619910.969139:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571364096 00000100:00100000:0.0:1788619910.969210:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 777 00000100:00100000:0.0:1788619910.969222:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+8:14908:x1875503571364096:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619910.969250:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619910.969273:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fc00/0x1cd7fe5368734104 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b67a expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.969492:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe536873410b lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.969512:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe536873410b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.969528:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe536873410b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619910.969563:0:11397:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1906088 00000004:00200000:0.0:1788619910.969569:0:11397:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619910.969613:0:11397:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe536873410b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.969645:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fc00/0x1cd7fe5368734104 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b67a expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.969665:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe536873410b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b67a expref: 9 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.969673:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619910.969676:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619910.969681:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6ee00, rc 0) 00000100:00100000:0.0:1788619910.969898:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571364096:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 665us (887us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619910.969906:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 777 00000100:00100000:3.0:1788619910.973874:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619910.973957:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619910.974072:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571364224 00000100:00100000:1.0:1788619910.974085:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571364224 00000100:00100000:1.0:1788619910.974184:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 778 00000100:00100000:1.0:1788619910.974196:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b37480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571364224:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619910.974223:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619910.974250:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4ba00/0x1cd7fe5368734112 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b681 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619910.974300:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe536873409b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b626 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619910.974384:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c71f3480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589069568:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619910.974468:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd9633c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619910.974543:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571364352 00000100:00100000:0.0:1788619910.974547:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571364352 00000100:00100000:0.0:1788619910.974573:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 601 00000100:00100000:0.0:1788619910.974578:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571364352:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619910.974586:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619910.974596:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000004390e69/0x1cd7fe5368734119 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b688 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619910.974631:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed62d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589069696:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619910.974656:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa0c0 going to sleep for 11 seconds 00002000:00010000:0.0:1788619910.976988:0:6697:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619907 -> 1788619908 00000100:00100000:0.0:1788619910.977031:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed62d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589069696:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619910.977108:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000004390e69/0x1cd7fe5368734119 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b688 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.977140:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000004390e69/0x1cd7fe5368734119 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b688 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.977154:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6fe00, rc 0) 00000100:00100000:0.0:1788619910.977233:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571364352:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2653us (3371us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619910.977240:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 601 00000100:00100000:1.0:1788619910.977370:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c71f3480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589069568:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619910.977385:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 608213 00010000:00010000:1.0:1788619910.977397:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4ba00/0x1cd7fe5368734112 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b681 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619910.977409:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619910.977411:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619910.977415:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4ba00/0x1cd7fe5368734112 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b681 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619910.977424:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f4ba00, rc 0) 00000100:00100000:1.0:1788619910.977443:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b37480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571364224:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3251us (3488us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619910.977448:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 778 00000100:00100000:3.0:1788619910.982145:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619910.982201:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619910.982285:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571364608 00000100:00100000:1.0:1788619910.982296:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571364608 00000100:00100000:1.0:1788619910.982397:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 602 00000100:00100000:1.0:1788619910.982405:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6859180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571364608:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619910.982425:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619910.982445:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000afbf5f1/0x1cd7fe5368734120 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b696 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619910.982508:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c71f3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589069824:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619910.982596:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd9633c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619910.983169:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571364480 00000100:00100000:0.0:1788619910.983175:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571364480 00000100:00100000:0.0:1788619910.983258:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 779 00000100:00100000:0.0:1788619910.983266:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c685aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571364480:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619910.983277:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619910.983291:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fc00/0x1cd7fe5368734127 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b68f expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.983318:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe536873409b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b626 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619910.983364:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed62d80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589069952:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619910.983408:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619910.986262:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed62d80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589069952:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619910.986309:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 608213 00010000:00010000:0.0:1788619910.986394:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fc00/0x1cd7fe5368734127 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b68f expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.986420:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619910.986424:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619910.986430:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fc00/0x1cd7fe5368734127 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b68f expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619910.986447:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6fc00, rc 0) 00000100:00100000:0.0:1788619910.986518:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c685aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571364480:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3252us (4396us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619910.986528:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 779 00000100:00100000:1.0:1788619910.990290:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c71f3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589069824:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619910.990483:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000afbf5f1/0x1cd7fe5368734120 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b696 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619910.990516:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000afbf5f1/0x1cd7fe5368734120 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b696 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619910.990530:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f4ae00, rc 0) 00000100:00100000:1.0:1788619910.990661:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6859180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571364608:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8254us (8459us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619910.990668:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 602 00000100:00100000:3.0:1788619910.994588:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619910.994669:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571364736 00000100:00100000:1.0:1788619910.994681:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571364736 00000100:00100000:1.0:1788619910.994744:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 603 00000100:00100000:1.0:1788619910.994756:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:14908:x1875503571364736:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619910.994779:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619910.994803:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f5d2a141/0x1cd7fe536873412e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b69d expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619910.994865:0:6698:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000cd0fba3b/0x1cd7fe53687340a2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b62d expref: 7 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619910.994875:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(7 9) 00010000:00010000:1.0:1788619910.994881:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(8 9) 00010000:00010000:1.0:1788619910.994900:0:6698:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f5d2a141/0x1cd7fe536873412e lrc: 4/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1527808->1560575] (req 1527808->1560575) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b69d expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619910.994943:0:6698:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000cd0fba3b/0x1cd7fe53687340a2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b62d expref: 7 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619910.994960:0:6698:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000cd0fba3b/0x1cd7fe53687340a2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b62d expref: 7 pid: 6698 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619910.994978:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c71f3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589070080:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619910.995081:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd9633c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619910.998546:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c71f3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589070080:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619910.998598:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f5d2a141/0x1cd7fe536873412e lrc: 3/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1527808->1560575] (req 1527808->1560575) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b69d expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619910.998627:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f4ba00, rc 0) 00000100:00100000:1.0:1788619910.998702:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:14908:x1875503571364736:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3947us (4125us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619910.998733:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 603 00000100:00100000:3.0:1788619910.999612:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619910.999879:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571364864 00000100:00100000:1.0:1788619911.000047:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571364864 00010000:00010000:1.0:1788619911.000096:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000cd0fba3b/0x1cd7fe53687340a2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b62d expref: 8 pid: 6698 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619911.000205:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 264 00000100:00100000:1.0:1788619911.000217:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8021850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571364864:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619911.000238:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619911.000267:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000cd0fba3b/0x1cd7fe53687340a2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b62d expref: 8 pid: 6698 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.000282:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000cd0fba3b/0x1cd7fe53687340a2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b62d expref: 8 pid: 6698 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.000311:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd0fba3b/0x1cd7fe53687340a2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b62d expref: 8 pid: 6698 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.000560:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(8 9) 00010000:00010000:1.0:1788619911.000565:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(8 9) 00010000:00010000:1.0:1788619911.000577:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000f5d2a141/0x1cd7fe536873412e lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1527808->1560575) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b69d expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.000590:0:5831:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f5d2a141/0x1cd7fe536873412e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1527808->1560575) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b69d expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.000663:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000f5d2a141/0x1cd7fe536873412e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1527808->1560575) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b69d expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619911.000680:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c71f3480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5831:1875503589070208:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619911.000802:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd9633c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619911.006116:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c71f3480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5831:1875503589070208:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619911.006163:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619911.006245:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8021850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571364864:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 6029us (6635us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619911.006258:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 264 00000100:00100000:2.0:1788619911.008275:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619911.009055:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571364992 00000100:00100000:3.0:1788619911.009069:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571364992 00002000:00100000:3.0:1788619911.009143:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1527808->1531903) req@ffff8e77c5cd5f80 x1875503571364992/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:7/0 lens 488/0 e 0 to 0 dl 1788619922 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619911.009167:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5cd5f80 with x1875503571364992 ext(1527808->1531903) 00002000:00010000:3.0:1788619911.010456:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5cd5f80 00000100:00100000:2.0:1788619911.010614:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 321 00000100:00100000:2.0:1788619911.010638:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571364992:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619911.011753:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571364992:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 1113us (3476us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619911.011766:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 321 00002000:00100000:2.0:1788619911.011777:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1527808->1531903) req@ffff8e77c5cd5f80 x1875503571364992/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:7/0 lens 488/4536 e 0 to 0 dl 1788619922 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619911.011787:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5cd5f80 with x1875503571364992 ext(1527808->1531903) 00002000:00010000:2.0:1788619911.011794:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5cd5f80 00000100:00100000:3.0:1788619911.016422:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619911.016998:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571365120 00000100:00100000:1.0:1788619911.017014:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571365120 00002000:00100000:1.0:1788619911.017094:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1531904->1535999) req@ffff8e78df19c700 x1875503571365120/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:7/0 lens 488/0 e 0 to 0 dl 1788619922 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619911.017107:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df19c700 with x1875503571365120 ext(1531904->1535999) 00002000:00010000:1.0:1788619911.017114:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df19c700 00000100:00100000:1.0:1788619911.017162:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 322 00000100:00100000:1.0:1788619911.017174:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571365120:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619911.019042:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571365120:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 1856us (2621us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619911.019058:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 322 00002000:00100000:1.0:1788619911.019081:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1531904->1535999) req@ffff8e78df19c700 x1875503571365120/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:7/0 lens 488/4536 e 0 to 0 dl 1788619922 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619911.019117:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df19c700 with x1875503571365120 ext(1531904->1535999) 00002000:00010000:1.0:1788619911.019124:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df19c700 00000100:00100000:3.0:1788619911.025869:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619911.026016:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571365248 00000100:00100000:2.0:1788619911.026030:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571365248 00002000:00100000:2.0:1788619911.026104:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1536000->1540095) req@ffff8e78df19dc00 x1875503571365248/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:7/0 lens 488/0 e 0 to 0 dl 1788619922 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619911.026114:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df19dc00 with x1875503571365248 ext(1536000->1540095) 00002000:00010000:2.0:1788619911.026125:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df19dc00 00000100:00100000:2.0:1788619911.026169:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 323 00000100:00100000:2.0:1788619911.026201:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571365248:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619911.026558:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571365248:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 358us (703us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619911.026564:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 323 00002000:00100000:2.0:1788619911.026569:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1536000->1540095) req@ffff8e78df19dc00 x1875503571365248/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:7/0 lens 488/4536 e 0 to 0 dl 1788619922 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619911.026574:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df19dc00 with x1875503571365248 ext(1536000->1540095) 00002000:00010000:2.0:1788619911.026577:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df19dc00 00000100:00100000:2.0:1788619911.030553:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619911.030875:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571365376 00000100:00100000:1.0:1788619911.030904:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571365376 00002000:00100000:1.0:1788619911.030976:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1540096->1544191) req@ffff8e77c5cd7100 x1875503571365376/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:7/0 lens 488/0 e 0 to 0 dl 1788619922 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619911.030989:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5cd7100 with x1875503571365376 ext(1540096->1544191) 00002000:00010000:1.0:1788619911.030996:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5cd7100 00000100:00100000:1.0:1788619911.031037:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 324 00000100:00100000:1.0:1788619911.031049:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571365376:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619911.032530:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571365376:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 1471us (1977us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619911.032543:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 324 00002000:00100000:1.0:1788619911.032553:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1540096->1544191) req@ffff8e77c5cd7100 x1875503571365376/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:7/0 lens 488/4536 e 0 to 0 dl 1788619922 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619911.032562:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5cd7100 with x1875503571365376 ext(1540096->1544191) 00002000:00010000:1.0:1788619911.032567:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5cd7100 00000100:00100000:3.0:1788619911.035752:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619911.036160:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571365504 00000100:00100000:2.0:1788619911.036172:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571365504 00002000:00100000:2.0:1788619911.036239:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1544192->1548287) req@ffff8e78df19c380 x1875503571365504/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:7/0 lens 488/0 e 0 to 0 dl 1788619922 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619911.036252:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df19c380 with x1875503571365504 ext(1544192->1548287) 00002000:00010000:2.0:1788619911.036259:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df19c380 00000100:00100000:2.0:1788619911.036304:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 325 00000100:00100000:2.0:1788619911.036316:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571365504:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619911.037346:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571365504:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 1003us (1576us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619911.037356:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 325 00002000:00100000:2.0:1788619911.037364:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1544192->1548287) req@ffff8e78df19c380 x1875503571365504/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:7/0 lens 488/4536 e 0 to 0 dl 1788619922 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619911.037371:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df19c380 with x1875503571365504 ext(1544192->1548287) 00002000:00010000:2.0:1788619911.037375:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df19c380 00000100:00100000:2.0:1788619911.040789:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619911.042962:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571365632 00000100:00100000:1.0:1788619911.042983:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571365632 00002000:00100000:1.0:1788619911.043074:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1548288->1552383) req@ffff8e77c5cd4700 x1875503571365632/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:7/0 lens 488/0 e 0 to 0 dl 1788619922 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619911.043088:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5cd4700 with x1875503571365632 ext(1548288->1552383) 00002000:00010000:1.0:1788619911.043096:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5cd4700 00000100:00100000:1.0:1788619911.043166:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 326 00000100:00100000:1.0:1788619911.043178:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571365632:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619911.043561:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571365632:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 383us (2786us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619911.043572:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 326 00002000:00100000:1.0:1788619911.043579:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1548288->1552383) req@ffff8e77c5cd4700 x1875503571365632/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:7/0 lens 488/4536 e 0 to 0 dl 1788619922 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619911.043587:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5cd4700 with x1875503571365632 ext(1548288->1552383) 00002000:00010000:1.0:1788619911.043591:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5cd4700 00000100:00100000:3.0:1788619911.048849:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619911.053959:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571365760 00000100:00100000:2.0:1788619911.053982:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571365760 00002000:00100000:2.0:1788619911.054068:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1552384->1556479) req@ffff8e7900e00e00 x1875503571365760/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:7/0 lens 488/0 e 0 to 0 dl 1788619922 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619911.054082:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e7900e00e00 with x1875503571365760 ext(1552384->1556479) 00002000:00010000:2.0:1788619911.054095:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e7900e00e00 00000100:00100000:2.0:1788619911.054144:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 327 00000100:00100000:2.0:1788619911.054156:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e00e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571365760:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619911.054573:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e00e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571365760:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 416us (5738us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619911.054583:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 327 00002000:00100000:2.0:1788619911.054591:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1552384->1556479) req@ffff8e7900e00e00 x1875503571365760/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:7/0 lens 488/4536 e 0 to 0 dl 1788619922 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619911.054601:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e7900e00e00 with x1875503571365760 ext(1552384->1556479) 00002000:00010000:2.0:1788619911.054605:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e7900e00e00 00000100:00100000:3.0:1788619911.063050:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619911.063590:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571365888 00000100:00100000:0.0:1788619911.063605:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571365888 00002000:00100000:0.0:1788619911.063673:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1556480->1560575) req@ffff8e7900e00380 x1875503571365888/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:7/0 lens 488/0 e 0 to 0 dl 1788619922 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619911.063686:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e7900e00380 with x1875503571365888 ext(1556480->1560575) 00002000:00010000:0.0:1788619911.063692:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e7900e00380 00000100:00100000:0.0:1788619911.063738:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 328 00000100:00100000:0.0:1788619911.063749:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e00380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571365888:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619911.064303:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e00380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571365888:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 552us (1266us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619911.064312:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 328 00002000:00100000:0.0:1788619911.064320:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1556480->1560575) req@ffff8e7900e00380 x1875503571365888/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:7/0 lens 488/4536 e 0 to 0 dl 1788619922 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619911.064377:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e7900e00380 with x1875503571365888 ext(1556480->1560575) 00002000:00010000:0.0:1788619911.064392:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e7900e00380 00000100:00100000:0.0:1788619911.070297:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619911.071846:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571366016 00000100:00100000:2.0:1788619911.071865:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571366016 00000100:00100000:2.0:1788619911.071966:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 780 00000100:00100000:2.0:1788619911.071981:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed62d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571366016:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619911.072010:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619911.072040:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7a00/0x1cd7fe5368734135 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b6a4 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.072089:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe536873409b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b626 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619911.072175:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cd4000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589070336:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619911.072296:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc000 going to sleep for 11 seconds 00000100:00100000:2.0:1788619911.077452:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cd4000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589070336:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619911.077494:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 608213 00010000:00010000:2.0:1788619911.077520:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7a00/0x1cd7fe5368734135 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6a4 expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.077548:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619911.077551:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619911.077558:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7a00/0x1cd7fe5368734135 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6a4 expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.077578:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7a00, rc 0) 00000100:00100000:2.0:1788619911.077658:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed62d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571366016:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5681us (7376us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619911.077670:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 780 00000100:00100000:2.0:1788619911.083430:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619911.083587:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571366144 00000100:00100000:1.0:1788619911.083601:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571366144 00000100:00100000:1.0:1788619911.083667:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 781 00000100:00100000:1.0:1788619911.083678:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571366144:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619911.085150:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619911.085263:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4ac00/0x1cd7fe536873413c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b6ab expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.085513:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe536873409b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b626 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619911.085574:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c71f3480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589070464:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619911.089196:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd9636c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619911.092610:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c71f3480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589070464:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619911.092650:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 608213 00010000:00010000:1.0:1788619911.092678:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4ac00/0x1cd7fe536873413c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6ab expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.092733:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619911.092737:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619911.092744:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4ac00/0x1cd7fe536873413c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6ab expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.092764:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f4ac00, rc 0) 00000100:00100000:1.0:1788619911.092847:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571366144:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 9169us (9430us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619911.092859:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 781 00000100:00100000:3.0:1788619911.111421:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619911.112028:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571366272 00000100:00100000:0.0:1788619911.112044:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571366272 00000100:00100000:0.0:1788619911.112124:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 782 00000100:00100000:0.0:1788619911.112136:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ce0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571366272:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619911.112243:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619911.112272:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368734143 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b6b2 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619911.112492:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe536873409b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b626 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619911.112554:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed62a00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589070592:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619911.112683:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa240 going to sleep for 11 seconds 00000100:00100000:0.0:1788619911.116381:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed62a00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589070592:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619911.116698:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 608213 00010000:00010000:0.0:1788619911.116741:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368734143 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6b2 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619911.116777:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619911.116781:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619911.117233:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368734143 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6b2 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619911.117257:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6e000, rc 0) 00000100:00100000:0.0:1788619911.118096:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ce0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571366272:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5275us (6013us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619911.118129:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 782 00000100:00100000:2.0:1788619911.123619:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619911.124050:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571366400 00000100:00100000:0.0:1788619911.124065:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571366400 00000100:00100000:0.0:1788619911.124206:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 604 00000100:00100000:0.0:1788619911.124218:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:14908:x1875503571366400:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619911.124243:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619911.124271:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000084d1816c/0x1cd7fe536873414a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b6b9 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619911.124313:0:6696:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000f5d2a141/0x1cd7fe536873412e lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1527808->1560575) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b69d expref: 7 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619911.124323:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(8 9) 00010000:00010000:0.0:1788619911.124371:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(8 9) 00010000:00010000:0.0:1788619911.124377:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000084d1816c/0x1cd7fe536873414a lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1552384->1556479] (req 1552384->1556479) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6b9 expref: 7 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619911.124432:0:6696:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000f5d2a141/0x1cd7fe536873412e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1527808->1560575) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b69d expref: 7 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619911.124448:0:6696:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000f5d2a141/0x1cd7fe536873412e lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1527808->1560575) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b69d expref: 7 pid: 6698 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619911.124465:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed62a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589070720:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619911.124574:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa240 going to sleep for 11 seconds 00000100:00100000:3.0:1788619911.137655:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619911.137774:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571366528 00000100:00100000:2.0:1788619911.137787:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571366528 00010000:00010000:2.0:1788619911.137839:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000f5d2a141/0x1cd7fe536873412e lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1527808->1560575) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b69d expref: 8 pid: 6698 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619911.137900:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 265 00000100:00100000:2.0:1788619911.137912:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ce1880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+8:2355:x1875503571366528:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619911.137955:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619911.137985:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000f5d2a141/0x1cd7fe536873412e lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1527808->1560575) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b69d expref: 8 pid: 6698 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.138001:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000f5d2a141/0x1cd7fe536873412e lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1527808->1560575) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b69d expref: 8 pid: 6698 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.138034:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(8 9) 00010000:00010000:2.0:1788619911.138049:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(8 9) 00010000:00010000:2.0:1788619911.138064:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000084d1816c/0x1cd7fe536873414a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1552384->1556479) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6b9 expref: 8 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.138076:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000084d1816c/0x1cd7fe536873414a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1552384->1556479) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6b9 expref: 8 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.138127:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000084d1816c/0x1cd7fe536873414a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1552384->1556479) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6b9 expref: 8 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619911.138146:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cd5880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589070848:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619911.138267:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc000 going to sleep for 11 seconds 00010000:00010000:0.0:1788619911.138591:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f5d2a141/0x1cd7fe536873412e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1527808->1560575) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b69d expref: 8 pid: 6698 timeout: 503 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619911.138613:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed62a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589070720:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619911.138650:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000084d1816c/0x1cd7fe536873414a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1552384->1556479) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6b9 expref: 7 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619911.138661:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6e000, rc 0) 00000100:00100000:0.0:1788619911.138697:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:14908:x1875503571366400:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 14482us (15096us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619911.138705:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 604 00000100:00100000:2.0:1788619911.142506:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cd5880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589070848:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619911.142548:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619911.142626:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ce1880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571366528:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 4715us (4985us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619911.142639:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 265 00000100:00100000:3.0:1788619911.157243:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619911.161009:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571366656 00000100:00100000:2.0:1788619911.161030:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571366656 00002000:00100000:2.0:1788619911.161241:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1552384->1556479) req@ffff8e77c5ce2a00 x1875503571366656/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:7/0 lens 488/0 e 0 to 0 dl 1788619922 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619911.161255:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5ce2a00 with x1875503571366656 ext(1552384->1556479) 00002000:00010000:2.0:1788619911.161264:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5ce2a00 00000100:00100000:2.0:1788619911.161433:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 329 00000100:00100000:2.0:1788619911.161446:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ce2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571366656:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619911.161846:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ce2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571366656:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 402us (4620us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619911.161855:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 329 00002000:00100000:2.0:1788619911.161864:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1552384->1556479) req@ffff8e77c5ce2a00 x1875503571366656/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:7/0 lens 488/4536 e 0 to 0 dl 1788619922 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619911.161872:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5ce2a00 with x1875503571366656 ext(1552384->1556479) 00002000:00010000:2.0:1788619911.161876:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5ce2a00 00000100:00100000:3.0:1788619911.167432:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619911.167789:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571366784 00000100:00100000:1.0:1788619911.167803:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571366784 00002000:00100000:1.0:1788619911.167867:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1556480->1560575) req@ffff8e77c5ce3b80 x1875503571366784/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:7/0 lens 488/0 e 0 to 0 dl 1788619922 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619911.167881:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5ce3b80 with x1875503571366784 ext(1556480->1560575) 00002000:00010000:1.0:1788619911.167897:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5ce3b80 00000100:00100000:1.0:1788619911.167927:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 330 00000100:00100000:1.0:1788619911.167938:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ce3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571366784:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619911.168530:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ce3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571366784:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 590us (1122us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619911.168539:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 330 00002000:00100000:1.0:1788619911.168547:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1556480->1560575) req@ffff8e77c5ce3b80 x1875503571366784/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:7/0 lens 488/4536 e 0 to 0 dl 1788619922 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619911.168556:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5ce3b80 with x1875503571366784 ext(1556480->1560575) 00002000:00010000:1.0:1788619911.168569:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5ce3b80 00000100:00100000:3.0:1788619911.176258:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619911.177668:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571366912 00000100:00100000:2.0:1788619911.177686:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571366912 00002000:00100000:2.0:1788619911.178276:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1560576->1564671) req@ffff8e77c5ce2680 x1875503571366912/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:7/0 lens 488/0 e 0 to 0 dl 1788619922 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619911.178290:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5ce2680 with x1875503571366912 ext(1560576->1564671) 00002000:00010000:2.0:1788619911.178298:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5ce2680 00000100:00100000:2.0:1788619911.178428:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 331 00000100:00100000:2.0:1788619911.178441:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ce2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571366912:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619911.178827:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ce2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571366912:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 387us (2584us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619911.178853:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 331 00002000:00100000:2.0:1788619911.178861:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1560576->1564671) req@ffff8e77c5ce2680 x1875503571366912/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:7/0 lens 488/4536 e 0 to 0 dl 1788619922 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619911.178871:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5ce2680 with x1875503571366912 ext(1560576->1564671) 00002000:00010000:2.0:1788619911.178875:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5ce2680 00000100:00100000:3.0:1788619911.184474:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619911.184603:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571367040 00000100:00100000:0.0:1788619911.184617:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571367040 00002000:00100000:0.0:1788619911.184685:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1564672->1568767) req@ffff8e77c5ce1c00 x1875503571367040/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:7/0 lens 488/0 e 0 to 0 dl 1788619922 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619911.184698:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5ce1c00 with x1875503571367040 ext(1564672->1568767) 00002000:00010000:0.0:1788619911.184706:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5ce1c00 00000100:00100000:0.0:1788619911.185214:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 332 00000100:00100000:0.0:1788619911.185228:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ce1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571367040:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619911.185843:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ce1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571367040:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 613us (1379us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619911.185856:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 332 00002000:00100000:0.0:1788619911.185866:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1564672->1568767) req@ffff8e77c5ce1c00 x1875503571367040/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:7/0 lens 488/4536 e 0 to 0 dl 1788619922 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619911.185902:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5ce1c00 with x1875503571367040 ext(1564672->1568767) 00002000:00010000:0.0:1788619911.185909:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5ce1c00 00000100:00100000:3.0:1788619911.197671:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619911.197875:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571367168 00000100:00100000:2.0:1788619911.197906:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571367168 00002000:00100000:2.0:1788619911.197981:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1568768->1572863) req@ffff8e77c5ce0e00 x1875503571367168/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:7/0 lens 488/0 e 0 to 0 dl 1788619922 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619911.197994:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5ce0e00 with x1875503571367168 ext(1568768->1572863) 00002000:00010000:2.0:1788619911.198000:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5ce0e00 00000100:00100000:2.0:1788619911.198047:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 333 00000100:00100000:2.0:1788619911.198058:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ce0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571367168:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619911.198466:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ce0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571367168:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 406us (807us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619911.198475:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 333 00002000:00100000:2.0:1788619911.198483:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1568768->1572863) req@ffff8e77c5ce0e00 x1875503571367168/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:7/0 lens 488/4536 e 0 to 0 dl 1788619922 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619911.198493:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5ce0e00 with x1875503571367168 ext(1568768->1572863) 00002000:00010000:2.0:1788619911.198497:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5ce0e00 00000100:00100000:3.0:1788619911.203123:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619911.205133:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571367296 00000100:00100000:0.0:1788619911.205154:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571367296 00002000:00100000:0.0:1788619911.205310:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1572864->1576959) req@ffff8e77c5ce1500 x1875503571367296/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:7/0 lens 488/0 e 0 to 0 dl 1788619922 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619911.205322:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5ce1500 with x1875503571367296 ext(1572864->1576959) 00002000:00010000:0.0:1788619911.205349:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5ce1500 00000100:00100000:0.0:1788619911.205394:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 334 00000100:00100000:0.0:1788619911.205416:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ce1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571367296:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619911.206271:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ce1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571367296:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 861us (3156us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619911.206283:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 334 00002000:00100000:0.0:1788619911.206293:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1572864->1576959) req@ffff8e77c5ce1500 x1875503571367296/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:7/0 lens 488/4536 e 0 to 0 dl 1788619922 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619911.206303:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5ce1500 with x1875503571367296 ext(1572864->1576959) 00002000:00010000:0.0:1788619911.206308:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5ce1500 00000100:00100000:3.0:1788619911.215475:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619911.215751:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571367424 00000100:00100000:1.0:1788619911.215766:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571367424 00002000:00100000:1.0:1788619911.216010:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1576960->1581055) req@ffff8e77c5ce3480 x1875503571367424/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:7/0 lens 488/0 e 0 to 0 dl 1788619922 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619911.216024:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5ce3480 with x1875503571367424 ext(1576960->1581055) 00002000:00010000:1.0:1788619911.216033:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5ce3480 00000100:00100000:1.0:1788619911.216079:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 335 00000100:00100000:1.0:1788619911.216091:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ce3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571367424:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619911.217017:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ce3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571367424:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 921us (1552us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619911.217030:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 335 00002000:00100000:1.0:1788619911.217052:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1576960->1581055) req@ffff8e77c5ce3480 x1875503571367424/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:7/0 lens 488/4536 e 0 to 0 dl 1788619922 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619911.217063:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5ce3480 with x1875503571367424 ext(1576960->1581055) 00002000:00010000:1.0:1788619911.217069:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5ce3480 00000100:00100000:3.0:1788619911.224587:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619911.224865:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571367552 00000100:00100000:0.0:1788619911.224879:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571367552 00002000:00100000:0.0:1788619911.224959:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1581056->1585151) req@ffff8e77c5ce1180 x1875503571367552/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:7/0 lens 488/0 e 0 to 0 dl 1788619922 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619911.224971:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5ce1180 with x1875503571367552 ext(1581056->1585151) 00002000:00010000:0.0:1788619911.224977:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5ce1180 00000100:00100000:0.0:1788619911.225044:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 336 00000100:00100000:0.0:1788619911.225055:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ce1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571367552:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619911.227132:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ce1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571367552:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 2064us (2544us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619911.227145:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 336 00002000:00100000:0.0:1788619911.227160:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1581056->1585151) req@ffff8e77c5ce1180 x1875503571367552/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:7/0 lens 488/4536 e 0 to 0 dl 1788619922 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619911.228142:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5ce1180 with x1875503571367552 ext(1581056->1585151) 00002000:00010000:0.0:1788619911.228166:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5ce1180 00000100:00100000:3.0:1788619911.231453:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619911.231537:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571367680 00000100:00100000:2.0:1788619911.231551:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571367680 00002000:00100000:2.0:1788619911.231629:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1585152->1589247) req@ffff8e77c5ce3800 x1875503571367680/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:7/0 lens 488/0 e 0 to 0 dl 1788619922 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619911.231667:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5ce3800 with x1875503571367680 ext(1585152->1589247) 00002000:00010000:2.0:1788619911.231674:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5ce3800 00000100:00100000:2.0:1788619911.231735:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 337 00000100:00100000:2.0:1788619911.231747:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ce3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571367680:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619911.232223:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ce3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571367680:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 476us (785us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619911.232232:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 337 00002000:00100000:2.0:1788619911.232241:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1585152->1589247) req@ffff8e77c5ce3800 x1875503571367680/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:7/0 lens 488/4536 e 0 to 0 dl 1788619922 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619911.232249:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5ce3800 with x1875503571367680 ext(1585152->1589247) 00002000:00010000:2.0:1788619911.232254:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5ce3800 00000100:00100000:3.0:1788619911.241543:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619911.241903:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571367808 00000100:00100000:0.0:1788619911.241918:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571367808 00002000:00100000:0.0:1788619911.241999:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1589248->1593343) req@ffff8e77c5ce0a80 x1875503571367808/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:7/0 lens 488/0 e 0 to 0 dl 1788619922 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619911.242013:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5ce0a80 with x1875503571367808 ext(1589248->1593343) 00002000:00010000:0.0:1788619911.242020:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5ce0a80 00000100:00100000:0.0:1788619911.242130:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 338 00000100:00100000:0.0:1788619911.242142:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ce0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571367808:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619911.243109:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ce0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571367808:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 959us (1568us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619911.243122:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 338 00002000:00100000:1.0:1788619911.243132:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1589248->1593343) req@ffff8e77c5ce0a80 x1875503571367808/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:7/0 lens 488/4536 e 0 to 0 dl 1788619922 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619911.243142:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5ce0a80 with x1875503571367808 ext(1589248->1593343) 00002000:00010000:1.0:1788619911.243148:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5ce0a80 00000100:00100000:3.0:1788619911.249598:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619911.249950:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571367936 00000100:00100000:1.0:1788619911.249985:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571367936 00002000:00100000:1.0:1788619911.250054:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1593344->1597439) req@ffff8e77c5ce2d80 x1875503571367936/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:7/0 lens 488/0 e 0 to 0 dl 1788619922 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619911.250068:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5ce2d80 with x1875503571367936 ext(1593344->1597439) 00002000:00010000:1.0:1788619911.250075:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5ce2d80 00000100:00100000:2.0:1788619911.250163:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 339 00000100:00100000:2.0:1788619911.250180:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ce2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571367936:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619911.250604:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ce2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571367936:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 429us (1021us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619911.250614:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 339 00002000:00100000:2.0:1788619911.250621:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1593344->1597439) req@ffff8e77c5ce2d80 x1875503571367936/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:7/0 lens 488/4536 e 0 to 0 dl 1788619922 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619911.250629:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5ce2d80 with x1875503571367936 ext(1593344->1597439) 00002000:00010000:2.0:1788619911.250633:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5ce2d80 00000100:00100000:3.0:1788619911.267312:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619911.267970:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571368064 00000100:00100000:1.0:1788619911.267988:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571368064 00002000:00100000:1.0:1788619911.268068:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1597440->1601535) req@ffff8e77c5ce2300 x1875503571368064/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:7/0 lens 488/0 e 0 to 0 dl 1788619922 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619911.268081:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5ce2300 with x1875503571368064 ext(1597440->1601535) 00002000:00010000:1.0:1788619911.268089:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5ce2300 00000100:00100000:1.0:1788619911.271040:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 340 00000100:00100000:1.0:1788619911.271074:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ce2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571368064:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619911.271487:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ce2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571368064:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 429us (6730us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619911.271496:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 340 00002000:00100000:1.0:1788619911.271508:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1597440->1601535) req@ffff8e77c5ce2300 x1875503571368064/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:7/0 lens 488/4536 e 0 to 0 dl 1788619922 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619911.271520:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5ce2300 with x1875503571368064 ext(1597440->1601535) 00002000:00010000:1.0:1788619911.271528:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5ce2300 00000100:00100000:3.0:1788619911.299394:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619911.300430:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571368192 00000100:00100000:2.0:1788619911.300449:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571368192 00002000:00100000:2.0:1788619911.300550:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1601536->1605631) req@ffff8e77c5ce1f80 x1875503571368192/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:7/0 lens 488/0 e 0 to 0 dl 1788619922 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619911.300563:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5ce1f80 with x1875503571368192 ext(1601536->1605631) 00002000:00010000:2.0:1788619911.300571:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5ce1f80 00000100:00100000:2.0:1788619911.300605:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 341 00000100:00100000:2.0:1788619911.300616:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ce1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571368192:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619911.302641:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ce1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571368192:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 2012us (3248us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619911.302658:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 341 00002000:00100000:2.0:1788619911.302670:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1601536->1605631) req@ffff8e77c5ce1f80 x1875503571368192/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:7/0 lens 488/4536 e 0 to 0 dl 1788619922 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619911.302680:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5ce1f80 with x1875503571368192 ext(1601536->1605631) 00002000:00010000:2.0:1788619911.302688:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5ce1f80 00000100:00100000:0.0:1788619911.312065:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619911.313091:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571368320 00000100:00100000:1.0:1788619911.313107:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571368320 00002000:00100000:1.0:1788619911.313429:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1552384->1605631) req@ffff8e78fed62a00 x1875503571368320/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:7/0 lens 488/0 e 0 to 0 dl 1788619922 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619911.313442:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fed62a00 with x1875503571368320 ext(1552384->1605631) 00002000:00010000:1.0:1788619911.313460:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fed62a00 00000100:00100000:1.0:1788619911.313506:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 342 00000100:00100000:1.0:1788619911.313518:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed62a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571368320:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:1.0:1788619911.323730:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed62a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+8:2353:x1875503571368320:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 10196us (11665us total) trans 4294967388 rc 0/0 00000100:00100000:1.0:1788619911.323749:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 342 00002000:00100000:1.0:1788619911.323764:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1552384->1605631) req@ffff8e78fed62a00 x1875503571368320/t4294967388(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:7/0 lens 488/448 e 0 to 0 dl 1788619922 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619911.324665:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fed62a00 with x1875503571368320 ext(1552384->1605631) 00002000:00010000:1.0:1788619911.324674:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fed62a00 00000100:00100000:2.0:1788619911.327449:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619911.327617:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571368448 00000100:00100000:0.0:1788619911.327629:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571368448 00000100:00100000:0.0:1788619911.327697:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 605 00000100:00100000:0.0:1788619911.327707:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+8:2352:x1875503571368448:12345-192.168.201.5@tcp:16:fsx.0 00000001:00080000:1.0:1788619911.436136:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967386 is committed 00000001:00080000:1.0:1788619911.436496:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967388 is committed 00000001:00080000:1.0:1788619911.436511:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967387 is committed 00000100:00100000:0.0:1788619911.436949:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571368448:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 108999us (109270us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619911.436987:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 605 00000100:00100000:0.0:1788619911.442578:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619911.445308:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571368576 00000100:00100000:2.0:1788619911.445344:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571368576 00000100:00100000:2.0:1788619911.445449:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 783 00000100:00100000:2.0:1788619911.445463:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571368576:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619911.445491:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619911.445519:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7600/0x1cd7fe5368734151 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b6c0 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.445567:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe536873409b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b626 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619911.445648:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cd6a00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589070976:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619911.446323:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc300 going to sleep for 11 seconds 00000100:00100000:2.0:1788619911.451441:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cd6a00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589070976:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619911.451478:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 608213 00010000:00010000:2.0:1788619911.451503:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7600/0x1cd7fe5368734151 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6c0 expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.451529:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619911.451532:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619911.451539:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7600/0x1cd7fe5368734151 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6c0 expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.451557:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7600, rc 0) 00000100:00100000:2.0:1788619911.451679:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571368576:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6218us (9119us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619911.451690:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 783 00000100:00100000:3.0:1788619911.460680:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619911.461561:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571368704 00000100:00100000:1.0:1788619911.461577:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571368704 00000100:00100000:0.0:1788619911.463290:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 784 00000100:00100000:0.0:1788619911.463319:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ce3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571368704:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619911.463372:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619911.463405:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e600/0x1cd7fe5368734158 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b6c7 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619911.463459:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe536873409b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b626 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619911.463520:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c1f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589071104:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619911.463719:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619911.475103:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c1f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589071104:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619911.475140:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 608213 00010000:00010000:3.0:1788619911.475247:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e600/0x1cd7fe5368734158 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6c7 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619911.475279:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619911.475282:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619911.475289:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e600/0x1cd7fe5368734158 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6c7 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619911.475306:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6e600, rc 0) 00000100:00100000:3.0:1788619911.475438:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ce3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571368704:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 12132us (14772us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619911.475467:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 784 00000100:00100000:2.0:1788619911.484581:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619911.484699:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571368960 00000100:00100000:0.0:1788619911.484787:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571368960 00000100:00100000:0.0:1788619911.484926:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 785 00000100:00100000:0.0:1788619911.484939:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571368960:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:0.0:1788619911.485021:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619911.485061:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe536873410b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b67a expref: 9 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619911.485089:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619911.485096:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788619911.485611:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe536873415f lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619911.485653:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe536873415f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619911.485665:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe536873415f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788619911.485750:0:5845:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967602 00000004:00010000:0.0:1788619911.485778:0:5845:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 608213 -> 608213 00000004:00010000:0.0:1788619911.485782:0:5845:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619910 -> 1788619911 00000004:00010000:0.0:1788619911.485785:0:5845:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619908 -> 1788619908 00000004:00010000:0.0:1788619911.485787:0:5845:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619910 -> 1788619911 00000004:00010000:0.0:1788619911.485790:0:5845:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 518 -> 518 00000004:00010000:0.0:1788619911.485798:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000004e16ba7d reply state 0000000069686a0e transno 4294967603 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe536873415f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619911.485809:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788619911.486556:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571368960:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 1613us (1983us total) trans 4294967603 rc 0/0 00000100:00100000:0.0:1788619911.486567:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 785 00000020:00080000:0.0:1788619911.486574:0:5845:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1788619911.489692:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619911.489841:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571369088 00000100:00100000:3.0:1788619911.489857:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571369088 00000100:00100000:3.0:1788619911.489952:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 786 00000100:00100000:3.0:1788619911.489966:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571369088:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619911.490002:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619911.490031:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368734166 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b6ce expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619911.490072:0:11397:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe536873409b lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b626 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619911.490085:0:11397:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368734166 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6ce expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619911.490134:0:11397:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe536873409b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b626 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619911.490158:0:11397:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe536873409b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b626 expref: 15 pid: 5846 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619911.490368:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5ce0700 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589071232:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619911.490496:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5c76e40 going to sleep for 11 seconds 00010000:00010000:2.0:1788619911.492136:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe536873415f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.492173:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe536873415f lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.492183:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe536873415f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.492190:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c6b6ea00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:2.0:1788619911.492222:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe536873415f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.492233:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe536873415f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.492254:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619911.492258:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788619911.492264:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe536873415f lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.492271:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe536873415f lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619911.496011:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5ce0700 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589071232:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619911.496059:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368734166 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6ce expref: 11 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.496082:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619911.496085:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619911.496090:0:11397:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368734166 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6ce expref: 11 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.496102:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d000, rc 0) 00000100:00100000:2.0:1788619911.496297:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571369088:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6332us (6621us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619911.496307:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 786 00000100:00100000:0.0:1788619911.500684:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619911.501468:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571369216 00000100:00100000:3.0:1788619911.501489:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571369216 00010000:00010000:3.0:1788619911.501561:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe536873409b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b626 expref: 16 pid: 5846 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619911.501645:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 266 00000100:00100000:3.0:1788619911.501659:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c77e0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571369216:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619911.501687:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619911.501958:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe536873409b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b626 expref: 16 pid: 5846 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619911.501977:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe536873409b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b626 expref: 16 pid: 5846 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619911.502009:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe536873409b lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b626 expref: 16 pid: 5846 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619911.502031:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619911.502039:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368734166 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6ce expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619911.502052:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368734166 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6ce expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619911.502063:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368734166 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6ce expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619911.502122:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368734166 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6ce expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619911.503136:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5ce0000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589071360:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619911.504109:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5c76480 going to sleep for 11 seconds 00000100:00100000:1.0:1788619911.509442:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5ce0000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589071360:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619911.509480:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619911.509560:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c77e0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571369216:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 7902us (8890us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619911.509573:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 266 00000100:00100000:0.0:1788619911.514444:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619911.514492:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619911.514603:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571369472 00000100:00100000:1.0:1788619911.514617:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571369472 00002000:00010000:1.0:1788619911.514677:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:1.0:1788619911.514683:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c77e3480 with x1875503571369472 ext(0->18446744073709551615) 00002000:00010000:1.0:1788619911.514697:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c77e3480. 00000100:00100000:1.0:1788619911.514739:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 343 00000100:00100000:1.0:1788619911.514768:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c77e3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571369472:12345-192.168.201.5@tcp:10:fsx.0 00000100:00100000:2.0:1788619911.515016:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571369344 00000100:00100000:2.0:1788619911.515028:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571369344 00002000:00010000:2.0:1788619911.515058:0:5857:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:2.0:1788619911.515064:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c77e1500 with x1875503571369344 00000100:00100000:2.0:1788619911.515092:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 450 00000100:00100000:2.0:1788619911.515100:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c77e1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571369344:12345-192.168.201.5@tcp:10:fsx.0 00000004:00010000:2.0:1788619911.515602:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 608213 -> 311952 00000004:00010000:2.0:1788619911.515608:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619911 -> 1788619911 00000004:00010000:2.0:1788619911.515611:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619908 -> 1788619908 00000004:00010000:2.0:1788619911.515613:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619911 -> 1788619911 00000004:00010000:2.0:1788619911.515615:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 518 -> 518 00000100:00100000:2.0:1788619911.515708:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c77e1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571369344:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 607us (1280us total) trans 4294967604 rc 0/0 00000100:00100000:2.0:1788619911.515718:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 450 00002000:00010000:2.0:1788619911.515722:0:5857:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:2.0:1788619911.515724:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c77e1500 with x1875503571369344 00002000:00010000:1.0:1788619911.517660:0:6702:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1906088 -> 0 00002000:00010000:1.0:1788619911.517674:0:6702:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619910 -> 1788619911 00002000:00010000:1.0:1788619911.517677:0:6702:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619908 -> 1788619908 00002000:00010000:1.0:1788619911.517679:0:6702:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619910 -> 1788619911 00002000:00010000:1.0:1788619911.517682:0:6702:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 135 -> 263 00000100:00100000:1.0:1788619911.517765:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c77e3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571369472:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 2991us (3264us total) trans 4294967389 rc 0/0 00000100:00100000:1.0:1788619911.517777:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 343 00002000:00010000:1.0:1788619911.517783:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:1.0:1788619911.517786:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c77e3480 with x1875503571369472 ext(0->18446744073709551615) 00002000:00010000:1.0:1788619911.517794:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c77e3480. 00000100:00100000:2.0:1788619911.528220:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619911.528495:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571369600 00000100:00100000:3.0:1788619911.528510:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571369600 00000100:00100000:3.0:1788619911.528594:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 787 00000100:00100000:3.0:1788619911.528608:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:14908:x1875503571369600:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619911.528643:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619911.528688:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe536873416d lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b6d5 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619911.528806:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368734174 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619911.528825:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368734174 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619911.528840:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368734174 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788619911.528875:0:5846:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 311952 00000004:00200000:3.0:1788619911.529020:0:5846:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788619911.529049:0:5846:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368734174 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619911.529086:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe536873416d lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6d5 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619911.529110:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368734174 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6d5 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619911.529144:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619911.529147:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788619911.529152:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c200, rc 0) 00000100:00100000:3.0:1788619911.529597:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+13:14908:x1875503571369600:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 721us (1125us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619911.529607:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 787 00000100:00100000:2.0:1788619911.539652:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619911.539765:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571369728 00000100:00100000:1.0:1788619911.539778:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571369728 00000100:00100000:1.0:1788619911.539846:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 606 00000100:00100000:1.0:1788619911.539857:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:14908:x1875503571369728:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619911.539912:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619911.539939:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000afbf5f1/0x1cd7fe536873417b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b6dc expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.539986:0:7654:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000084d1816c/0x1cd7fe536873414a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1552384->1556479) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6b9 expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.539996:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=1s v=(8 9) 00010000:00010000:1.0:1788619911.540001:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(1 8) 00010000:00010000:1.0:1788619911.540007:0:7654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000afbf5f1/0x1cd7fe536873417b lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1929216->1941503] (req 1929216->1941503) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6dc expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.540069:0:7654:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000084d1816c/0x1cd7fe536873414a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1552384->1556479) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6b9 expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.540084:0:7654:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000084d1816c/0x1cd7fe536873414a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1552384->1556479) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6b9 expref: 6 pid: 6696 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619911.540101:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c71f3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589071488:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619911.540217:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963a80 going to sleep for 11 seconds 00000100:00100000:0.0:1788619911.547555:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619911.547727:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571369856 00000100:00100000:2.0:1788619911.547741:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571369856 00010000:00010000:2.0:1788619911.547796:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000084d1816c/0x1cd7fe536873414a lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1552384->1556479) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6b9 expref: 7 pid: 6696 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619911.547856:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 267 00000100:00100000:2.0:1788619911.547869:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c77e1180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571369856:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619911.547911:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619911.547955:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000084d1816c/0x1cd7fe536873414a lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1552384->1556479) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6b9 expref: 7 pid: 6696 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.547991:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000084d1816c/0x1cd7fe536873414a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1552384->1556479) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6b9 expref: 7 pid: 6696 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.548038:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(1 8) 00010000:00010000:2.0:1788619911.548044:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(1 8) 00010000:00010000:2.0:1788619911.548056:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000000afbf5f1/0x1cd7fe536873417b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1929216->1941503) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6dc expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.548069:0:13330:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000afbf5f1/0x1cd7fe536873417b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1929216->1941503) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6dc expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.548122:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000000afbf5f1/0x1cd7fe536873417b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1929216->1941503) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6dc expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619911.548139:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cd5c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589071616:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619911.548274:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc300 going to sleep for 11 seconds 00010000:00010000:1.0:1788619911.550195:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000084d1816c/0x1cd7fe536873414a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1552384->1556479) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6b9 expref: 7 pid: 6696 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619911.550229:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c71f3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589071488:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619911.550274:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000afbf5f1/0x1cd7fe536873417b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1929216->1941503) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6dc expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.550291:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f4ae00, rc 0) 00000100:00100000:1.0:1788619911.550440:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+6:14908:x1875503571369728:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 10582us (10800us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619911.550449:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 606 00000100:00100000:0.0:1788619911.562629:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619911.562860:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571369984 00000100:00100000:1.0:1788619911.562872:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571369984 00000100:00100000:1.0:1788619911.562971:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 607 00000100:00100000:1.0:1788619911.562986:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c77e3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571369984:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619911.563053:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619911.563080:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3a77911/0x1cd7fe5368734182 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b6e3 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619911.563435:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c71f3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589071744:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619911.563558:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963a80 going to sleep for 11 seconds 00000100:00100000:2.0:1788619911.563811:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cd5c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589071616:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619911.563842:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619911.563892:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c77e1180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+8:2355:x1875503571369856:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 16010us (16340us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619911.563903:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 267 00002000:00010000:1.0:1788619911.570454:0:6698:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size: 0 -> 1939321 00000100:00100000:1.0:1788619911.570510:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c71f3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589071744:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619911.570557:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d3a77911/0x1cd7fe5368734182 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6e3 expref: 7 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.570590:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3a77911/0x1cd7fe5368734182 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6e3 expref: 7 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.570606:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f4aa00, rc 0) 00000100:00100000:1.0:1788619911.570689:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c77e3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571369984:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7706us (8077us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619911.570700:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 607 00000100:00100000:3.0:1788619911.579448:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619911.580060:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571370112 00000100:00100000:0.0:1788619911.580077:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571370112 00000100:00100000:0.0:1788619911.580433:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 608 00000100:00100000:0.0:1788619911.580447:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ce0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571370112:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619911.580475:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619911.580499:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ff7ffc7/0x1cd7fe5368734189 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b6ea expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619911.580599:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c77e0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589071872:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619911.580737:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619911.585539:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c77e0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589071872:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619911.585594:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003ff7ffc7/0x1cd7fe5368734189 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6ea expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.585650:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ff7ffc7/0x1cd7fe5368734189 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6ea expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.585667:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6e200, rc 0) 00000100:00100000:2.0:1788619911.585867:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ce0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571370112:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5422us (6725us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619911.585889:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 608 00000100:00100000:2.0:1788619911.592459:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619911.592570:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571370240 00000100:00100000:0.0:1788619911.592585:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571370240 00000100:00100000:0.0:1788619911.592727:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 609 00000100:00100000:0.0:1788619911.592742:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571370240:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619911.592771:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619911.592800:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005414d418/0x1cd7fe5368734190 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b6f1 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619911.592911:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c77e2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589072000:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619911.593369:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa780 going to sleep for 11 seconds 00000100:00100000:0.0:1788619911.600707:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c77e2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589072000:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619911.600763:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005414d418/0x1cd7fe5368734190 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6f1 expref: 7 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619911.600796:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005414d418/0x1cd7fe5368734190 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6f1 expref: 7 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619911.600839:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6f200, rc 0) 00000100:00100000:0.0:1788619911.601137:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571370240:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8397us (8692us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619911.601779:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 609 00000100:00100000:2.0:1788619911.607656:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619911.608595:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571370368 00000100:00100000:1.0:1788619911.608612:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571370368 00000100:00100000:1.0:1788619911.608809:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 610 00000100:00100000:1.0:1788619911.608824:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571370368:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619911.608849:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619911.608871:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3a77911/0x1cd7fe5368734197 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b6f8 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619911.609008:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c71f3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589072128:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619911.609286:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963a80 going to sleep for 11 seconds 00000100:00100000:1.0:1788619911.613272:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c71f3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589072128:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619911.613441:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d3a77911/0x1cd7fe5368734197 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6f8 expref: 7 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.613481:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3a77911/0x1cd7fe5368734197 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6f8 expref: 7 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.613500:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f4aa00, rc 0) 00000100:00100000:1.0:1788619911.613632:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571370368:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4810us (5989us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619911.613642:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 610 00000100:00100000:0.0:1788619911.620860:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619911.622834:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571370496 00000100:00100000:2.0:1788619911.622856:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571370496 00000100:00100000:2.0:1788619911.623152:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 788 00000100:00100000:2.0:1788619911.623359:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c77e2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:14908:x1875503571370496:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619911.623395:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619911.623426:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6800/0x1cd7fe536873419e lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b6ff expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.623547:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe53687341a5 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.623568:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe53687341a5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.623585:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe53687341a5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788619911.623648:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 311952 00000004:00200000:2.0:1788619911.623654:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788619911.623679:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe53687341a5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.623790:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6800/0x1cd7fe536873419e lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6ff expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.623813:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe53687341a5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6ff expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.623824:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619911.623827:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788619911.623832:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7000, rc 0) 00000100:00100000:2.0:1788619911.624018:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c77e2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:14908:x1875503571370496:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 853us (3173us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619911.624026:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 788 00000100:00100000:0.0:1788619911.627528:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619911.628658:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571370624 00000100:00100000:1.0:1788619911.628674:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571370624 00000100:00100000:1.0:1788619911.628767:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 789 00000100:00100000:1.0:1788619911.628779:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c77e2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571370624:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619911.628806:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619911.628832:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4aa00/0x1cd7fe53687341ac lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b706 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.628896:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368734166 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6ce expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619911.628984:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c71f3480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589072256:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619911.629096:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963a80 going to sleep for 11 seconds 00000100:00100000:1.0:1788619911.635620:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c71f3480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589072256:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619911.635660:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 311952 00010000:00010000:1.0:1788619911.635686:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4aa00/0x1cd7fe53687341ac lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b706 expref: 17 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.635787:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619911.635791:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619911.635799:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4aa00/0x1cd7fe53687341ac lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b706 expref: 17 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.635817:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f4aa00, rc 0) 00000100:00100000:1.0:1788619911.635972:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c77e2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571370624:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7193us (8457us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619911.635983:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 789 00000100:00100000:1.0:1788619911.640289:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619911.640420:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571370752 00000100:00100000:2.0:1788619911.640434:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571370752 00000100:00100000:2.0:1788619911.640553:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 790 00000100:00100000:2.0:1788619911.640566:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2352:x1875503571370752:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619911.640594:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619911.640621:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6800/0x1cd7fe53687341b3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b70d expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.640672:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368734166 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6ce expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619911.640761:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cd5500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589072384:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619911.640931:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc300 going to sleep for 11 seconds 00000100:00100000:2.0:1788619911.644833:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cd5500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589072384:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619911.644889:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 311952 00010000:00010000:2.0:1788619911.644924:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6800/0x1cd7fe53687341b3 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b70d expref: 17 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.644952:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619911.644955:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619911.644962:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6800/0x1cd7fe53687341b3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b70d expref: 17 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.644982:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6800, rc 0) 00000100:00100000:2.0:1788619911.645064:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2352:x1875503571370752:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4500us (4788us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619911.645074:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 790 00000100:00100000:3.0:1788619911.650461:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619911.650654:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571370880 00000100:00100000:2.0:1788619911.650666:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571370880 00000100:00100000:2.0:1788619911.650741:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 791 00000100:00100000:2.0:1788619911.650751:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ce0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571370880:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619911.650776:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619911.650804:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7a00/0x1cd7fe53687341ba lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b714 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.650855:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368734166 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6ce expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619911.650950:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cd5500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589072512:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619911.651048:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc300 going to sleep for 11 seconds 00000100:00100000:2.0:1788619911.654676:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cd5500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589072512:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619911.654732:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 311952 00010000:00010000:2.0:1788619911.654760:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7a00/0x1cd7fe53687341ba lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b714 expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.654789:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619911.654792:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619911.654800:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7a00/0x1cd7fe53687341ba lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b714 expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.654819:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7a00, rc 0) 00000100:00100000:2.0:1788619911.655027:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ce0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571370880:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4275us (4578us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619911.655039:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 791 00000100:00100000:3.0:1788619911.660376:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619911.660962:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571371008 00000100:00100000:1.0:1788619911.660977:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571371008 00000100:00100000:1.0:1788619911.661097:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 792 00000100:00100000:1.0:1788619911.661109:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6f86d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571371008:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619911.661138:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619911.661321:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4aa00/0x1cd7fe53687341c1 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b71b expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.661418:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368734166 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6ce expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619911.661472:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5019180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589072640:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619911.661635:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963a80 going to sleep for 11 seconds 00000100:00100000:1.0:1788619911.664663:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5019180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589072640:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619911.664699:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 311952 00010000:00010000:1.0:1788619911.664787:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4aa00/0x1cd7fe53687341c1 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b71b expref: 17 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.664814:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619911.664817:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619911.664824:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4aa00/0x1cd7fe53687341c1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b71b expref: 17 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.664844:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f4aa00, rc 0) 00000100:00100000:1.0:1788619911.665030:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6f86d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571371008:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3923us (4706us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619911.665042:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 792 00000100:00100000:2.0:1788619911.669791:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619911.669898:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571371264 00000100:00100000:0.0:1788619911.670070:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571371264 00000100:00100000:0.0:1788619911.670157:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 793 00000100:00100000:0.0:1788619911.670178:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+13:14908:x1875503571371264:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:0.0:1788619911.670254:0:5846:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619911.670292:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368734174 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6d5 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619911.670317:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619911.670370:0:5846:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788619911.670666:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f400/0x1cd7fe53687341c8 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619911.670685:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe53687341a5 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6ff expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619911.670694:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f400/0x1cd7fe53687341c8 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619911.670735:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe53687341a5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6ff expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619911.670770:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe53687341a5 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6ff expref: 15 pid: 5845 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619911.670789:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c77e1f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589072768:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619911.671092:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa780 going to sleep for 11 seconds 00000100:00100000:0.0:1788619911.678957:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c77e1f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589072768:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619911.678995:0:5846:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f400/0x1cd7fe53687341c8 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619911.680137:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619911.680535:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571371392 00000100:00100000:1.0:1788619911.680545:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571371392 00010000:00010000:1.0:1788619911.680591:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe53687341a5 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6ff expref: 16 pid: 5845 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619911.680633:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 268 00000100:00100000:1.0:1788619911.680641:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd7480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571371392:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619911.680676:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619911.680682:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe53687341a5 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6ff expref: 16 pid: 5845 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.680694:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe53687341a5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6ff expref: 16 pid: 5845 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.680720:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe53687341a5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6ff expref: 16 pid: 5845 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.680741:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619911.680747:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f400/0x1cd7fe53687341c8 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.680846:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f400/0x1cd7fe53687341c8 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.680854:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f400/0x1cd7fe53687341c8 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.680889:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619911.681018:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd7480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571371392:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 375us (887us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619911.681027:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 268 00010000:00010000:0.0:1788619911.681095:0:5846:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f400/0x1cd7fe53687341c8 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619911.681104:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f400/0x1cd7fe53687341c8 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788619911.681267:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967605 00000004:00010000:0.0:1788619911.681365:0:5846:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 311952 -> 311952 00000004:00010000:0.0:1788619911.681369:0:5846:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619911 -> 1788619911 00000004:00010000:0.0:1788619911.681372:0:5846:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619908 -> 1788619908 00000004:00010000:0.0:1788619911.681375:0:5846:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619911 -> 1788619911 00000004:00010000:0.0:1788619911.681377:0:5846:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 518 -> 518 00000004:00010000:0.0:1788619911.681385:0:5846:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000ca4eb75e reply state 000000008ef79d26 transno 4294967606 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f400/0x1cd7fe53687341c8 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619911.681395:0:5846:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788619911.681480:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+15:14908:x1875503571371264:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 11306us (11704us total) trans 4294967606 rc 0/0 00000100:00100000:0.0:1788619911.681499:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 793 00000020:00080000:0.0:1788619911.681503:0:5846:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788619911.685746:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f400/0x1cd7fe53687341c8 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619911.685897:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f400/0x1cd7fe53687341c8 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619911.685907:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f400/0x1cd7fe53687341c8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619911.685914:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c6b6f400 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788619911.685920:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f400/0x1cd7fe53687341c8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619911.685930:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f400/0x1cd7fe53687341c8 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619911.685952:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619911.685955:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788619911.685961:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f400/0x1cd7fe53687341c8 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619911.685968:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f400/0x1cd7fe53687341c8 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619911.693135:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619911.693485:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571371520 00000100:00100000:1.0:1788619911.693499:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571371520 00000100:00100000:1.0:1788619911.693572:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 611 00000100:00100000:1.0:1788619911.693583:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:14908:x1875503571371520:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619911.693627:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619911.693650:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3a77911/0x1cd7fe53687341cf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b722 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.693695:0:6698:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000000afbf5f1/0x1cd7fe536873417b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1929216->1941503) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6dc expref: 5 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.693706:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(1 8) 00010000:00010000:1.0:1788619911.693711:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 8) 00010000:00010000:1.0:1788619911.693718:0:6698:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3a77911/0x1cd7fe53687341cf lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b722 expref: 7 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.693776:0:6698:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000000afbf5f1/0x1cd7fe536873417b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1929216->1941503) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6dc expref: 5 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.693795:0:6698:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000000afbf5f1/0x1cd7fe536873417b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1929216->1941503) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6dc expref: 5 pid: 7654 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619911.693810:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5019180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589072896:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619911.694063:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963a80 going to sleep for 11 seconds 00000100:00100000:3.0:1788619911.701723:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619911.702050:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571371648 00000100:00100000:0.0:1788619911.702063:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571371648 00002000:00100000:0.0:1788619911.702128:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1929216->1939320) req@ffff8e77c6f86300 x1875503571371648/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:7/0 lens 10600/0 e 0 to 0 dl 1788619922 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619911.702141:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6f86300 with x1875503571371648 ext(1929216->1939320) 00010000:00010000:0.0:1788619911.702174:0:6703:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000000afbf5f1/0x1cd7fe536873417b lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1929216->1941503) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6dc expref: 6 pid: 7654 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619911.702193:0:6703:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000000afbf5f1/0x1cd7fe536873417b lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1929216->1941503) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6dc expref: 6 pid: 7654 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788619911.702202:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff8e77c6f86300 00000100:00100000:0.0:1788619911.702247:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 12 00000100:00100000:0.0:1788619911.702259:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6f86300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571371648:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:1.0:1788619911.702487:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5019180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589072896:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619911.702522:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d3a77911/0x1cd7fe53687341cf lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b722 expref: 7 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.702541:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f4aa00, rc 0) 00000100:00100000:1.0:1788619911.702620:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:14908:x1875503571371520:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 9039us (9503us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619911.702628:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 611 00000100:00100000:0.0:1788619911.703032:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6f86300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571371648:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 774us (1335us total) trans 4294967390 rc 0/0 00000100:00100000:0.0:1788619911.703044:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 12 00002000:00100000:0.0:1788619911.703052:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1929216->1939320) req@ffff8e77c6f86300 x1875503571371648/t4294967390(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:7/0 lens 10600/448 e 0 to 0 dl 1788619922 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619911.703061:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6f86300 with x1875503571371648 ext(1929216->1939320) 00010000:00010000:0.0:1788619911.703068:0:6703:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000000afbf5f1/0x1cd7fe536873417b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1929216->1941503) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6dc expref: 8 pid: 7654 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619911.703081:0:6703:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000000afbf5f1/0x1cd7fe536873417b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1929216->1941503) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6dc expref: 8 pid: 7654 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788619911.703090:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff8e77c6f86300 00000100:00100000:3.0:1788619911.708589:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619911.709692:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571371776 00000100:00100000:2.0:1788619911.709708:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571371776 00010000:00010000:2.0:1788619911.709780:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000000afbf5f1/0x1cd7fe536873417b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1929216->1941503) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6dc expref: 8 pid: 7654 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619911.709839:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 269 00000100:00100000:2.0:1788619911.709850:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6f85f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571371776:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619911.709901:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619911.709933:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000000afbf5f1/0x1cd7fe536873417b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1929216->1941503) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6dc expref: 8 pid: 7654 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.709948:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000000afbf5f1/0x1cd7fe536873417b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1929216->1941503) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6dc expref: 8 pid: 7654 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.709976:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000afbf5f1/0x1cd7fe536873417b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1929216->1941503) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6dc expref: 8 pid: 7654 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.709998:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 8) 00010000:00010000:2.0:1788619911.710003:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 8) 00010000:00010000:2.0:1788619911.710013:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000d3a77911/0x1cd7fe53687341cf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b722 expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.710025:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3a77911/0x1cd7fe53687341cf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b722 expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.710076:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000d3a77911/0x1cd7fe53687341cf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b722 expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619911.711303:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cd7800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589073024:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619911.711454:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc300 going to sleep for 11 seconds 00000100:00100000:2.0:1788619911.721230:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619911.721276:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619911.721518:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571372032 00000100:00100000:1.0:1788619911.721532:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571372032 00002000:00010000:1.0:1788619911.721586:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:1.0:1788619911.721592:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5cd9880 with x1875503571372032 ext(0->18446744073709551615) 00002000:00010000:1.0:1788619911.721608:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5cd9880. 00000100:00100000:1.0:1788619911.721656:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 344 00000100:00100000:1.0:1788619911.721668:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571372032:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:1.0:1788619911.722086:0:6702:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1939321 -> 0 00002000:00010000:1.0:1788619911.722091:0:6702:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619911 -> 1788619911 00002000:00010000:1.0:1788619911.722094:0:6702:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619908 -> 1788619908 00002000:00010000:1.0:1788619911.722097:0:6702:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619911 -> 1788619911 00002000:00010000:1.0:1788619911.722100:0:6702:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 263 -> 263 00000100:00100000:1.0:1788619911.722189:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+8:2354:x1875503571372032:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 522us (974us total) trans 4294967391 rc 0/0 00000100:00100000:1.0:1788619911.722198:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 344 00002000:00010000:1.0:1788619911.722203:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:1.0:1788619911.722206:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5cd9880 with x1875503571372032 ext(0->18446744073709551615) 00002000:00010000:1.0:1788619911.722211:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5cd9880. 00000100:00100000:0.0:1788619911.723553:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571371904 00000100:00100000:0.0:1788619911.723572:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571371904 00002000:00010000:0.0:1788619911.723624:0:5859:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:0.0:1788619911.723631:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5cdaa00 with x1875503571371904 00000100:00100000:1.0:1788619911.724081:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 451 00000100:00100000:1.0:1788619911.724096:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cdaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+14:2354:x1875503571371904:12345-192.168.201.5@tcp:10:fsx.0 00000004:00010000:1.0:1788619911.724553:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 311952 -> 962214 00000004:00010000:1.0:1788619911.724558:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619911 -> 1788619911 00000004:00010000:1.0:1788619911.724570:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619908 -> 1788619908 00000004:00010000:1.0:1788619911.724573:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619911 -> 1788619911 00000004:00010000:1.0:1788619911.724575:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 518 -> 518 00000100:00100000:1.0:1788619911.724661:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cdaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+15:2354:x1875503571371904:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 569us (3382us total) trans 4294967607 rc 0/0 00000100:00100000:3.0:1788619911.726491:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cd7800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589073024:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619911.726545:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619911.728018:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6f85f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2353:x1875503571371776:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 18155us (19429us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619911.728035:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 269 00000100:00100000:1.0:1788619911.728989:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 451 00002000:00010000:1.0:1788619911.729002:0:5857:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:1.0:1788619911.729008:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5cdaa00 with x1875503571371904 00000100:00100000:1.0:1788619911.730202:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619911.730365:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571372160 00000100:00100000:2.0:1788619911.730376:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571372160 00000100:00100000:2.0:1788619911.730433:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 794 00000100:00100000:2.0:1788619911.730457:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5019180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+15:14908:x1875503571372160:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619911.730481:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619911.730500:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6600/0x1cd7fe53687341d6 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b729 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.730586:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7e00/0x1cd7fe53687341dd lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.730603:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7e00/0x1cd7fe53687341dd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.730616:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7e00/0x1cd7fe53687341dd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788619911.730645:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 962214 00000004:00200000:2.0:1788619911.730649:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788619911.730666:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7e00/0x1cd7fe53687341dd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.730703:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6600/0x1cd7fe53687341d6 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b729 expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.730724:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7e00/0x1cd7fe53687341dd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b729 expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.730767:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619911.730770:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788619911.730774:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7e00, rc 0) 00000100:00100000:2.0:1788619911.730843:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5019180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+16:14908:x1875503571372160:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 389us (650us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619911.730849:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 794 00000100:00100000:0.0:1788619911.734472:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619911.734587:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571372288 00000100:00100000:1.0:1788619911.734600:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571372288 00000100:00100000:1.0:1788619911.734795:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 795 00000100:00100000:1.0:1788619911.734809:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c77e1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:14908:x1875503571372288:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619911.734838:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619911.734865:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4b800/0x1cd7fe53687341e4 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b730 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.734916:0:9417:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368734166 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6ce expref: 15 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.734928:0:9417:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4b800/0x1cd7fe53687341e4 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b730 expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.734978:0:9417:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368734166 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6ce expref: 15 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.734994:0:9417:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368734166 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6ce expref: 15 pid: 11397 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619911.735029:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78dfa42300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589073152:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619911.735139:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963a80 going to sleep for 11 seconds 00000100:00100000:1.0:1788619911.739911:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78dfa42300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589073152:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619911.739962:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4b800/0x1cd7fe53687341e4 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b730 expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.739989:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619911.739993:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619911.739998:0:9417:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4b800/0x1cd7fe53687341e4 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b730 expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.740011:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f4b800, rc 0) 00000100:00100000:1.0:1788619911.740096:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c77e1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:14908:x1875503571372288:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5290us (5638us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619911.740107:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 795 00000100:00100000:0.0:1788619911.740528:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619911.740594:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571372416 00000100:00100000:1.0:1788619911.740603:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571372416 00010000:00010000:1.0:1788619911.740643:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368734166 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6ce expref: 16 pid: 11397 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619911.740704:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 270 00000100:00100000:1.0:1788619911.740715:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c77e3100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571372416:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619911.740732:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619911.740765:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368734166 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6ce expref: 16 pid: 11397 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.740799:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368734166 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6ce expref: 16 pid: 11397 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.740828:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368734166 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b6ce expref: 16 pid: 11397 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.740848:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619911.740855:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4b800/0x1cd7fe53687341e4 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b730 expref: 15 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.740865:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4b800/0x1cd7fe53687341e4 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b730 expref: 15 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.740886:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4b800/0x1cd7fe53687341e4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b730 expref: 15 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.740943:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4b800/0x1cd7fe53687341e4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b730 expref: 15 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619911.740960:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78dfa42300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589073280:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619911.741040:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963a80 going to sleep for 11 seconds 00000100:00100000:1.0:1788619911.748518:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619911.748601:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571372544 00000100:00100000:3.0:1788619911.748614:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571372544 00002000:00100000:3.0:1788619911.748817:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78dfa40e00 x1875503571372544/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:7/0 lens 488/0 e 0 to 0 dl 1788619922 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619911.748832:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78dfa40e00 with x1875503571372544 00000100:00100000:3.0:1788619911.748890:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 452 00000100:00100000:3.0:1788619911.748903:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa40e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571372544:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619911.749591:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa40e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571372544:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 685us (1083us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619911.749603:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 452 00002000:00100000:3.0:1788619911.749613:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78dfa40e00 x1875503571372544/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:7/0 lens 488/4536 e 0 to 0 dl 1788619922 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619911.749634:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78dfa40e00 with x1875503571372544 00000100:00100000:2.0:1788619911.754378:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78dfa42300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589073280:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619911.754429:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619911.754602:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c77e3100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+15:2355:x1875503571372416:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 13886us (14073us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619911.754616:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 270 00000100:00100000:0.0:1788619911.757374:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619911.758584:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571372672 00000100:00100000:2.0:1788619911.758608:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571372672 00002000:00100000:2.0:1788619911.758717:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c77e0700 x1875503571372672/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:7/0 lens 488/0 e 0 to 0 dl 1788619922 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619911.758732:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c77e0700 with x1875503571372672 00000100:00100000:2.0:1788619911.758781:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 453 00000100:00100000:2.0:1788619911.758791:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c77e0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2352:x1875503571372672:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619911.759220:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c77e0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2352:x1875503571372672:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 428us (1894us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619911.759230:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 453 00002000:00100000:2.0:1788619911.759237:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c77e0700 x1875503571372672/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:7/0 lens 488/4536 e 0 to 0 dl 1788619922 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619911.759273:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c77e0700 with x1875503571372672 00000100:00100000:1.0:1788619911.764536:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619911.764645:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571372800 00000100:00100000:0.0:1788619911.764659:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571372800 00000100:00100000:0.0:1788619911.764903:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 796 00000100:00100000:0.0:1788619911.764917:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa42300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:14908:x1875503571372800:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619911.764947:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619911.764976:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e200/0x1cd7fe53687341eb lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b737 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619911.765099:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fc00/0x1cd7fe53687341f2 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619911.765120:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fc00/0x1cd7fe53687341f2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619911.765135:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fc00/0x1cd7fe53687341f2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619911.765303:0:5846:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 962214 00000004:00200000:0.0:1788619911.765309:0:5846:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619911.765379:0:5846:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fc00/0x1cd7fe53687341f2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619911.765416:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e200/0x1cd7fe53687341eb lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b737 expref: 18 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619911.765441:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fc00/0x1cd7fe53687341f2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b737 expref: 17 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619911.765477:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619911.765481:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619911.765486:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6fc00, rc 0) 00000100:00100000:0.0:1788619911.765619:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa42300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+17:14908:x1875503571372800:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 704us (1097us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619911.765627:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 796 00000100:00100000:1.0:1788619911.770240:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619911.770539:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571372928 00000100:00100000:2.0:1788619911.770552:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571372928 00000100:00100000:2.0:1788619911.770620:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 612 00000100:00100000:2.0:1788619911.770633:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571372928:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619911.770658:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619911.770694:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001dc9de0/0x1cd7fe53687341f9 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b73e expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619911.770891:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cdbb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589073408:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619911.771052:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccb40 going to sleep for 11 seconds 00000100:00100000:2.0:1788619911.781274:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cdbb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589073408:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619911.781450:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000001dc9de0/0x1cd7fe53687341f9 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b73e expref: 8 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.781484:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001dc9de0/0x1cd7fe53687341f9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b73e expref: 8 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.781500:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6a00, rc 0) 00000100:00100000:2.0:1788619911.781620:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571372928:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 10988us (11396us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619911.781631:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 612 00000100:00100000:2.0:1788619911.786656:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619911.786888:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571373056 00000100:00100000:0.0:1788619911.786901:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571373056 00000100:00100000:0.0:1788619911.787101:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 613 00000100:00100000:0.0:1788619911.787114:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe13b450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571373056:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619911.787148:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619911.787179:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ff7ffc7/0x1cd7fe5368734200 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b745 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619911.787259:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c77e0000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589073536:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619911.787409:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa780 going to sleep for 11 seconds 00000100:00100000:0.0:1788619911.791613:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c77e0000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589073536:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619911.791678:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003ff7ffc7/0x1cd7fe5368734200 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b745 expref: 8 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619911.792059:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ff7ffc7/0x1cd7fe5368734200 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b745 expref: 8 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619911.792077:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6e200, rc 0) 00000100:00100000:0.0:1788619911.792428:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe13b450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571373056:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5314us (5784us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619911.792440:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 613 00000100:00100000:0.0:1788619911.798898:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619911.799046:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571373184 00000100:00100000:1.0:1788619911.799059:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571373184 00000100:00100000:1.0:1788619911.799322:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 614 00000100:00100000:1.0:1788619911.799437:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c77e3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571373184:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619911.799463:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619911.799488:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001cadc580/0x1cd7fe5368734207 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b74c expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619911.799570:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589073664:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619911.799839:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963300 going to sleep for 11 seconds 00000100:00100000:1.0:1788619911.803565:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589073664:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619911.803617:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001cadc580/0x1cd7fe5368734207 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b74c expref: 8 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.803650:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001cadc580/0x1cd7fe5368734207 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b74c expref: 8 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.803682:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f4a000, rc 0) 00000100:00100000:1.0:1788619911.803895:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c77e3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571373184:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4461us (5021us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619911.803905:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 614 00000100:00100000:0.0:1788619911.808888:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619911.809088:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571373312 00000100:00100000:1.0:1788619911.809101:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571373312 00000100:00100000:1.0:1788619911.809371:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 615 00000100:00100000:1.0:1788619911.809386:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c77e0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2353:x1875503571373312:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619911.809412:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619911.809440:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000afbf5f1/0x1cd7fe536873420e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b753 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619911.809545:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589073792:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619911.809752:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963300 going to sleep for 11 seconds 00000100:00100000:1.0:1788619911.818651:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589073792:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619911.818694:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000afbf5f1/0x1cd7fe536873420e lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b753 expref: 8 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.818719:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000afbf5f1/0x1cd7fe536873420e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b753 expref: 8 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.818734:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f4ae00, rc 0) 00000100:00100000:1.0:1788619911.818816:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c77e0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2353:x1875503571373312:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 9433us (9954us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619911.818826:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 615 00000100:00100000:3.0:1788619911.822488:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619911.823167:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571373568 00000100:00100000:2.0:1788619911.823202:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571373568 00000100:00100000:2.0:1788619911.823294:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 797 00000100:00100000:2.0:1788619911.823307:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6f87100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+15:14908:x1875503571373568:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:2.0:1788619911.823416:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619911.823455:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7e00/0x1cd7fe53687341dd lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b729 expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.823478:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619911.823485:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788619911.823795:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe5368734215 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.823813:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fc00/0x1cd7fe53687341f2 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b737 expref: 16 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.823823:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe5368734215 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.823865:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fc00/0x1cd7fe53687341f2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b737 expref: 16 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.823894:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fc00/0x1cd7fe53687341f2 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b737 expref: 16 pid: 5846 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619911.825373:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cdbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589073920:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619911.825536:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccb40 going to sleep for 11 seconds 00000100:00100000:1.0:1788619911.838316:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619911.838423:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571373696 00000100:00100000:3.0:1788619911.838433:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571373696 00000100:00100000:2.0:1788619911.838475:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cdbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589073920:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619911.838521:0:5845:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe5368734215 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619911.838654:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fc00/0x1cd7fe53687341f2 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b737 expref: 17 pid: 5846 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619911.838972:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 271 00000100:00100000:3.0:1788619911.838984:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8023c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2354:x1875503571373696:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619911.839000:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619911.839008:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fc00/0x1cd7fe53687341f2 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b737 expref: 17 pid: 5846 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619911.839022:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fc00/0x1cd7fe53687341f2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b737 expref: 17 pid: 5846 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619911.839045:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fc00/0x1cd7fe53687341f2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b737 expref: 17 pid: 5846 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619911.839062:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619911.839066:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe5368734215 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619911.839075:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe5368734215 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619911.839082:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe5368734215 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619911.839127:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788619911.839203:0:5845:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe5368734215 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.839215:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe5368734215 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788619911.839459:0:5845:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967608 00000004:00010000:2.0:1788619911.839532:0:5845:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 962214 -> 962214 00000004:00010000:2.0:1788619911.839537:0:5845:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619911 -> 1788619911 00000004:00010000:2.0:1788619911.839540:0:5845:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619908 -> 1788619908 00000004:00010000:2.0:1788619911.839543:0:5845:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619911 -> 1788619911 00000004:00010000:2.0:1788619911.839545:0:5845:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 518 -> 518 00000004:00010000:2.0:1788619911.839555:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000025111a00 reply state 00000000c96ba593 transno 4294967609 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe5368734215 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619911.839565:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788619911.839726:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6f87100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+17:14908:x1875503571373568:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 16418us (17250us total) trans 4294967609 rc 0/0 00000100:00100000:2.0:1788619911.839737:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 797 00000020:00080000:2.0:1788619911.839745:0:5845:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1788619911.839823:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8023c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571373696:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 838us (1512us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619911.839836:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 271 00010000:00010000:2.0:1788619911.846989:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe5368734215 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.847007:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe5368734215 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.847014:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe5368734215 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.847041:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78cdcc6a00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:2.0:1788619911.847047:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe5368734215 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.847056:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe5368734215 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.847073:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619911.847076:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788619911.847081:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe5368734215 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.847087:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe5368734215 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619911.848971:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619911.849102:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571373824 00000100:00100000:2.0:1788619911.849112:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571373824 00000100:00100000:2.0:1788619911.849168:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 798 00000100:00100000:2.0:1788619911.849179:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+16:14908:x1875503571373824:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619911.849201:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619911.849220:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7e00/0x1cd7fe536873421c lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b75a expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.849249:0:9417:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4b800/0x1cd7fe53687341e4 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b730 expref: 15 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.849260:0:9417:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7e00/0x1cd7fe536873421c lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b75a expref: 17 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.849309:0:9417:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4b800/0x1cd7fe53687341e4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b730 expref: 15 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.849323:0:9417:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4b800/0x1cd7fe53687341e4 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b730 expref: 15 pid: 9417 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619911.849356:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cdbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589074048:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619911.849537:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccb40 going to sleep for 11 seconds 00000100:00100000:3.0:1788619911.857911:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619911.857945:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cdbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589074048:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619911.857984:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7e00/0x1cd7fe536873421c lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b75a expref: 17 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619911.858023:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571373952 00000100:00100000:0.0:1788619911.858039:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571373952 00010000:00010000:2.0:1788619911.858072:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619911.858078:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619911.858084:0:9417:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7e00/0x1cd7fe536873421c lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b75a expref: 17 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.858097:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7e00, rc 0) 00002000:00100000:0.0:1788619911.858126:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6f85f80 x1875503571373952/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:7/0 lens 488/0 e 0 to 0 dl 1788619922 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619911.858139:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6f85f80 with x1875503571373952 00010000:00010000:0.0:1788619911.858149:0:5858:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4b800/0x1cd7fe53687341e4 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b730 expref: 16 pid: 9417 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619911.858173:0:5858:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4b800/0x1cd7fe53687341e4 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b730 expref: 16 pid: 9417 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788619911.858183:0:5858:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e77c6f85f80 00000100:00100000:2.0:1788619911.858220:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+17:14908:x1875503571373824:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 9039us (9253us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619911.858230:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 798 00000100:00100000:0.0:1788619911.858310:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 19 00000100:00100000:0.0:1788619911.858376:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6f85f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571373952:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:2.0:1788619911.866449:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 962214 -> 962214 00000004:00010000:2.0:1788619911.866462:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619911 -> 1788619911 00000004:00010000:2.0:1788619911.866465:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619908 -> 1788619908 00000004:00010000:2.0:1788619911.866467:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619911 -> 1788619911 00000004:00010000:2.0:1788619911.866480:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 518 -> 518 00000100:00100000:2.0:1788619911.866588:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6f85f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2354:x1875503571373952:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 8260us (8680us total) trans 4294967610 rc 0/0 00000100:00100000:2.0:1788619911.866601:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 19 00002000:00100000:2.0:1788619911.866618:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6f85f80 x1875503571373952/t4294967610(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:7/0 lens 488/448 e 0 to 0 dl 1788619922 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619911.866629:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6f85f80 with x1875503571373952 00010000:00010000:2.0:1788619911.866641:0:5858:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4b800/0x1cd7fe53687341e4 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b730 expref: 18 pid: 9417 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.866658:0:5858:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4b800/0x1cd7fe53687341e4 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b730 expref: 18 pid: 9417 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788619911.866674:0:5858:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e77c6f85f80 00000100:00100000:3.0:1788619911.872252:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619911.872618:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571374080 00000100:00100000:1.0:1788619911.872629:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571374080 00010000:00010000:1.0:1788619911.872962:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4b800/0x1cd7fe53687341e4 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b730 expref: 18 pid: 9417 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619911.873007:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 272 00000100:00100000:1.0:1788619911.873017:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6f84700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2355:x1875503571374080:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619911.873032:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619911.873075:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4b800/0x1cd7fe53687341e4 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b730 expref: 18 pid: 9417 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.873089:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4b800/0x1cd7fe53687341e4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b730 expref: 18 pid: 9417 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.873114:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4b800/0x1cd7fe53687341e4 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b730 expref: 18 pid: 9417 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.873133:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619911.873148:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7e00/0x1cd7fe536873421c lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b75a expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.873160:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7e00/0x1cd7fe536873421c lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b75a expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.873169:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7e00/0x1cd7fe536873421c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b75a expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619911.873211:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7e00/0x1cd7fe536873421c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b75a expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619911.873224:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501c000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589074176:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619911.873308:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963300 going to sleep for 11 seconds 00000100:00100000:1.0:1788619911.885275:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619911.885387:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501c000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589074176:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619911.885420:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619911.885537:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571374208 00000100:00100000:3.0:1788619911.885559:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6f84700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2355:x1875503571374080:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 12542us (13317us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619911.885571:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 272 00000100:00100000:0.0:1788619911.885846:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571374208 00002000:00010000:0.0:1788619911.886152:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1356034->18446744073709551615) 00002000:00100000:0.0:1788619911.886158:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c50c7100 with x1875503571374208 ext(1356034->18446744073709551615) 00002000:00010000:0.0:1788619911.886170:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c50c7100. 00000100:00100000:0.0:1788619911.886578:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 345 00000100:00100000:0.0:1788619911.886591:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+8:2355:x1875503571374208:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:0.0:1788619911.886845:0:6703:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 0 -> 1356034 00002000:00010000:0.0:1788619911.886849:0:6703:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619911 -> 1788619911 00002000:00010000:0.0:1788619911.886852:0:6703:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619908 -> 1788619908 00002000:00010000:0.0:1788619911.886854:0:6703:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619911 -> 1788619911 00002000:00010000:0.0:1788619911.886857:0:6703:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 263 -> 263 00000100:00100000:0.0:1788619911.886931:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571374208:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 343us (1667us total) trans 4294967392 rc 0/0 00000100:00100000:0.0:1788619911.886939:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 345 00002000:00010000:0.0:1788619911.886943:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1356034->18446744073709551615) 00002000:00100000:0.0:1788619911.886946:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c50c7100 with x1875503571374208 ext(1356034->18446744073709551615) 00002000:00010000:0.0:1788619911.886950:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c50c7100. 00000100:00100000:1.0:1788619911.899088:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619911.899147:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619911.899560:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571374464 00000100:00100000:0.0:1788619911.899571:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571374464 00002000:00100000:0.0:1788619911.899625:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c50c6680 x1875503571374464/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:7/0 lens 488/0 e 0 to 0 dl 1788619922 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619911.899637:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c50c6680 with x1875503571374464 00000100:00100000:0.0:1788619911.899688:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 454 00000100:00100000:0.0:1788619911.899699:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+17:2354:x1875503571374464:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619911.900833:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571374336 00000100:00100000:2.0:1788619911.900841:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571374336 00002000:00100000:2.0:1788619911.901195:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1359871) req@ffff8e77c50c7b80 x1875503571374336/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:7/0 lens 488/0 e 0 to 0 dl 1788619922 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619911.901207:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c50c7b80 with x1875503571374336 ext(1048576->1359871) 00002000:00010000:2.0:1788619911.901213:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c50c7b80 00000100:00100000:2.0:1788619911.901246:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 346 00000100:00100000:2.0:1788619911.901256:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571374336:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619911.921240:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+17:2354:x1875503571374464:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 21531us (22080us total) trans 0 rc 962560/962560 00000100:00100000:0.0:1788619911.921255:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 454 00002000:00100000:0.0:1788619911.921269:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c50c6680 x1875503571374464/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:7/0 lens 488/440 e 0 to 0 dl 1788619922 ref 1 fl Complete:/600/0 rc 962560/962560 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619911.921279:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c50c6680 with x1875503571374464 00000100:00100000:2.0:1788619911.933091:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571374336:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 31825us (34001us total) trans 0 rc 311296/311296 00000100:00100000:2.0:1788619911.933106:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 346 00002000:00100000:2.0:1788619911.933133:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1359871) req@ffff8e77c50c7b80 x1875503571374336/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:7/0 lens 488/440 e 0 to 0 dl 1788619922 ref 1 fl Complete:/600/0 rc 311296/311296 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619911.933145:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c50c7b80 with x1875503571374336 ext(1048576->1359871) 00002000:00010000:2.0:1788619911.933152:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c50c7b80 00000100:00100000:3.0:1788619911.938446:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619911.938843:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571374592 00000100:00100000:0.0:1788619911.938857:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571374592 00002000:00100000:0.0:1788619911.938938:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1347584->1356033) req@ffff8e77c6f84000 x1875503571374592/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:7/0 lens 8944/0 e 0 to 0 dl 1788619922 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619911.938950:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6f84000 with x1875503571374592 ext(1347584->1356033) 00002000:00010000:0.0:1788619911.938958:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6f84000 00000100:00100000:0.0:1788619911.939000:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 347 00000100:00100000:0.0:1788619911.939012:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6f84000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571374592:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:0.0:1788619911.940027:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6f84000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571374592:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 1009us (1591us total) trans 4294967393 rc 0/0 00000100:00100000:0.0:1788619911.940040:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 347 00002000:00100000:0.0:1788619911.940050:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1347584->1356033) req@ffff8e77c6f84000 x1875503571374592/t4294967393(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:7/0 lens 8944/448 e 0 to 0 dl 1788619922 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619911.940061:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6f84000 with x1875503571374592 ext(1347584->1356033) 00002000:00010000:0.0:1788619911.940067:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6f84000 00000100:00100000:3.0:1788619911.943403:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619911.943566:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571374720 00000100:00100000:2.0:1788619911.943581:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571374720 00000100:00100000:2.0:1788619911.943672:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 616 00000100:00100000:2.0:1788619911.943685:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6f84380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571374720:12345-192.168.201.5@tcp:16:fsx.0 00000001:00080000:1.0:1788619911.973075:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967393 is committed 00000001:00080000:1.0:1788619911.973147:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967390 is committed 00000100:00100000:3.0:1788619911.973364:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6f84380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571374720:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 29642us (29938us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619911.973381:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 616 00010000:00010000:3.0:1788619911.977073:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788619911.977419:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619911.977537:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571374848 00000100:00100000:0.0:1788619911.977550:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571374848 00000100:00100000:0.0:1788619911.977673:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 799 00000100:00100000:0.0:1788619911.977688:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+17:14908:x1875503571374848:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619911.977728:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619911.977756:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f600/0x1cd7fe5368734223 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b761 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619911.977892:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe536873422a lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619911.977933:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe536873422a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619911.977947:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe536873422a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619911.977983:0:5846:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1356034 00000004:00200000:0.0:1788619911.977989:0:5846:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619911.978013:0:5846:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe536873422a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619911.978046:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f600/0x1cd7fe5368734223 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b761 expref: 19 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619911.978069:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe536873422a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b761 expref: 18 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619911.978079:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619911.978082:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619911.978087:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6ee00, rc 0) 00000100:00100000:0.0:1788619911.978248:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+18:14908:x1875503571374848:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 563us (839us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619911.978257:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 799 00000100:00100000:1.0:1788619911.983677:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619911.983739:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619911.983990:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571374976 00000100:00100000:2.0:1788619911.984004:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571374976 00000100:00100000:2.0:1788619911.984081:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 800 00000100:00100000:2.0:1788619911.984095:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2354:x1875503571374976:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619911.984134:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619911.984175:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe5368734231 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b768 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.984225:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7e00/0x1cd7fe536873421c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b75a expref: 17 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619911.984283:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cdbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589074304:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619911.984456:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccd80 going to sleep for 11 seconds 00000004:00010000:2.0:1788619911.986953:0:5845:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619908 -> 1788619911 00000100:00100000:2.0:1788619911.987000:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cdbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589074304:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619911.987032:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 962214 00010000:00010000:2.0:1788619911.987055:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe5368734231 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b768 expref: 18 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.987107:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619911.987111:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619911.987118:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe5368734231 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b768 expref: 18 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.987136:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297a00, rc 0) 00000100:00100000:2.0:1788619911.987225:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2354:x1875503571374976:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3133us (3565us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619911.987236:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 800 00000100:00100000:0.0:1788619911.987376:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571375104 00000100:00100000:0.0:1788619911.987386:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571375104 00000100:00100000:0.0:1788619911.987453:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 617 00000100:00100000:0.0:1788619911.987465:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571375104:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619911.987481:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619911.987510:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4614856/0x1cd7fe5368734238 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b76f expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619911.987586:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c77e0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589074432:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619911.987658:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa240 going to sleep for 11 seconds 00002000:00010000:0.0:1788619911.990936:0:6697:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619908 -> 1788619911 00000100:00100000:0.0:1788619911.990980:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c77e0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589074432:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619911.991031:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a4614856/0x1cd7fe5368734238 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b76f expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619911.991064:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4614856/0x1cd7fe5368734238 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b76f expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619911.991080:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6f000, rc 0) 00000100:00100000:0.0:1788619911.991162:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571375104:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3699us (7421us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619911.991172:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 617 00000100:00100000:3.0:1788619911.995568:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619911.995653:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571375232 00000100:00100000:2.0:1788619911.995665:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571375232 00000100:00100000:2.0:1788619911.995742:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 801 00000100:00100000:2.0:1788619911.995754:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6f85880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+17:14908:x1875503571375232:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619911.995784:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619911.995812:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296c00/0x1cd7fe536873423f lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b776 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.995852:0:9417:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7e00/0x1cd7fe536873421c lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b75a expref: 17 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.995865:0:9417:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296c00/0x1cd7fe536873423f lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b776 expref: 18 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.995921:0:9417:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7e00/0x1cd7fe536873421c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b75a expref: 17 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619911.995963:0:9417:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7e00/0x1cd7fe536873421c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b75a expref: 17 pid: 9417 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619911.995980:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cdbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589074560:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619911.996093:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccd80 going to sleep for 11 seconds 00000100:00100000:2.0:1788619912.000972:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cdbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589074560:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619912.001025:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296c00/0x1cd7fe536873423f lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b776 expref: 18 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.001050:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619912.001053:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619912.001059:0:9417:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296c00/0x1cd7fe536873423f lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b776 expref: 18 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.001072:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296c00, rc 0) 00000100:00100000:2.0:1788619912.001161:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6f85880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+18:14908:x1875503571375232:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5407us (5838us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619912.001172:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 801 00000100:00100000:3.0:1788619912.005197:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619912.005299:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571375360 00000100:00100000:2.0:1788619912.005312:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571375360 00010000:00010000:2.0:1788619912.005417:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7e00/0x1cd7fe536873421c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b75a expref: 18 pid: 9417 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619912.005460:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 273 00000100:00100000:2.0:1788619912.005472:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6f86a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+18:2354:x1875503571375360:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619912.005510:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619912.005545:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7e00/0x1cd7fe536873421c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b75a expref: 18 pid: 9417 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.005562:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7e00/0x1cd7fe536873421c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b75a expref: 18 pid: 9417 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.005590:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7e00/0x1cd7fe536873421c lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b75a expref: 18 pid: 9417 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.005612:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619912.005617:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296c00/0x1cd7fe536873423f lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b776 expref: 17 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.005632:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296c00/0x1cd7fe536873423f lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b776 expref: 17 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.005643:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296c00/0x1cd7fe536873423f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b776 expref: 17 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.005690:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296c00/0x1cd7fe536873423f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b776 expref: 17 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619912.005720:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cdbb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589074688:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619912.005824:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccd80 going to sleep for 11 seconds 00000100:00080000:0.0:1788619912.005907:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788619912.011549:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619912.012244:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571375488 00000100:00100000:0.0:1788619912.012260:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571375488 00002000:00100000:0.0:1788619912.012373:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6f87b80 x1875503571375488/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:8/0 lens 488/0 e 0 to 0 dl 1788619923 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619912.012386:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6f87b80 with x1875503571375488 00000100:00100000:0.0:1788619912.012424:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 455 00000100:00100000:0.0:1788619912.012435:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6f87b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2354:x1875503571375488:12345-192.168.201.5@tcp:3:fsx.0 00000100:00080000:3.0:1788619912.012939:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1788619912.013074:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6f87b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2354:x1875503571375488:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 638us (1538us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619912.013085:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 455 00002000:00100000:0.0:1788619912.013094:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6f87b80 x1875503571375488/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:8/0 lens 488/4536 e 0 to 0 dl 1788619923 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619912.013104:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6f87b80 with x1875503571375488 00000100:00100000:2.0:1788619912.013284:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cdbb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589074688:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619912.013351:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619912.013479:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6f86a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+17:2354:x1875503571375360:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 8009us (8297us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619912.013490:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 273 00000100:00100000:3.0:1788619912.017808:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619912.017938:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571375616 00000100:00100000:0.0:1788619912.017953:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571375616 00002000:00100000:0.0:1788619912.018025:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6f86680 x1875503571375616/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:8/0 lens 488/0 e 0 to 0 dl 1788619923 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619912.018038:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6f86680 with x1875503571375616 00000100:00100000:0.0:1788619912.018084:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 456 00000100:00100000:0.0:1788619912.018095:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6f86680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2355:x1875503571375616:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619912.018482:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6f86680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2355:x1875503571375616:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 385us (687us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619912.018490:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 456 00002000:00100000:0.0:1788619912.018498:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6f86680 x1875503571375616/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:8/0 lens 488/4536 e 0 to 0 dl 1788619923 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619912.018507:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6f86680 with x1875503571375616 00000100:00100000:3.0:1788619912.023129:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619912.023369:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571375744 00000100:00100000:0.0:1788619912.023383:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571375744 00002000:00100000:0.0:1788619912.023453:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6f85c00 x1875503571375744/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:8/0 lens 488/0 e 0 to 0 dl 1788619923 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619912.023465:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6f85c00 with x1875503571375744 00000100:00100000:0.0:1788619912.023538:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 457 00000100:00100000:0.0:1788619912.023551:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6f85c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2352:x1875503571375744:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619912.023852:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6f85c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2352:x1875503571375744:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 302us (737us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619912.023861:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 457 00002000:00100000:0.0:1788619912.023880:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6f85c00 x1875503571375744/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:8/0 lens 488/4536 e 0 to 0 dl 1788619923 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619912.023889:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6f85c00 with x1875503571375744 00000100:00100000:3.0:1788619912.028684:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619912.028865:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571375872 00000100:00100000:1.0:1788619912.028890:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571375872 00002000:00100000:1.0:1788619912.028960:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6f84e00 x1875503571375872/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:8/0 lens 488/0 e 0 to 0 dl 1788619923 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619912.028973:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6f84e00 with x1875503571375872 00000100:00100000:1.0:1788619912.029021:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 458 00000100:00100000:1.0:1788619912.029032:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6f84e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2354:x1875503571375872:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619912.029419:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6f84e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2354:x1875503571375872:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 386us (748us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619912.029427:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 458 00002000:00100000:1.0:1788619912.029435:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6f84e00 x1875503571375872/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:8/0 lens 488/4536 e 0 to 0 dl 1788619923 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619912.029444:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6f84e00 with x1875503571375872 00000100:00100000:3.0:1788619912.035098:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619912.035801:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571376000 00000100:00100000:1.0:1788619912.035825:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571376000 00002000:00100000:1.0:1788619912.035912:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6f85500 x1875503571376000/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:8/0 lens 488/0 e 0 to 0 dl 1788619923 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619912.035925:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6f85500 with x1875503571376000 00000100:00100000:1.0:1788619912.036039:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 459 00000100:00100000:1.0:1788619912.036051:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6f85500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2353:x1875503571376000:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619912.036425:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6f85500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2353:x1875503571376000:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 374us (1339us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619912.036455:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 459 00002000:00100000:1.0:1788619912.036463:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6f85500 x1875503571376000/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:8/0 lens 488/4536 e 0 to 0 dl 1788619923 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619912.036472:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6f85500 with x1875503571376000 00000100:00100000:3.0:1788619912.041162:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619912.041383:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571376128 00000100:00100000:0.0:1788619912.041397:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571376128 00002000:00100000:0.0:1788619912.041466:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6f87480 x1875503571376128/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:8/0 lens 488/0 e 0 to 0 dl 1788619923 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619912.041480:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6f87480 with x1875503571376128 00000100:00100000:0.0:1788619912.041525:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 460 00000100:00100000:0.0:1788619912.041537:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6f87480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2355:x1875503571376128:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619912.041849:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6f87480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2355:x1875503571376128:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 313us (711us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619912.041857:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 460 00002000:00100000:0.0:1788619912.041865:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6f87480 x1875503571376128/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:8/0 lens 488/4536 e 0 to 0 dl 1788619923 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619912.041883:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6f87480 with x1875503571376128 00000100:00100000:3.0:1788619912.046919:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619912.047238:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571376256 00000100:00100000:1.0:1788619912.047253:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571376256 00002000:00100000:1.0:1788619912.047322:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6f85180 x1875503571376256/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:8/0 lens 488/0 e 0 to 0 dl 1788619923 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619912.047380:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6f85180 with x1875503571376256 00000100:00100000:1.0:1788619912.047418:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 461 00000100:00100000:1.0:1788619912.047429:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6f85180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2352:x1875503571376256:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:1.0:1788619912.051910:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 962214 -> 962214 00000004:00010000:1.0:1788619912.051925:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619911 -> 1788619911 00000004:00010000:1.0:1788619912.051928:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619911 -> 1788619908 00000004:00010000:1.0:1788619912.051931:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619911 -> 1788619911 00000004:00010000:1.0:1788619912.051933:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 518 -> 518 00000100:00100000:1.0:1788619912.052240:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6f85180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2352:x1875503571376256:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 4802us (5327us total) trans 4294967611 rc 0/0 00000100:00100000:1.0:1788619912.052254:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 461 00002000:00100000:1.0:1788619912.052271:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6f85180 x1875503571376256/t4294967611(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:8/0 lens 488/448 e 0 to 0 dl 1788619923 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619912.052281:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6f85180 with x1875503571376256 00000100:00100000:3.0:1788619912.057993:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619912.058181:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571376384 00000100:00100000:0.0:1788619912.058196:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571376384 00000100:00100000:0.0:1788619912.058273:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 802 00000100:00100000:0.0:1788619912.058284:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6f87800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2355:x1875503571376384:12345-192.168.201.5@tcp:16:fsx.0 00000001:00080000:1.0:1788619912.117405:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967606 is committed 00000001:00080000:1.0:1788619912.117433:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967601 is committed 00000001:00080000:1.0:1788619912.117483:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967611 is committed 00000001:00080000:1.0:1788619912.117494:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967607 is committed 00000001:00080000:1.0:1788619912.117516:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967609 is committed 00000100:00100000:0.0:1788619912.117761:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6f87800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2355:x1875503571376384:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 59470us (59774us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619912.117779:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 802 00000100:00100000:0.0:1788619912.121642:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619912.121823:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571376512 00000100:00100000:1.0:1788619912.121837:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571376512 00000100:00100000:1.0:1788619912.121919:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 803 00000100:00100000:1.0:1788619912.121931:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c77e0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571376512:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619912.121959:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619912.121986:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4ac00/0x1cd7fe5368734246 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b77d expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619912.122043:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296c00/0x1cd7fe536873423f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b776 expref: 11 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619912.122095:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c7480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589074816:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619912.122222:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963f00 going to sleep for 11 seconds 00000004:00010000:3.0:1788619912.125893:0:5845:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619908 -> 1788619911 00000004:00010000:3.0:1788619912.125906:0:5845:0:(mdt_lvb.c:201:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks: 518 -> 646 00000100:00100000:3.0:1788619912.125941:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c7480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589074816:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619912.125974:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 962214 00010000:00010000:3.0:1788619912.125999:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4ac00/0x1cd7fe5368734246 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b77d expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619912.126025:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619912.126028:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619912.126034:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4ac00/0x1cd7fe5368734246 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b77d expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619912.126052:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f4ac00, rc 0) 00000100:00100000:3.0:1788619912.126254:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c77e0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571376512:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4323us (4625us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619912.126264:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 803 00000100:00100000:1.0:1788619912.130573:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619912.130791:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571376640 00000100:00100000:2.0:1788619912.130803:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571376640 00000100:00100000:0.0:1788619912.131090:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 804 00000100:00100000:0.0:1788619912.131117:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571376640:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619912.131545:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619912.131576:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f600/0x1cd7fe536873424d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b784 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.131630:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296c00/0x1cd7fe536873423f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b776 expref: 11 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619912.131682:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c77e1880 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589074944:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619912.131818:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa300 going to sleep for 11 seconds 00000100:00100000:2.0:1788619912.134927:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c77e1880 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589074944:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619912.134965:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 962214 00010000:00010000:2.0:1788619912.134989:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f600/0x1cd7fe536873424d lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b784 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.135016:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619912.135020:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619912.135026:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f600/0x1cd7fe536873424d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b784 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.135042:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6f600, rc 0) 00000100:00100000:2.0:1788619912.135113:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571376640:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4011us (4555us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619912.135123:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 804 00000100:00100000:2.0:1788619912.139685:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619912.139843:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571376896 00000100:00100000:0.0:1788619912.139856:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571376896 00000100:00100000:0.0:1788619912.139950:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 805 00000100:00100000:0.0:1788619912.139963:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cdbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:14908:x1875503571376896:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:0.0:1788619912.140548:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe5368734254 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.140572:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe536873422a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b761 expref: 8 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.140581:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe5368734254 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.140641:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe536873422a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b761 expref: 8 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.140656:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe536873422a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b761 expref: 8 pid: 5846 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619912.140673:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c77e2a00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589075072:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619912.141023:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619912.145655:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619912.145782:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571377024 00000100:00100000:2.0:1788619912.145795:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571377024 00010000:00010000:2.0:1788619912.145848:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe536873422a lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b761 expref: 9 pid: 5846 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619912.145918:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 274 00000100:00100000:2.0:1788619912.145929:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6f84a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571377024:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619912.145968:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619912.145976:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe536873422a lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b761 expref: 9 pid: 5846 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.145991:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe536873422a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b761 expref: 9 pid: 5846 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.146019:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619912.146024:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe5368734254 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.146036:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe5368734254 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.146045:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe5368734254 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.146068:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619912.146144:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6f84a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571377024:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 216us (505us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619912.146151:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 274 00010000:00010000:0.0:1788619912.147800:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe536873422a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b761 expref: 8 pid: 5846 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619912.147831:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c77e2a00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589075072:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619912.147863:0:5846:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe5368734254 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619912.148580:0:5846:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe5368734254 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619912.148591:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe5368734254 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788619912.148831:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967612 00000004:00010000:1.0:1788619912.149388:0:5846:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 962214 -> 962214 00000004:00010000:1.0:1788619912.149393:0:5846:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619911 -> 1788619912 00000004:00010000:1.0:1788619912.149397:0:5846:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619911 -> 1788619908 00000004:00010000:1.0:1788619912.149399:0:5846:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619911 -> 1788619912 00000004:00010000:1.0:1788619912.149401:0:5846:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 646 -> 646 00000004:00010000:1.0:1788619912.149431:0:5846:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000d8841a8a reply state 00000000965e8dd9 transno 4294967613 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe5368734254 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619912.149443:0:5846:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788619912.149639:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cdbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:14908:x1875503571376896:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 9676us (9964us total) trans 4294967613 rc 0/0 00000100:00100000:1.0:1788619912.149648:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 805 00000020:00080000:1.0:1788619912.149654:0:5846:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788619912.153905:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe5368734254 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.153930:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe5368734254 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.153938:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe5368734254 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.153945:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c6b6ea00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788619912.153951:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe5368734254 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.153981:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe5368734254 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.154004:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619912.154008:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788619912.154015:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe5368734254 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.154022:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe5368734254 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619912.154451:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619912.154548:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571377152 00000100:00100000:2.0:1788619912.154560:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571377152 00000100:00100000:2.0:1788619912.154647:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 618 00000100:00100000:2.0:1788619912.154662:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:14908:x1875503571377152:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619912.154686:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619912.154812:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098665f62/0x1cd7fe536873425b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b78b expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.154879:0:7654:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000d3a77911/0x1cd7fe53687341cf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b722 expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.154890:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 8) 00010000:00010000:2.0:1788619912.154895:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(3 8) 00010000:00010000:2.0:1788619912.154901:0:7654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098665f62/0x1cd7fe536873425b lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b78b expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.154950:0:7654:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000d3a77911/0x1cd7fe53687341cf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b722 expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.154966:0:7654:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000d3a77911/0x1cd7fe53687341cf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b722 expref: 5 pid: 6698 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619912.154983:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cda680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589075200:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619912.155096:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc300 going to sleep for 11 seconds 00000100:00080000:0.0:1788619912.155419:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1788619912.171043:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619912.171431:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cda680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589075200:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619912.171480:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000098665f62/0x1cd7fe536873425b lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b78b expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.171510:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297800, rc 0) 00000100:00100000:2.0:1788619912.171596:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+6:14908:x1875503571377152:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 16938us (17152us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619912.171630:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 618 00000100:00100000:3.0:1788619912.171760:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571377280 00000100:00100000:3.0:1788619912.171769:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571377280 00010000:00010000:3.0:1788619912.171834:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000d3a77911/0x1cd7fe53687341cf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b722 expref: 6 pid: 6698 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619912.171876:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 275 00000100:00100000:3.0:1788619912.171887:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c77e2a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571377280:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619912.171903:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619912.171942:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d3a77911/0x1cd7fe53687341cf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b722 expref: 6 pid: 6698 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619912.171956:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d3a77911/0x1cd7fe53687341cf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b722 expref: 6 pid: 6698 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619912.171984:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3a77911/0x1cd7fe53687341cf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b722 expref: 6 pid: 6698 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619912.172002:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(3 8) 00010000:00010000:3.0:1788619912.172007:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(3 8) 00010000:00010000:3.0:1788619912.172017:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000098665f62/0x1cd7fe536873425b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b78b expref: 5 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619912.172030:0:13330:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098665f62/0x1cd7fe536873425b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b78b expref: 5 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619912.172074:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000098665f62/0x1cd7fe536873425b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b78b expref: 5 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619912.172092:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501c000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589075328:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619912.172187:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164c00 going to sleep for 11 seconds 00000100:00080000:2.0:1788619912.172417:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1788619912.176509:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501c000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589075328:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619912.176544:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619912.176616:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c77e2a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571377280:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 4730us (5586us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619912.176630:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 275 00000100:00100000:0.0:1788619912.185523:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619912.185648:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571377408 00000100:00100000:2.0:1788619912.185662:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571377408 00002000:00010000:2.0:1788619912.185729:0:6701:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1053309->18446744073709551615) 00002000:00100000:2.0:1788619912.185735:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c77e3b80 with x1875503571377408 ext(1053309->18446744073709551615) 00002000:00010000:2.0:1788619912.185761:0:6701:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c77e3b80. 00000100:00100000:2.0:1788619912.185802:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 348 00000100:00100000:2.0:1788619912.185814:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c77e3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571377408:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:2.0:1788619912.186309:0:6701:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1356034 -> 1053309 00002000:00010000:2.0:1788619912.186315:0:6701:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619911 -> 1788619912 00002000:00010000:2.0:1788619912.186318:0:6701:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619911 -> 1788619908 00002000:00010000:2.0:1788619912.186319:0:6701:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619911 -> 1788619912 00002000:00010000:2.0:1788619912.186321:0:6701:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 263 -> 135 00000100:00100000:2.0:1788619912.186478:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c77e3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571377408:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 664us (968us total) trans 4294967394 rc 0/0 00000100:00100000:2.0:1788619912.186488:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 348 00002000:00010000:2.0:1788619912.186493:0:6701:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1053309->18446744073709551615) 00002000:00100000:2.0:1788619912.186497:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c77e3b80 with x1875503571377408 ext(1053309->18446744073709551615) 00002000:00010000:2.0:1788619912.186502:0:6701:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c77e3b80. 00000100:00100000:2.0:1788619912.198652:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619912.198887:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571377536 00000100:00100000:3.0:1788619912.198901:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571377536 00000100:00100000:3.0:1788619912.198986:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 806 00000100:00100000:3.0:1788619912.198997:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cda680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+8:14908:x1875503571377536:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619912.199043:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619912.199070:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe5368734262 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b792 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619912.199469:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368734269 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619912.199490:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368734269 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619912.199504:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368734269 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788619912.199541:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1053309 00000004:00200000:3.0:1788619912.199546:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788619912.199570:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368734269 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619912.199603:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe5368734262 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b792 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619912.199631:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368734269 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b792 expref: 9 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619912.199641:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619912.199644:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788619912.199648:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683ce00, rc 0) 00000100:00100000:3.0:1788619912.199735:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cda680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571377536:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 739us (1098us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619912.199744:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 806 00000100:00100000:2.0:1788619912.203428:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619912.203484:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619912.203629:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571377792 00000100:00100000:1.0:1788619912.203642:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571377792 00000100:00100000:1.0:1788619912.203722:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 619 00000100:00100000:1.0:1788619912.203733:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571377792:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619912.203762:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619912.203785:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d124dcf7/0x1cd7fe5368734270 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b7a0 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619912.203858:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c4a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589075456:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619912.203955:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963f00 going to sleep for 11 seconds 00000100:00100000:3.0:1788619912.206910:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571377664 00000100:00100000:3.0:1788619912.206931:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571377664 00000100:00100000:3.0:1788619912.207019:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 807 00000100:00100000:3.0:1788619912.207030:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571377664:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619912.207057:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619912.207086:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368734277 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b799 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619912.207133:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296c00/0x1cd7fe536873423f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b776 expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619912.207813:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501c000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589075584:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619912.208942:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164c00 going to sleep for 11 seconds 00002000:00010000:0.0:1788619912.209480:0:6698:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619908 -> 1788619911 00000100:00100000:0.0:1788619912.209515:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c4a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589075456:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619912.209563:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d124dcf7/0x1cd7fe5368734270 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b7a0 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.209595:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d124dcf7/0x1cd7fe5368734270 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b7a0 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.209639:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f4a200, rc 0) 00000100:00100000:0.0:1788619912.209708:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571377792:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5975us (6221us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619912.209718:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 619 00000004:00010000:3.0:1788619912.214522:0:9417:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619908 -> 1788619911 00000100:00100000:3.0:1788619912.214569:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501c000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589075584:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619912.214600:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 962214 00010000:00010000:3.0:1788619912.214626:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368734277 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b799 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619912.214652:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619912.214655:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619912.214662:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368734277 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b799 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619912.214681:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683dc00, rc 0) 00000100:00100000:3.0:1788619912.214772:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571377664:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7743us (11356us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619912.214808:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 807 00000100:00100000:2.0:1788619912.223118:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619912.224224:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619912.224391:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571378048 00000100:00100000:3.0:1788619912.224406:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571378048 00000100:00100000:3.0:1788619912.224485:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 620 00000100:00100000:3.0:1788619912.224499:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c77e2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571378048:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619912.224524:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619912.224550:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001391204f/0x1cd7fe536873427e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b7ae expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619912.224633:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589075712:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619912.224753:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164c00 going to sleep for 11 seconds 00000100:00100000:3.0:1788619912.228471:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589075712:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619912.228527:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001391204f/0x1cd7fe536873427e lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b7ae expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619912.228560:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001391204f/0x1cd7fe536873427e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b7ae expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619912.228576:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683da00, rc 0) 00000100:00100000:3.0:1788619912.228657:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c77e2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571378048:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4161us (4438us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619912.228667:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 620 00000100:00100000:1.0:1788619912.231529:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571377920 00000100:00100000:1.0:1788619912.231551:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571377920 00000100:00100000:1.0:1788619912.231717:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 808 00000100:00100000:1.0:1788619912.231731:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571377920:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619912.231768:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619912.231798:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4be00/0x1cd7fe5368734285 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b7a7 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619912.231848:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296c00/0x1cd7fe536873423f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b776 expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619912.231947:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c4a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589075840:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619912.232076:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963840 going to sleep for 11 seconds 00000100:00100000:1.0:1788619912.237660:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c4a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589075840:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619912.237701:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 962214 00010000:00010000:1.0:1788619912.237728:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4be00/0x1cd7fe5368734285 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b7a7 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619912.237767:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619912.237771:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619912.237778:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4be00/0x1cd7fe5368734285 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b7a7 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619912.237801:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f4be00, rc 0) 00000100:00100000:1.0:1788619912.240564:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571377920:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8819us (17444us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619912.240580:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 808 00000100:00100000:0.0:1788619912.244641:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619912.244686:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619912.244763:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571378304 00000100:00100000:3.0:1788619912.244775:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571378304 00000100:00100000:3.0:1788619912.244857:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 621 00000100:00100000:3.0:1788619912.244878:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c77e1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571378304:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619912.244901:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619912.244930:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001391204f/0x1cd7fe536873428c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b7bc expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619912.245002:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589075968:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619912.245096:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164c00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619912.250562:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589075968:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619912.250640:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001391204f/0x1cd7fe536873428c lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b7bc expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.250672:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001391204f/0x1cd7fe536873428c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b7bc expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.250686:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683da00, rc 0) 00000100:00100000:2.0:1788619912.250762:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c77e1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571378304:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5887us (6124us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619912.250772:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 621 00000100:00100000:1.0:1788619912.253715:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571378176 00000100:00100000:1.0:1788619912.253738:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571378176 00000100:00100000:0.0:1788619912.254079:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 809 00000100:00100000:0.0:1788619912.254093:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c77e1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571378176:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619912.254121:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619912.254166:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f600/0x1cd7fe5368734293 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b7b5 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.254217:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296c00/0x1cd7fe536873423f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b776 expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619912.254270:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c77e0e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589076096:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619912.254458:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619912.263805:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c77e0e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589076096:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619912.263848:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 962214 00010000:00010000:0.0:1788619912.263886:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f600/0x1cd7fe5368734293 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b7b5 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.263912:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619912.263994:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619912.264003:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f600/0x1cd7fe5368734293 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b7b5 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.264023:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6f600, rc 0) 00000100:00100000:0.0:1788619912.264256:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c77e1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571378176:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 10163us (19565us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619912.264266:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 809 00000100:00100000:3.0:1788619912.269787:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619912.269832:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619912.270110:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571378560 00000100:00100000:2.0:1788619912.270124:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571378560 00000100:00100000:2.0:1788619912.270219:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 622 00000100:00100000:2.0:1788619912.270231:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571378560:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619912.270255:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619912.270281:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003086a8b7/0x1cd7fe536873429a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b7ca expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619912.270391:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cdb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589076224:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619912.270497:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc300 going to sleep for 11 seconds 00000100:00100000:0.0:1788619912.271948:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571378432 00000100:00100000:0.0:1788619912.271962:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571378432 00000100:00100000:0.0:1788619912.272021:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 810 00000100:00100000:0.0:1788619912.272032:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571378432:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619912.272056:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619912.272079:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e800/0x1cd7fe53687342a1 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b7c3 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.272125:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296c00/0x1cd7fe536873423f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b776 expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619912.272579:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c77e0e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589076352:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619912.272697:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619912.280143:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c77e0e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589076352:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619912.280194:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 962214 00010000:00010000:0.0:1788619912.280223:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e800/0x1cd7fe53687342a1 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b7c3 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.280252:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619912.280274:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619912.280282:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e800/0x1cd7fe53687342a1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b7c3 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.280301:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6e800, rc 0) 00000100:00100000:0.0:1788619912.280454:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571378432:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8422us (10680us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619912.280466:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 810 00000100:00100000:0.0:1788619912.283819:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cdb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589076224:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619912.283886:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003086a8b7/0x1cd7fe536873429a lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b7ca expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.285368:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003086a8b7/0x1cd7fe536873429a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b7ca expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.285395:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296400, rc 0) 00000100:00100000:0.0:1788619912.285636:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571378560:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 15404us (15799us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619912.285647:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 622 00000100:00100000:3.0:1788619912.292637:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619912.292878:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571378688 00000100:00100000:1.0:1788619912.292893:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571378688 00000100:00100000:1.0:1788619912.292984:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 811 00000100:00100000:1.0:1788619912.292998:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571378688:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619912.293028:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619912.293059:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a800/0x1cd7fe53687342a8 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b7d1 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619912.293127:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296c00/0x1cd7fe536873423f lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b776 expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619912.293151:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a800/0x1cd7fe53687342a8 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b7d1 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619912.293199:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296c00/0x1cd7fe536873423f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b776 expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619912.293216:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296c00/0x1cd7fe536873423f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b776 expref: 13 pid: 9417 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619912.293234:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c4a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589076480:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619912.293633:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963840 going to sleep for 11 seconds 00000100:00100000:1.0:1788619912.299848:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c4a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589076480:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619912.299922:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a800/0x1cd7fe53687342a8 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b7d1 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619912.300158:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00010000:00010000:1.0:1788619912.300215:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619912.300219:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619912.300225:0:5846:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a800/0x1cd7fe53687342a8 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b7d1 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619912.300240:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f4a800, rc 0) 00000100:00100000:0.0:1788619912.300372:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571378816 00000100:00100000:0.0:1788619912.300383:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571378816 00010000:00010000:0.0:1788619912.300443:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296c00/0x1cd7fe536873423f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b776 expref: 14 pid: 9417 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619912.300489:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 276 00000100:00100000:0.0:1788619912.300501:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571378816:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619912.300542:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619912.300578:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296c00/0x1cd7fe536873423f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b776 expref: 14 pid: 9417 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.300593:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296c00/0x1cd7fe536873423f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b776 expref: 14 pid: 9417 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.300620:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296c00/0x1cd7fe536873423f lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b776 expref: 14 pid: 9417 timeout: 504 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.300639:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619912.300644:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a800/0x1cd7fe53687342a8 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b7d1 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.300655:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a800/0x1cd7fe53687342a8 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b7d1 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.300665:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a800/0x1cd7fe53687342a8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b7d1 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.300721:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a800/0x1cd7fe53687342a8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b7d1 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619912.300740:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c77e0e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589076608:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619912.302102:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:14908:x1875503571378688:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 9096us (9470us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619912.302201:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 811 00000100:00100000:0.0:1788619912.302341:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619912.308005:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619912.308142:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c77e0e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589076608:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619912.308173:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619912.308253:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571378816:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 7753us (8095us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619912.308266:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 276 00000100:00100000:3.0:1788619912.310037:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571378944 00000100:00100000:3.0:1788619912.310057:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571378944 00000100:00100000:3.0:1788619912.310173:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 462 00000100:00100000:3.0:1788619912.310186:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571378944:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:3.0:1788619912.311127:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 962214 -> 962214 00000004:00010000:3.0:1788619912.311294:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619912 -> 1788619912 00000004:00010000:3.0:1788619912.311297:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619911 -> 1788619911 00000004:00010000:3.0:1788619912.311299:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619912 -> 1788619912 00000004:00010000:3.0:1788619912.311302:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 646 -> 646 00000100:00100000:3.0:1788619912.311531:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571378944:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 1344us (3604us total) trans 4294967614 rc 0/0 00000100:00100000:3.0:1788619912.311542:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 462 00000100:00100000:1.0:1788619912.317090:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619912.317253:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571379072 00000100:00100000:0.0:1788619912.317267:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571379072 00000100:00100000:0.0:1788619912.317404:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 623 00000100:00100000:0.0:1788619912.317418:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571379072:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619912.317568:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619912.317594:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d058bad0/0x1cd7fe53687342af lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b7d8 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619912.317684:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c77e1180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589076736:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619912.317925:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaacc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619912.321539:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c77e1180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589076736:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619912.321594:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d058bad0/0x1cd7fe53687342af lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b7d8 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.321629:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d058bad0/0x1cd7fe53687342af lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b7d8 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.321646:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6e800, rc 0) 00000100:00100000:2.0:1788619912.321835:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571379072:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4418us (4757us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619912.321846:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 623 00000100:00100000:1.0:1788619912.328613:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619912.328829:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571379200 00000100:00100000:3.0:1788619912.328843:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571379200 00000100:00100000:3.0:1788619912.329123:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 624 00000100:00100000:3.0:1788619912.329146:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5026680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571379200:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619912.329171:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619912.329196:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe53687342b6 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b7df expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619912.329277:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cd5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589076864:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619912.329446:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164300 going to sleep for 11 seconds 00000100:00100000:3.0:1788619912.334235:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cd5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589076864:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619912.334296:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe53687342b6 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b7df expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619912.334682:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe53687342b6 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b7df expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619912.334819:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683dc00, rc 0) 00000100:00100000:3.0:1788619912.335030:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5026680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571379200:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5885us (6432us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619912.335040:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 624 00000100:00100000:1.0:1788619912.339579:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619912.339683:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571379328 00000100:00100000:0.0:1788619912.339819:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571379328 00000100:00100000:0.0:1788619912.339920:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 812 00000100:00100000:0.0:1788619912.340062:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7581500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571379328:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619912.340097:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619912.340126:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f600/0x1cd7fe53687342bd lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b7e6 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.340315:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fc00/0x1cd7fe53687342c4 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.340516:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fc00/0x1cd7fe53687342c4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.340537:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fc00/0x1cd7fe53687342c4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619912.340576:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1053309 00000004:00200000:0.0:1788619912.340581:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619912.340605:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fc00/0x1cd7fe53687342c4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.340661:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f600/0x1cd7fe53687342bd lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b7e6 expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.340683:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fc00/0x1cd7fe53687342c4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b7e6 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.340884:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619912.340887:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619912.340892:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6fc00, rc 0) 00000100:00100000:0.0:1788619912.341144:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7581500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571379328:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 1212us (1578us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619912.341153:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 812 00000100:00100000:1.0:1788619912.346122:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619912.346288:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571379456 00000100:00100000:0.0:1788619912.346301:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571379456 00000100:00100000:0.0:1788619912.346600:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 813 00000100:00100000:0.0:1788619912.346612:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5696a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571379456:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619912.346638:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619912.346665:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fe00/0x1cd7fe53687342cb lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b7ed expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.346726:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a800/0x1cd7fe53687342a8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b7d1 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619912.346788:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c77e3100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589076992:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619912.346902:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaacc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619912.350199:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c77e3100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589076992:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619912.350260:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 962214 00010000:00010000:0.0:1788619912.350290:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fe00/0x1cd7fe53687342cb lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b7ed expref: 15 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.350320:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619912.350356:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619912.350364:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fe00/0x1cd7fe53687342cb lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b7ed expref: 15 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.350386:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6fe00, rc 0) 00000100:00100000:0.0:1788619912.350473:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5696a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571379456:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3862us (4364us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619912.350485:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 813 00000100:00100000:1.0:1788619912.355710:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619912.355881:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571379584 00000100:00100000:2.0:1788619912.355895:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571379584 00000100:00100000:2.0:1788619912.355987:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 814 00000100:00100000:2.0:1788619912.356001:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e647aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571379584:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619912.356030:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619912.356058:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296600/0x1cd7fe53687342d2 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b7f4 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.356112:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a800/0x1cd7fe53687342a8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b7d1 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619912.356179:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cd9180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589077120:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619912.356303:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccf00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619912.362836:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cd9180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589077120:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619912.362917:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 962214 00010000:00010000:2.0:1788619912.362945:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296600/0x1cd7fe53687342d2 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b7f4 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.362974:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619912.362977:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619912.362984:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296600/0x1cd7fe53687342d2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b7f4 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.363002:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296600, rc 0) 00000100:00100000:2.0:1788619912.364034:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e647aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571379584:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8028us (8341us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619912.364047:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 814 00000100:00100000:3.0:1788619912.368891:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619912.369584:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571379712 00000100:00100000:0.0:1788619912.369603:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571379712 00000100:00100000:0.0:1788619912.369694:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 815 00000100:00100000:0.0:1788619912.369708:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571379712:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619912.369737:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619912.369781:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f600/0x1cd7fe53687342d9 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b7fb expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.369835:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a800/0x1cd7fe53687342a8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b7d1 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619912.369904:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c77e3100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589077248:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619912.370133:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaacc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619912.372888:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c77e3100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589077248:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619912.372935:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 962214 00010000:00010000:3.0:1788619912.373029:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f600/0x1cd7fe53687342d9 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b7fb expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619912.373084:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619912.373088:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619912.373095:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f600/0x1cd7fe53687342d9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b7fb expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619912.373114:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6f600, rc 0) 00000100:00100000:3.0:1788619912.373321:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571379712:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3614us (4453us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619912.373398:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 815 00000100:00100000:0.0:1788619912.377418:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619912.377741:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571379840 00000100:00100000:3.0:1788619912.377755:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571379840 00000100:00100000:3.0:1788619912.377825:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 816 00000100:00100000:3.0:1788619912.377836:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c77e0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2353:x1875503571379840:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619912.377858:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619912.377893:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe53687342e0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b802 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619912.377942:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a800/0x1cd7fe53687342a8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b7d1 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619912.378114:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cd6680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589077376:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619912.378323:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164300 going to sleep for 11 seconds 00000100:00100000:3.0:1788619912.382279:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cd6680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589077376:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619912.382319:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 962214 00010000:00010000:3.0:1788619912.382484:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe53687342e0 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b802 expref: 15 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619912.382539:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619912.382542:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619912.382549:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe53687342e0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b802 expref: 15 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619912.382567:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683dc00, rc 0) 00000100:00100000:2.0:1788619912.382890:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c77e0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2353:x1875503571379840:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5052us (5482us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619912.382901:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 816 00000100:00100000:0.0:1788619912.393701:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619912.393885:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571379968 00000100:00100000:2.0:1788619912.393898:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571379968 00000100:00100000:2.0:1788619912.394064:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 625 00000100:00100000:2.0:1788619912.394077:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c77e3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571379968:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619912.394102:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619912.394126:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6252eaa/0x1cd7fe53687342e7 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b809 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619912.394286:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cd9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589077504:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619912.394444:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccf00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619912.402556:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cd9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589077504:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619912.402649:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a6252eaa/0x1cd7fe53687342e7 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b809 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.402684:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6252eaa/0x1cd7fe53687342e7 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b809 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.402700:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297600, rc 0) 00000100:00100000:2.0:1788619912.402774:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c77e3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571379968:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8699us (9087us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619912.402783:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 625 00000100:00100000:3.0:1788619912.406527:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619912.406996:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571380096 00000100:00100000:0.0:1788619912.407010:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571380096 00000100:00100000:0.0:1788619912.407088:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 626 00000100:00100000:0.0:1788619912.407100:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571380096:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619912.407124:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619912.411652:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000738676cd/0x1cd7fe53687342ee lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b810 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619912.411768:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c77e1180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589077632:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619912.411919:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619912.423047:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c77e1180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589077632:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619912.423103:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000738676cd/0x1cd7fe53687342ee lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b810 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.423150:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000738676cd/0x1cd7fe53687342ee lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b810 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.423167:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6ec00, rc 0) 00000100:00100000:2.0:1788619912.423250:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571380096:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 16150us (16735us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619912.423260:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 626 00000100:00100000:3.0:1788619912.430808:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619912.430937:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571380224 00000100:00100000:0.0:1788619912.430951:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571380224 00000100:00100000:0.0:1788619912.431024:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 627 00000100:00100000:0.0:1788619912.431036:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571380224:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619912.431062:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619912.431092:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000112ced4b/0x1cd7fe53687342f5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b817 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619912.431175:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c77e0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589077760:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619912.431286:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa780 going to sleep for 11 seconds 00000100:00100000:0.0:1788619912.435537:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c77e0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589077760:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619912.435605:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000112ced4b/0x1cd7fe53687342f5 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b817 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.435643:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000112ced4b/0x1cd7fe53687342f5 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b817 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.435693:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6fa00, rc 0) 00000100:00100000:0.0:1788619912.435788:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571380224:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4752us (4990us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619912.435798:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 627 00000100:00100000:1.0:1788619912.443084:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619912.443937:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571380352 00000100:00100000:3.0:1788619912.443953:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571380352 00000100:00100000:3.0:1788619912.444039:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 628 00000100:00100000:3.0:1788619912.444051:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571380352:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619912.444075:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619912.444097:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe53687342fc lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b81e expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619912.444375:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cd4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589077888:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619912.445042:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164300 going to sleep for 11 seconds 00000100:00100000:1.0:1788619912.451241:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cd4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589077888:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619912.451303:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe53687342fc lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b81e expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619912.451356:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe53687342fc lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b81e expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619912.451373:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683dc00, rc 0) 00000100:00100000:1.0:1788619912.451581:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571380352:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7531us (8512us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619912.451592:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 628 00000100:00100000:2.0:1788619912.456186:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619912.456381:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571380480 00000100:00100000:1.0:1788619912.456398:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571380480 00002000:00100000:1.0:1788619912.456480:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5cd9180 x1875503571380480/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:8/0 lens 488/0 e 0 to 0 dl 1788619923 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619912.456494:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5cd9180 with x1875503571380480 00000100:00100000:1.0:1788619912.456539:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 463 00000100:00100000:1.0:1788619912.456571:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571380480:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619912.457070:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571380480:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 500us (897us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619912.457079:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 463 00002000:00100000:1.0:1788619912.457087:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5cd9180 x1875503571380480/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:8/0 lens 488/440 e 0 to 0 dl 1788619923 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619912.457096:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5cd9180 with x1875503571380480 00000100:00100000:0.0:1788619912.463673:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619912.464021:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571380608 00000100:00100000:2.0:1788619912.464035:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571380608 00000100:00100000:2.0:1788619912.464114:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 629 00000100:00100000:2.0:1788619912.464155:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c77e0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571380608:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619912.464180:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619912.464204:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe5368734303 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b825 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619912.464285:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cdb800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589078016:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619912.464414:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccf00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619912.468257:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cdb800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589078016:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619912.468315:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe5368734303 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b825 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.468392:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe5368734303 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b825 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.468410:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296e00, rc 0) 00000100:00100000:2.0:1788619912.468491:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c77e0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571380608:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4338us (4832us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619912.468500:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 629 00000100:00100000:3.0:1788619912.474012:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619912.474178:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571380736 00000100:00100000:2.0:1788619912.474192:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571380736 00002000:00100000:2.0:1788619912.474262:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5cd5180 x1875503571380736/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:8/0 lens 488/0 e 0 to 0 dl 1788619923 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619912.474275:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5cd5180 with x1875503571380736 00000100:00100000:2.0:1788619912.474323:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 464 00000100:00100000:2.0:1788619912.474365:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571380736:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619912.474794:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571380736:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 430us (798us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619912.474803:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 464 00002000:00100000:2.0:1788619912.474811:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5cd5180 x1875503571380736/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:8/0 lens 488/440 e 0 to 0 dl 1788619923 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619912.474820:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5cd5180 with x1875503571380736 00000100:00100000:0.0:1788619912.480488:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619912.480618:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571380864 00000100:00100000:2.0:1788619912.480632:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571380864 00000100:00100000:2.0:1788619912.480718:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 630 00000100:00100000:2.0:1788619912.480730:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c77e1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:14908:x1875503571380864:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619912.480997:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619912.481022:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe536873430a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b82c expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.481068:0:7654:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000098665f62/0x1cd7fe536873425b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b78b expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.481079:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=1s v=(3 8) 00010000:00010000:2.0:1788619912.481084:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(4 8) 00010000:00010000:2.0:1788619912.481090:0:7654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe536873430a lrc: 4/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->1052671] (req 1048576->1052671) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b82c expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.481149:0:7654:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000098665f62/0x1cd7fe536873425b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b78b expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.481166:0:7654:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000098665f62/0x1cd7fe536873425b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b78b expref: 6 pid: 7654 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619912.481219:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cdb800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589078144:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619912.481382:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccf00 going to sleep for 11 seconds 00000100:00100000:3.0:1788619912.486527:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619912.486920:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571380992 00000100:00100000:0.0:1788619912.486934:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571380992 00010000:00010000:0.0:1788619912.486993:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000098665f62/0x1cd7fe536873425b lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b78b expref: 7 pid: 7654 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619912.487049:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 277 00000100:00100000:0.0:1788619912.487061:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571380992:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619912.487080:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619912.487108:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000098665f62/0x1cd7fe536873425b lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b78b expref: 7 pid: 7654 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.487135:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000098665f62/0x1cd7fe536873425b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b78b expref: 7 pid: 7654 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.487173:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(4 8) 00010000:00010000:0.0:1788619912.487178:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(4 8) 00010000:00010000:0.0:1788619912.487190:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe536873430a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b82c expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.487238:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe536873430a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b82c expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.487292:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe536873430a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b82c expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619912.487311:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c33b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589078272:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619912.487463:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa780 going to sleep for 11 seconds 00010000:00010000:2.0:1788619912.488409:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098665f62/0x1cd7fe536873425b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b78b expref: 7 pid: 7654 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619912.488444:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cdb800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589078144:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619912.488481:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe536873430a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b82c expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.488497:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296e00, rc 0) 00000100:00100000:2.0:1788619912.488567:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c77e1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:14908:x1875503571380864:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7839us (8093us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619912.488577:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 630 00000100:00100000:3.0:1788619912.497050:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619912.497255:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571381120 00000100:00100000:1.0:1788619912.497269:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571381120 00002000:00100000:1.0:1788619912.497386:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1052671) req@ffff8e77c5cd4e00 x1875503571381120/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:8/0 lens 488/0 e 0 to 0 dl 1788619923 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619912.497400:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5cd4e00 with x1875503571381120 ext(1048576->1052671) 00002000:00010000:1.0:1788619912.497407:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5cd4e00 00000100:00100000:1.0:1788619912.497582:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 349 00000100:00100000:1.0:1788619912.497605:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571381120:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619912.498171:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571381120:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 576us (1132us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619912.498180:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 349 00002000:00100000:1.0:1788619912.498188:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1052671) req@ffff8e77c5cd4e00 x1875503571381120/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:8/0 lens 488/4536 e 0 to 0 dl 1788619923 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619912.498198:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5cd4e00 with x1875503571381120 ext(1048576->1052671) 00002000:00010000:1.0:1788619912.498203:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5cd4e00 00000100:00100000:0.0:1788619912.498401:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c33b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589078272:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619912.498442:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619912.499102:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571380992:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 12040us (12587us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619912.499115:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 277 00000100:00100000:3.0:1788619912.505940:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619912.506056:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571381248 00000100:00100000:1.0:1788619912.506070:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571381248 00002000:00100000:1.0:1788619912.506275:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1052672->1056767) req@ffff8e79010ff100 x1875503571381248/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:8/0 lens 488/0 e 0 to 0 dl 1788619923 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619912.506287:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e79010ff100 with x1875503571381248 ext(1052672->1056767) 00002000:00010000:1.0:1788619912.506296:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e79010ff100 00000100:00100000:1.0:1788619912.506408:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 350 00000100:00100000:1.0:1788619912.506421:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010ff100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571381248:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619912.506990:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010ff100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571381248:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 568us (1063us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619912.507001:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 350 00002000:00100000:1.0:1788619912.507010:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1052672->1056767) req@ffff8e79010ff100 x1875503571381248/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:8/0 lens 488/4536 e 0 to 0 dl 1788619923 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619912.507019:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e79010ff100 with x1875503571381248 ext(1052672->1056767) 00002000:00010000:1.0:1788619912.507024:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e79010ff100 00000100:00100000:1.0:1788619912.514711:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619912.514884:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571381376 00000100:00100000:0.0:1788619912.514899:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571381376 00000100:00100000:0.0:1788619912.514972:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 631 00000100:00100000:0.0:1788619912.514985:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b10a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:14908:x1875503571381376:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619912.515007:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619912.515032:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000244b3d37/0x1cd7fe5368734311 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b833 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.515075:0:6698:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe536873430a lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b82c expref: 5 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.515107:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(4 8) 00010000:00010000:0.0:1788619912.515113:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(4 8) 00010000:00010000:0.0:1788619912.515119:0:6698:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000244b3d37/0x1cd7fe5368734311 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1269760->1294335] (req 1269760->1294335) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b833 expref: 7 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.515176:0:6698:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe536873430a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b82c expref: 5 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.515194:0:6698:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe536873430a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b82c expref: 5 pid: 7654 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619912.515211:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c33b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589078400:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619912.515313:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa780 going to sleep for 11 seconds 00000100:00100000:2.0:1788619912.523234:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c33b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589078400:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619912.523283:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000244b3d37/0x1cd7fe5368734311 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1269760->1294335] (req 1269760->1294335) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b833 expref: 7 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.523308:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6f400, rc 0) 00000100:00100000:2.0:1788619912.523406:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b10a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:14908:x1875503571381376:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8422us (8708us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619912.523416:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 631 00000100:00100000:3.0:1788619912.527218:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619912.527394:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571381504 00000100:00100000:0.0:1788619912.527408:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571381504 00010000:00010000:0.0:1788619912.527479:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe536873430a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b82c expref: 6 pid: 7654 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619912.527538:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 278 00000100:00100000:0.0:1788619912.527550:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8020450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571381504:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619912.527571:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619912.527610:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe536873430a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b82c expref: 6 pid: 7654 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.527650:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe536873430a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b82c expref: 6 pid: 7654 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.527680:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe536873430a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b82c expref: 6 pid: 7654 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.527702:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(4 8) 00010000:00010000:0.0:1788619912.527706:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(4 8) 00010000:00010000:0.0:1788619912.527715:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000244b3d37/0x1cd7fe5368734311 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1269760->1294335) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b833 expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.527726:0:5831:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000244b3d37/0x1cd7fe5368734311 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1269760->1294335) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b833 expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.527774:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000244b3d37/0x1cd7fe5368734311 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1269760->1294335) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b833 expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619912.527790:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c32680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5831:1875503589078528:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619912.527901:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa300 going to sleep for 11 seconds 00000100:00100000:0.0:1788619912.532583:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c32680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5831:1875503589078528:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619912.532693:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619912.532816:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8020450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571381504:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 5266us (5613us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619912.532828:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 278 00000100:00100000:0.0:1788619912.535689:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619912.535892:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571381632 00000100:00100000:2.0:1788619912.536053:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571381632 00000100:00100000:2.0:1788619912.536130:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 817 00000100:00100000:2.0:1788619912.536142:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c32680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571381632:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619912.536169:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619912.536195:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297800/0x1cd7fe5368734318 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b83a expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.536245:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a800/0x1cd7fe53687342a8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b7d1 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619912.536296:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cdb800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589078656:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619912.536663:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccf00 going to sleep for 11 seconds 00000004:00010000:2.0:1788619912.539765:0:5846:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619911 -> 1788619912 00000100:00100000:2.0:1788619912.539948:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cdb800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589078656:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619912.539980:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 962214 00010000:00010000:2.0:1788619912.540006:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297800/0x1cd7fe5368734318 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b83a expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.540111:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619912.540115:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619912.540122:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297800/0x1cd7fe5368734318 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b83a expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.540141:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297800, rc 0) 00000100:00100000:2.0:1788619912.540226:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c32680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571381632:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4085us (4549us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619912.540237:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 817 00000100:00100000:2.0:1788619912.555899:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619912.556122:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571381760 00000100:00100000:3.0:1788619912.556136:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571381760 00000100:00100000:3.0:1788619912.556279:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 818 00000100:00100000:3.0:1788619912.556347:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cdb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571381760:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619912.556376:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619912.556405:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe536873431f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b841 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619912.556454:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a800/0x1cd7fe53687342a8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b7d1 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619912.556507:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010ff800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589078784:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619912.556624:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164e40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619912.562065:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010ff800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589078784:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619912.562102:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 962214 00010000:00010000:0.0:1788619912.562129:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe536873431f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b841 expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.562155:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619912.562159:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619912.562167:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe536873431f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b841 expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.562184:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d200, rc 0) 00000100:00100000:0.0:1788619912.562267:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cdb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571381760:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5941us (6989us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619912.562278:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 818 00000100:00100000:0.0:1788619912.577618:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619912.577740:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571382016 00000100:00100000:2.0:1788619912.577752:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571382016 00000100:00100000:2.0:1788619912.578126:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 819 00000100:00100000:2.0:1788619912.578142:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c33b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571382016:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:2.0:1788619912.578217:0:9417:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619912.578255:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fc00/0x1cd7fe53687342c4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b7e6 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.578955:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619912.578965:0:9417:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788619912.579954:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297400/0x1cd7fe5368734326 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.579982:0:9417:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368734269 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b792 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.579993:0:9417:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297400/0x1cd7fe5368734326 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.581510:0:9417:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368734269 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b792 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.581541:0:9417:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368734269 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b792 expref: 10 pid: 5845 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619912.581562:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cd8a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589078912:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619912.581696:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccf00 going to sleep for 11 seconds 00000100:00100000:0.0:1788619912.587997:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cd8a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589078912:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619912.588061:0:9417:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297400/0x1cd7fe5368734326 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619912.590648:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619912.591922:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571382144 00000100:00100000:3.0:1788619912.591939:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571382144 00010000:00010000:3.0:1788619912.592010:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368734269 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b792 expref: 11 pid: 5845 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619912.592074:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 279 00000100:00100000:3.0:1788619912.592086:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6f84700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571382144:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619912.592105:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619912.592113:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368734269 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b792 expref: 11 pid: 5845 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619912.592129:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368734269 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b792 expref: 11 pid: 5845 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619912.592155:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368734269 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b792 expref: 11 pid: 5845 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619912.592200:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619912.592206:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297400/0x1cd7fe5368734326 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619912.592218:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297400/0x1cd7fe5368734326 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619912.592228:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297400/0x1cd7fe5368734326 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619912.592266:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788619912.592571:0:9417:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297400/0x1cd7fe5368734326 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.592581:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297400/0x1cd7fe5368734326 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788619912.592758:0:9417:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967615 00000004:00010000:0.0:1788619912.592835:0:9417:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 962214 -> 962214 00000004:00010000:0.0:1788619912.592848:0:9417:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619912 -> 1788619912 00000004:00010000:0.0:1788619912.592851:0:9417:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619912 -> 1788619911 00000004:00010000:0.0:1788619912.592853:0:9417:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619912 -> 1788619912 00000004:00010000:0.0:1788619912.592856:0:9417:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 646 -> 646 00000004:00010000:0.0:1788619912.592877:0:9417:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000005f4d14c8 reply state 00000000548a80e7 transno 4294967616 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297400/0x1cd7fe5368734326 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619912.592887:0:9417:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788619912.593149:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c33b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:14908:x1875503571382016:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 15009us (15544us total) trans 4294967616 rc 0/0 00000100:00100000:0.0:1788619912.593160:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 819 00000020:00080000:0.0:1788619912.593167:0:9417:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1788619912.596054:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6f84700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571382144:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3953us (5403us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619912.596074:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 279 00010000:00010000:2.0:1788619912.601714:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297400/0x1cd7fe5368734326 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.601740:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297400/0x1cd7fe5368734326 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.601749:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297400/0x1cd7fe5368734326 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.601756:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78e0297400 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:2.0:1788619912.601763:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297400/0x1cd7fe5368734326 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.601772:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297400/0x1cd7fe5368734326 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.601805:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619912.601809:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788619912.601815:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297400/0x1cd7fe5368734326 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.601822:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297400/0x1cd7fe5368734326 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619912.602814:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619912.602898:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571382272 00000100:00100000:3.0:1788619912.602910:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571382272 00000100:00100000:3.0:1788619912.603028:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 820 00000100:00100000:3.0:1788619912.603041:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cdad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:14908:x1875503571382272:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619912.603071:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619912.603092:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe536873432d lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b848 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619912.603129:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a800/0x1cd7fe53687342a8 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b7d1 expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619912.603140:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe536873432d lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b848 expref: 16 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619912.603208:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a800/0x1cd7fe53687342a8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b7d1 expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619912.603226:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a800/0x1cd7fe53687342a8 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b7d1 expref: 9 pid: 5846 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619912.603242:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010ff480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589079040:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619912.603394:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164600 going to sleep for 11 seconds 00000100:00100000:0.0:1788619912.621111:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619912.621320:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010ff480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589079040:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619912.621423:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe536873432d lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b848 expref: 16 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619912.621449:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619912.621452:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619912.621458:0:5846:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe536873432d lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b848 expref: 16 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619912.621471:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683ce00, rc 0) 00000100:00100000:3.0:1788619912.621547:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cdad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:14908:x1875503571382272:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 18508us (18742us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619912.621558:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 820 00000100:00100000:2.0:1788619912.621974:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571382400 00000100:00100000:2.0:1788619912.621984:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571382400 00010000:00010000:2.0:1788619912.622037:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a800/0x1cd7fe53687342a8 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b7d1 expref: 10 pid: 5846 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619912.622079:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 280 00000100:00100000:2.0:1788619912.622090:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6f86a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571382400:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619912.622124:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619912.622158:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a800/0x1cd7fe53687342a8 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b7d1 expref: 10 pid: 5846 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.622171:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a800/0x1cd7fe53687342a8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b7d1 expref: 10 pid: 5846 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.622198:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a800/0x1cd7fe53687342a8 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b7d1 expref: 10 pid: 5846 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.622215:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619912.622220:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe536873432d lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b848 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.622231:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe536873432d lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b848 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.622240:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe536873432d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b848 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.622299:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe536873432d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b848 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619912.622314:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cda300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589079168:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619912.623086:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc840 going to sleep for 11 seconds 00000100:00100000:2.0:1788619912.639706:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cda300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589079168:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619912.639769:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619912.639848:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6f86a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571382400:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 17757us (18749us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619912.639873:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 280 00000100:00100000:0.0:1788619912.644273:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619912.644397:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571382528 00000100:00100000:3.0:1788619912.644413:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571382528 00002000:00010000:3.0:1788619912.644486:0:6701:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1624460->18446744073709551615) 00002000:00100000:3.0:1788619912.644493:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6f84a80 with x1875503571382528 ext(1624460->18446744073709551615) 00002000:00010000:3.0:1788619912.644528:0:6701:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6f84a80. 00000100:00100000:3.0:1788619912.644569:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 351 00000100:00100000:3.0:1788619912.644581:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6f84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571382528:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:3.0:1788619912.644925:0:6701:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1053309 -> 1624460 00002000:00010000:3.0:1788619912.644931:0:6701:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619912 -> 1788619912 00002000:00010000:3.0:1788619912.644933:0:6701:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619911 -> 1788619908 00002000:00010000:3.0:1788619912.644936:0:6701:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619912 -> 1788619912 00002000:00010000:3.0:1788619912.644938:0:6701:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 135 -> 135 00000100:00100000:3.0:1788619912.645024:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6f84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571382528:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 445us (768us total) trans 4294967395 rc 0/0 00000100:00100000:3.0:1788619912.645033:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 351 00002000:00010000:3.0:1788619912.645038:0:6701:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1624460->18446744073709551615) 00002000:00100000:3.0:1788619912.645041:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6f84a80 with x1875503571382528 ext(1624460->18446744073709551615) 00002000:00010000:3.0:1788619912.645046:0:6701:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6f84a80. 00000100:00100000:0.0:1788619912.651284:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619912.651448:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571382656 00000100:00100000:3.0:1788619912.651463:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571382656 00000100:00100000:3.0:1788619912.651543:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 821 00000100:00100000:3.0:1788619912.651555:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c77e2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571382656:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619912.651585:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619912.651614:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe5368734334 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b84f expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619912.651741:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe536873433b lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619912.651761:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe536873433b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619912.651795:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe536873433b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788619912.651830:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1624460 00000004:00200000:3.0:1788619912.651835:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788619912.651869:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe536873433b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619912.651903:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe5368734334 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b84f expref: 11 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619912.651927:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe536873433b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b84f expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619912.651937:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619912.651940:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788619912.651945:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683de00, rc 0) 00000100:00100000:3.0:1788619912.652037:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c77e2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:14908:x1875503571382656:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 484us (768us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619912.652044:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 821 00000100:00100000:1.0:1788619912.656641:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619912.656705:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619912.656803:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571382784 00000100:00100000:0.0:1788619912.656817:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571382784 00000100:00100000:0.0:1788619912.656919:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 822 00000100:00100000:0.0:1788619912.656933:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b13b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571382784:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619912.656962:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619912.656989:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e200/0x1cd7fe5368734342 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b856 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.657039:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe536873432d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b848 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619912.657094:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cd4380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589079296:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619912.657214:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaab40 going to sleep for 11 seconds 00000100:00100000:3.0:1788619912.657887:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571382912 00000100:00100000:3.0:1788619912.657895:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571382912 00000100:00100000:3.0:1788619912.657944:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 632 00000100:00100000:3.0:1788619912.657953:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b11500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571382912:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619912.657966:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619912.657981:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad29727f/0x1cd7fe5368734349 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b85d expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619912.658055:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010ff480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589079424:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619912.658111:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164600 going to sleep for 11 seconds 00002000:00010000:3.0:1788619912.662385:0:6696:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619908 -> 1788619911 00000100:00100000:3.0:1788619912.662440:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010ff480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589079424:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619912.662492:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ad29727f/0x1cd7fe5368734349 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b85d expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619912.662527:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad29727f/0x1cd7fe5368734349 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b85d expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619912.662545:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c000, rc 0) 00000100:00100000:3.0:1788619912.662626:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b11500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571382912:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4673us (5919us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619912.662637:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 632 00000100:00100000:0.0:1788619912.664108:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cd4380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589079296:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619912.664153:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 962214 00010000:00010000:0.0:1788619912.664177:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e200/0x1cd7fe5368734342 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b856 expref: 11 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.664203:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619912.664207:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619912.664213:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e200/0x1cd7fe5368734342 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b856 expref: 11 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.664229:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6e200, rc 0) 00000100:00100000:0.0:1788619912.664310:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b13b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571382784:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7379us (7685us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619912.664320:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 822 00000100:00100000:1.0:1788619912.670657:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619912.670763:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571383040 00000100:00100000:3.0:1788619912.670777:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571383040 00000100:00100000:3.0:1788619912.670868:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 823 00000100:00100000:3.0:1788619912.670884:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5770700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571383040:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619912.670927:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619912.670955:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368734350 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b864 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619912.671007:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe536873432d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b848 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619912.671088:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010ff480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589079552:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619912.671212:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164600 going to sleep for 11 seconds 00000100:00100000:2.0:1788619912.676171:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619912.676461:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010ff480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589079552:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619912.676497:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 962214 00010000:00010000:0.0:1788619912.676524:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368734350 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b864 expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.676555:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619912.676559:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619912.676567:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368734350 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b864 expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.676587:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c000, rc 0) 00000100:00100000:0.0:1788619912.676676:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5770700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571383040:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5796us (6036us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619912.676719:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 823 00000100:00100000:3.0:1788619912.676940:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571383168 00000100:00100000:3.0:1788619912.676949:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571383168 00000100:00100000:3.0:1788619912.677009:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 633 00000100:00100000:3.0:1788619912.677021:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cda300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571383168:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619912.677039:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619912.677060:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001fc792ba/0x1cd7fe5368734357 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b86b expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619912.677142:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589079680:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619912.677212:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164240 going to sleep for 11 seconds 00000100:00100000:3.0:1788619912.682299:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589079680:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619912.682393:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001fc792ba/0x1cd7fe5368734357 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b86b expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619912.682437:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001fc792ba/0x1cd7fe5368734357 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b86b expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619912.682454:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d800, rc 0) 00000100:00100000:3.0:1788619912.682531:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cda300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571383168:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5511us (6377us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619912.682542:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 633 00000100:00100000:1.0:1788619912.692022:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619912.692088:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619912.692403:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571383424 00000100:00100000:3.0:1788619912.692416:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571383424 00000100:00100000:0.0:1788619912.692438:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571383296 00000100:00100000:0.0:1788619912.692443:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571383296 00000100:00100000:3.0:1788619912.692492:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 634 00000100:00100000:0.0:1788619912.692492:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 824 00000100:00100000:3.0:1788619912.692502:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ce1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571383424:12345-192.168.201.5@tcp:101:fsx.0 00000100:00100000:0.0:1788619912.692503:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79008ac000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571383296:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619912.692526:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619912.692526:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619912.692549:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007736e663/0x1cd7fe5368734365 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b879 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.692550:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e200/0x1cd7fe536873435e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b872 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.692592:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe536873432d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b848 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619912.692621:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589079808:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619912.692622:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cd4380 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589079936:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619912.695594:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaab40 going to sleep for 11 seconds 00000100:00100000:2.0:1788619912.695746:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164240 going to sleep for 11 seconds 00000100:00100000:3.0:1788619912.699731:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589079808:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619912.699790:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007736e663/0x1cd7fe5368734365 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b879 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619912.699824:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007736e663/0x1cd7fe5368734365 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b879 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619912.699852:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d400, rc 0) 00000100:00100000:3.0:1788619912.699945:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ce1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571383424:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7443us (7854us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619912.699955:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 634 00000100:00100000:0.0:1788619912.701478:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cd4380 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589079936:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619912.701516:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 962214 00010000:00010000:0.0:1788619912.701544:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e200/0x1cd7fe536873435e lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b872 expref: 11 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.701568:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619912.701571:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619912.701578:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e200/0x1cd7fe536873435e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b872 expref: 11 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.701595:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6e200, rc 0) 00000100:00100000:0.0:1788619912.701744:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79008ac000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571383296:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 9241us (9736us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619912.701776:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 824 00000100:00100000:2.0:1788619912.708182:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619912.708320:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571383552 00000100:00100000:0.0:1788619912.708432:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571383552 00000100:00100000:0.0:1788619912.708514:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 825 00000100:00100000:0.0:1788619912.708526:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571383552:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619912.708555:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619912.708583:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fe00/0x1cd7fe536873436c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b880 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.708634:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe536873432d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b848 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619912.708697:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cd4380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589080064:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619912.709156:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaab40 going to sleep for 11 seconds 00000100:00100000:1.0:1788619912.719472:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619912.719970:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571383680 00000100:00100000:0.0:1788619912.719984:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571383680 00000100:00100000:0.0:1788619912.720063:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 635 00000100:00100000:0.0:1788619912.720075:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ce0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571383680:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619912.720099:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619912.720122:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d058bad0/0x1cd7fe5368734373 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b887 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619912.720210:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cd7800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589080192:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619912.720311:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619912.731761:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cd4380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589080064:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619912.731826:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 962214 00010000:00010000:0.0:1788619912.731850:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fe00/0x1cd7fe536873436c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b880 expref: 11 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.731891:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619912.731895:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619912.731902:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fe00/0x1cd7fe536873436c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b880 expref: 11 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.731922:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6fe00, rc 0) 00000100:00100000:0.0:1788619912.732080:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571383552:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 23555us (23912us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619912.732091:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 825 00000100:00100000:0.0:1788619912.740172:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cd7800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589080192:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619912.740228:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d058bad0/0x1cd7fe5368734373 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b887 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.740260:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d058bad0/0x1cd7fe5368734373 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b887 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.740274:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6e800, rc 0) 00000100:00100000:0.0:1788619912.740401:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ce0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571383680:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 20327us (20944us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619912.740412:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 635 00000100:00100000:2.0:1788619912.745521:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619912.745852:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571383808 00000100:00100000:0.0:1788619912.745873:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571383808 00000100:00100000:0.0:1788619912.746451:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 826 00000100:00100000:0.0:1788619912.746466:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cdb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:14908:x1875503571383808:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619912.746493:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619912.746524:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e800/0x1cd7fe536873437a lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b88e expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.746568:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe536873432d lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b848 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.746580:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e800/0x1cd7fe536873437a lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b88e expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.746629:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe536873432d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b848 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.746646:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe536873432d lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b848 expref: 15 pid: 5846 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619912.746662:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cd7800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589080320:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619912.746770:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619912.753562:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619912.754005:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cd7800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589080320:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619912.754054:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e800/0x1cd7fe536873437a lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b88e expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.754079:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619912.754083:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619912.754105:0:5846:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e800/0x1cd7fe536873437a lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b88e expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.754116:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6e800, rc 0) 00000100:00100000:0.0:1788619912.754187:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cdb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571383808:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7727us (8677us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619912.754195:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 826 00000100:00100000:1.0:1788619912.754787:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571383936 00000100:00100000:1.0:1788619912.754797:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571383936 00010000:00010000:1.0:1788619912.754842:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe536873432d lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b848 expref: 16 pid: 5846 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619912.754993:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 281 00000100:00100000:1.0:1788619912.755015:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd8700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571383936:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619912.755033:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619912.755068:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe536873432d lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b848 expref: 16 pid: 5846 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619912.755082:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe536873432d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b848 expref: 16 pid: 5846 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619912.755108:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe536873432d lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b848 expref: 16 pid: 5846 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619912.755135:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619912.755141:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e800/0x1cd7fe536873437a lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b88e expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619912.755152:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e800/0x1cd7fe536873437a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b88e expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619912.755163:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e800/0x1cd7fe536873437a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b88e expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619912.755207:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e800/0x1cd7fe536873437a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b88e expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619912.755223:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c74f6300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589080448:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619912.755310:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963300 going to sleep for 11 seconds 00000100:00100000:2.0:1788619912.760832:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619912.760846:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c74f6300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589080448:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619912.760905:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619912.760995:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd8700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571383936:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 5980us (7442us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619912.761009:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 281 00000100:00100000:0.0:1788619912.761160:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571384064 00000100:00100000:0.0:1788619912.761170:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571384064 00002000:00100000:0.0:1788619912.761233:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5cd8000 x1875503571384064/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:8/0 lens 488/0 e 0 to 0 dl 1788619923 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619912.761246:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5cd8000 with x1875503571384064 00000100:00100000:0.0:1788619912.761288:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 465 00000100:00100000:0.0:1788619912.761300:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571384064:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619912.761610:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571384064:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 312us (793us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619912.761618:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 465 00002000:00100000:0.0:1788619912.761624:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5cd8000 x1875503571384064/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:8/0 lens 488/4536 e 0 to 0 dl 1788619923 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619912.761631:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5cd8000 with x1875503571384064 00000100:00100000:2.0:1788619912.768491:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619912.768562:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571384192 00000100:00100000:1.0:1788619912.768576:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571384192 00002000:00100000:1.0:1788619912.768639:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5cd8380 x1875503571384192/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:8/0 lens 488/0 e 0 to 0 dl 1788619923 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619912.768649:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5cd8380 with x1875503571384192 00000100:00100000:1.0:1788619912.768686:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 466 00000100:00100000:1.0:1788619912.768698:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571384192:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619912.768987:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571384192:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 290us (510us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619912.768995:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 466 00002000:00100000:1.0:1788619912.769003:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5cd8380 x1875503571384192/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:8/0 lens 488/4536 e 0 to 0 dl 1788619923 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619912.769012:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5cd8380 with x1875503571384192 00000100:00100000:2.0:1788619912.775279:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619912.778771:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571384320 00000100:00100000:1.0:1788619912.778796:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571384320 00000100:00100000:1.0:1788619912.778919:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 636 00000100:00100000:1.0:1788619912.778938:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571384320:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619912.778963:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619912.778987:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b15229b7/0x1cd7fe5368734381 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b895 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619912.779091:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c74f6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589080576:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619912.779225:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd9639c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619912.786267:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c74f6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589080576:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619912.786372:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b15229b7/0x1cd7fe5368734381 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b895 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619912.786409:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b15229b7/0x1cd7fe5368734381 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b895 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619912.786429:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f4a400, rc 0) 00000100:00100000:0.0:1788619912.786583:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571384320:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7644us (11316us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619912.786596:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 636 00000100:00100000:2.0:1788619912.792429:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619912.792564:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571384448 00000100:00100000:3.0:1788619912.792581:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571384448 00000100:00100000:3.0:1788619912.792651:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 637 00000100:00100000:3.0:1788619912.792663:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cdb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571384448:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619912.792694:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619912.792736:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007736e663/0x1cd7fe5368734388 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b89c expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619912.792822:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589080704:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619912.792951:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164240 going to sleep for 11 seconds 00000100:00100000:2.0:1788619912.798554:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589080704:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619912.798601:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007736e663/0x1cd7fe5368734388 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b89c expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.798628:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007736e663/0x1cd7fe5368734388 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b89c expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.798641:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d400, rc 0) 00000100:00100000:2.0:1788619912.798718:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cdb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571384448:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6056us (6312us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619912.798728:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 637 00000100:00100000:1.0:1788619912.806440:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619912.806528:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571384576 00000100:00100000:3.0:1788619912.806542:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571384576 00002000:00100000:3.0:1788619912.806613:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c74f6300 x1875503571384576/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:8/0 lens 504/0 e 0 to 0 dl 1788619923 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619912.806625:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c74f6300 with x1875503571384576 00000100:00100000:3.0:1788619912.806665:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 467 00000100:00100000:3.0:1788619912.806676:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571384576:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619912.822954:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571384576:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 16265us (16512us total) trans 0 rc 901120/901120 00000100:00100000:3.0:1788619912.822971:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 467 00002000:00100000:3.0:1788619912.822989:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c74f6300 x1875503571384576/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:8/0 lens 504/440 e 0 to 0 dl 1788619923 ref 1 fl Complete:/600/0 rc 901120/901120 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619912.823001:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c74f6300 with x1875503571384576 00000100:00100000:0.0:1788619912.829072:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619912.829215:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571384704 00000100:00100000:3.0:1788619912.829230:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571384704 00000100:00100000:3.0:1788619912.829310:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 638 00000100:00100000:3.0:1788619912.829323:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571384704:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619912.829389:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619912.829414:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000485aa315/0x1cd7fe536873438f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b8a3 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619912.829529:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fe300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589080832:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619912.829653:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164240 going to sleep for 11 seconds 00000100:00100000:3.0:1788619912.835086:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fe300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589080832:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619912.835142:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000485aa315/0x1cd7fe536873438f lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b8a3 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619912.835178:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000485aa315/0x1cd7fe536873438f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b8a3 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619912.835283:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683cc00, rc 0) 00000100:00100000:3.0:1788619912.835907:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571384704:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6582us (6848us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619912.835920:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 638 00000100:00100000:1.0:1788619912.843282:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619912.844107:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571384832 00000100:00100000:0.0:1788619912.844122:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571384832 00000100:00100000:0.0:1788619912.844219:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 639 00000100:00100000:0.0:1788619912.844230:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6859180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571384832:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619912.844252:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619912.844274:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ff7ffc7/0x1cd7fe5368734396 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b8aa expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619912.844737:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cd4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589080960:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619912.844933:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaab40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619912.850792:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cd4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589080960:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619912.850870:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003ff7ffc7/0x1cd7fe5368734396 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b8aa expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.850905:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ff7ffc7/0x1cd7fe5368734396 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b8aa expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.850919:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6e200, rc 0) 00000100:00100000:0.0:1788619912.851115:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6859180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571384832:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6886us (7848us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619912.851124:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 639 00000100:00100000:3.0:1788619912.857219:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619912.857395:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571384960 00000100:00100000:1.0:1788619912.857408:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571384960 00002000:00100000:1.0:1788619912.857479:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1269760->1294335) req@ffff8e79010fe300 x1875503571384960/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:8/0 lens 488/0 e 0 to 0 dl 1788619923 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619912.857491:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e79010fe300 with x1875503571384960 ext(1269760->1294335) 00002000:00010000:1.0:1788619912.857498:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e79010fe300 00000100:00100000:1.0:1788619912.857576:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 352 00000100:00100000:1.0:1788619912.857588:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010fe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571384960:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:2.0:1788619912.863552:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010fe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+10:2353:x1875503571384960:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 5947us (6329us total) trans 4294967396 rc 0/0 00000100:00100000:2.0:1788619912.863585:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 352 00002000:00100000:2.0:1788619912.863602:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1269760->1294335) req@ffff8e79010fe300 x1875503571384960/t4294967396(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:8/0 lens 488/448 e 0 to 0 dl 1788619923 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619912.863612:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e79010fe300 with x1875503571384960 ext(1269760->1294335) 00002000:00010000:2.0:1788619912.863620:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e79010fe300 00000100:00100000:1.0:1788619912.867110:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619912.867264:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571385088 00000100:00100000:0.0:1788619912.867280:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571385088 00000100:00100000:0.0:1788619912.867659:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 353 00000100:00100000:0.0:1788619912.867673:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+10:2355:x1875503571385088:12345-192.168.201.5@tcp:22:fsx.0 00002000:00010000:0.0:1788619912.868309:0:6703:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1624460 -> 1624460 00002000:00010000:0.0:1788619912.868316:0:6703:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619912 -> 1788619912 00002000:00010000:0.0:1788619912.868363:0:6703:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619911 -> 1788619911 00002000:00010000:0.0:1788619912.868366:0:6703:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619912 -> 1788619912 00002000:00010000:0.0:1788619912.868369:0:6703:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 135 -> 135 00000100:00100000:0.0:1788619912.868458:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+11:2355:x1875503571385088:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 787us (1363us total) trans 4294967397 rc 0/0 00000100:00100000:0.0:1788619912.868468:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 353 00000100:00100000:0.0:1788619912.872529:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619912.872642:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571385216 00000100:00100000:3.0:1788619912.872656:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571385216 00000100:00100000:3.0:1788619912.872865:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 640 00000100:00100000:3.0:1788619912.872879:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571385216:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619912.872905:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619912.872928:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000485aa315/0x1cd7fe536873439d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b8b1 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619912.873011:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589081088:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619912.876220:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164240 going to sleep for 11 seconds 00000100:00100000:2.0:1788619912.876313:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589081088:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619912.876381:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000485aa315/0x1cd7fe536873439d lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b8b1 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.876412:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000485aa315/0x1cd7fe536873439d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b8b1 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.876429:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683cc00, rc 0) 00000100:00100000:2.0:1788619912.876556:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571385216:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3679us (4044us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619912.876567:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 640 00000100:00100000:1.0:1788619912.880417:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619912.880527:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571385344 00000100:00100000:2.0:1788619912.880541:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571385344 00000100:00100000:2.0:1788619912.880612:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 641 00000100:00100000:2.0:1788619912.880624:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa14a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571385344:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619912.880648:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619912.880673:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002ac89266/0x1cd7fe53687343a4 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b8b8 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619912.880823:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d55c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589081216:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619912.880947:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccf00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619912.884976:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d55c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589081216:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619912.885029:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002ac89266/0x1cd7fe53687343a4 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b8b8 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.885064:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002ac89266/0x1cd7fe53687343a4 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b8b8 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.885081:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa06000, rc 0) 00000100:00100000:2.0:1788619912.885170:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa14a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571385344:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4547us (4767us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619912.885180:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 641 00000100:00100000:3.0:1788619912.892088:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619912.892990:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571385472 00000100:00100000:2.0:1788619912.893007:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571385472 00000100:00100000:2.0:1788619912.893146:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 642 00000100:00100000:2.0:1788619912.893160:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010ff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571385472:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619912.893187:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619912.893212:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002ac89266/0x1cd7fe53687343ab lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b8bf expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619912.893298:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d55c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589081344:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619912.893446:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccf00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619912.896374:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d55c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589081344:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619912.896426:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002ac89266/0x1cd7fe53687343ab lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b8bf expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.896457:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002ac89266/0x1cd7fe53687343ab lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b8bf expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.896475:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa06000, rc 0) 00000100:00100000:2.0:1788619912.896557:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010ff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571385472:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3400us (4483us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619912.896567:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 642 00000100:00100000:1.0:1788619912.901651:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619912.901785:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571385600 00000100:00100000:0.0:1788619912.901798:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571385600 00000100:00100000:0.0:1788619912.901878:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 827 00000100:00100000:0.0:1788619912.901891:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d5a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+15:14908:x1875503571385600:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619912.901921:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619912.901949:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f800/0x1cd7fe53687343b2 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b8c6 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.902071:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe53687343b9 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.902091:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe53687343b9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.902104:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe53687343b9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619912.902263:0:11397:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1624460 00000004:00200000:0.0:1788619912.902269:0:11397:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619912.902292:0:11397:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe53687343b9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.902323:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f800/0x1cd7fe53687343b2 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b8c6 expref: 17 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.902379:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe53687343b9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b8c6 expref: 16 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.902389:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619912.902392:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619912.902396:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6ee00, rc 0) 00000100:00100000:0.0:1788619912.902519:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d5a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+16:14908:x1875503571385600:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 631us (884us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619912.902527:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 827 00000100:00100000:3.0:1788619912.906012:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619912.906182:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571385728 00000100:00100000:0.0:1788619912.906196:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571385728 00000100:00100000:0.0:1788619912.906369:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 828 00000100:00100000:0.0:1788619912.906383:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010ff800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571385728:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619912.906410:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619912.906435:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe53687343c0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b8cd expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.906484:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e800/0x1cd7fe536873437a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b88e expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619912.906539:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5ff5880 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589081472:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619912.906642:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaab40 going to sleep for 11 seconds 00000004:00010000:0.0:1788619912.909935:0:9417:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619911 -> 1788619912 00000100:00100000:0.0:1788619912.909983:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5ff5880 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589081472:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619912.910018:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 962214 00010000:00010000:0.0:1788619912.910060:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe53687343c0 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b8cd expref: 17 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.910088:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619912.910091:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619912.910097:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe53687343c0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b8cd expref: 17 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.910116:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6ea00, rc 0) 00000100:00100000:0.0:1788619912.910260:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010ff800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571385728:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3879us (4263us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619912.910270:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 828 00000100:00100000:2.0:1788619912.915029:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619912.915127:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571385856 00000100:00100000:0.0:1788619912.915147:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571385856 00000100:00100000:0.0:1788619912.915248:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 829 00000100:00100000:0.0:1788619912.915263:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d55c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571385856:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619912.915303:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619912.915359:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f800/0x1cd7fe53687343c7 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b8d4 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.915412:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e800/0x1cd7fe536873437a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b88e expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619912.915468:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5ff5880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589081600:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619912.915597:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaab40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619912.920293:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5ff5880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589081600:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619912.920367:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 962214 00010000:00010000:0.0:1788619912.920412:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f800/0x1cd7fe53687343c7 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b8d4 expref: 17 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.920441:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619912.920444:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619912.920451:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f800/0x1cd7fe53687343c7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b8d4 expref: 17 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.920469:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6f800, rc 0) 00000100:00100000:0.0:1788619912.920548:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d55c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571385856:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5289us (5542us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619912.920558:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 829 00000100:00100000:2.0:1788619912.924142:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619912.924238:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571385984 00000100:00100000:0.0:1788619912.924251:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571385984 00000100:00100000:0.0:1788619912.924372:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 830 00000100:00100000:0.0:1788619912.924385:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d54e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571385984:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619912.924413:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619912.924438:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe53687343ce lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b8db expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.924489:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e800/0x1cd7fe536873437a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b88e expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619912.924542:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5ff5880 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589081728:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619912.924665:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaab40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619912.927911:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5ff5880 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589081728:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619912.927951:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 962214 00010000:00010000:0.0:1788619912.927999:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe53687343ce lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b8db expref: 17 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.928026:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619912.928029:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619912.928037:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe53687343ce lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b8db expref: 17 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.928056:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6ea00, rc 0) 00000100:00100000:0.0:1788619912.928179:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d54e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571385984:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3796us (4051us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619912.928189:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 830 00000100:00100000:3.0:1788619912.934609:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619912.934938:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571386112 00000100:00100000:0.0:1788619912.934952:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571386112 00000100:00100000:0.0:1788619912.935035:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 831 00000100:00100000:0.0:1788619912.935049:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010fce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571386112:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619912.935080:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619912.935108:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe53687343d5 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b8e2 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.935159:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e800/0x1cd7fe536873437a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b88e expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619912.935214:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5ff5880 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589081856:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619912.936616:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaab40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619912.938935:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5ff5880 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589081856:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619912.938974:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 962214 00010000:00010000:0.0:1788619912.939023:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe53687343d5 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b8e2 expref: 17 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.939050:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619912.939053:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619912.939060:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe53687343d5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b8e2 expref: 17 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.939079:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6ea00, rc 0) 00000100:00100000:0.0:1788619912.939166:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010fce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571386112:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4119us (4571us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619912.939177:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 831 00000100:00100000:2.0:1788619912.943971:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619912.944191:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571386368 00000100:00100000:0.0:1788619912.944205:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571386368 00000100:00100000:0.0:1788619912.944292:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 832 00000100:00100000:0.0:1788619912.944304:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d55500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:14908:x1875503571386368:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:0.0:1788619912.944430:0:5846:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619912.944465:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe53687343b9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b8c6 expref: 16 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.944487:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619912.944493:0:5846:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788619912.944923:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe53687343dc lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.944943:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe536873433b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b84f expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.944977:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe53687343dc lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.945022:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe536873433b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b84f expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.945040:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe536873433b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b84f expref: 10 pid: 5845 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619912.945058:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5ff5880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589081984:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619912.945166:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaab40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619912.947204:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5ff5880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589081984:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619912.947245:0:5846:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe53687343dc lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619912.948476:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619912.949144:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571386496 00000100:00100000:0.0:1788619912.949159:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571386496 00010000:00010000:0.0:1788619912.949218:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe536873433b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b84f expref: 11 pid: 5845 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619912.949271:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 282 00000100:00100000:0.0:1788619912.949293:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d57480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571386496:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619912.949313:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619912.949321:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe536873433b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b84f expref: 11 pid: 5845 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.949397:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe536873433b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b84f expref: 11 pid: 5845 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.949427:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe536873433b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b84f expref: 11 pid: 5845 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.949450:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619912.949456:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe53687343dc lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.949467:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe53687343dc lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.949477:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe53687343dc lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.949513:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619912.949578:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d57480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571386496:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 297us (1112us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619912.949586:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 282 00010000:00010000:0.0:1788619912.949618:0:5846:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe53687343dc lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.949627:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe53687343dc lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788619912.949800:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967617 00000004:00010000:0.0:1788619912.949875:0:5846:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 962214 -> 962214 00000004:00010000:0.0:1788619912.949879:0:5846:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619912 -> 1788619912 00000004:00010000:0.0:1788619912.949888:0:5846:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619912 -> 1788619911 00000004:00010000:0.0:1788619912.949890:0:5846:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619912 -> 1788619912 00000004:00010000:0.0:1788619912.949893:0:5846:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 646 -> 646 00000004:00010000:0.0:1788619912.949901:0:5846:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000f1121e49 reply state 00000000899da346 transno 4294967618 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe53687343dc lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619912.949911:0:5846:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788619912.950002:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d55500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+18:14908:x1875503571386368:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 5700us (6047us total) trans 4294967618 rc 0/0 00000100:00100000:0.0:1788619912.950011:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 832 00000020:00080000:0.0:1788619912.950015:0:5846:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1788619912.952661:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00010000:00010000:0.0:1788619912.952757:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe53687343dc lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.952770:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe53687343dc lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.952778:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe53687343dc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.952784:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c6b6ea00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788619912.952789:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe53687343dc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.952797:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe53687343dc lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.952812:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619912.952815:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788619912.952820:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe53687343dc lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.952844:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe53687343dc lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619912.953053:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571386624 00000100:00100000:2.0:1788619912.953069:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571386624 00000100:00100000:2.0:1788619912.953148:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 833 00000100:00100000:2.0:1788619912.953159:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a8ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+17:14908:x1875503571386624:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619912.953188:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619912.953213:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe53687343e3 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b8e9 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.953250:0:11397:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e800/0x1cd7fe536873437a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b88e expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.953295:0:11397:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe53687343e3 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b8e9 expref: 18 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.953391:0:11397:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e800/0x1cd7fe536873437a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b88e expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.953407:0:11397:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e800/0x1cd7fe536873437a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b88e expref: 9 pid: 5846 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619912.953424:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d55180 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589082112:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619912.953534:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccf00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619912.956441:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619912.956480:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d55180 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589082112:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619912.956524:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe53687343e3 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b8e9 expref: 18 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.956543:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619912.956545:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619912.956549:0:11397:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe53687343e3 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b8e9 expref: 18 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.956559:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa07a00, rc 0) 00000100:00100000:0.0:1788619912.956626:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a8ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+18:14908:x1875503571386624:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3468us (3973us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619912.956635:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 833 00000100:00100000:1.0:1788619912.957296:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571386752 00000100:00100000:1.0:1788619912.957309:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571386752 00002000:00100000:1.0:1788619912.957416:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5d57800 x1875503571386752/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:8/0 lens 488/0 e 0 to 0 dl 1788619923 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619912.957430:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5d57800 with x1875503571386752 00010000:00010000:1.0:1788619912.957440:0:5859:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e800/0x1cd7fe536873437a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b88e expref: 10 pid: 5846 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619912.957457:0:5859:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e800/0x1cd7fe536873437a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b88e expref: 10 pid: 5846 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788619912.957466:0:5859:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e77c5d57800 00000100:00100000:1.0:1788619912.957504:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 20 00000100:00100000:1.0:1788619912.957515:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d57800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571386752:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:0.0:1788619912.966809:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 962214 -> 962214 00000004:00010000:0.0:1788619912.966819:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619912 -> 1788619912 00000004:00010000:0.0:1788619912.966822:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619911 -> 1788619911 00000004:00010000:0.0:1788619912.966824:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619912 -> 1788619912 00000004:00010000:0.0:1788619912.966826:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 646 -> 646 00000100:00100000:0.0:1788619912.966919:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d57800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571386752:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 9401us (10489us total) trans 4294967619 rc 0/0 00000100:00100000:0.0:1788619912.966930:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 20 00002000:00100000:0.0:1788619912.966942:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5d57800 x1875503571386752/t4294967619(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:8/0 lens 488/448 e 0 to 0 dl 1788619923 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619912.966961:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5d57800 with x1875503571386752 00010000:00010000:0.0:1788619912.966970:0:5859:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e800/0x1cd7fe536873437a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b88e expref: 12 pid: 5846 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.966985:0:5859:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e800/0x1cd7fe536873437a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b88e expref: 12 pid: 5846 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788619912.966992:0:5859:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e77c5d57800 00000100:00100000:3.0:1788619912.970972:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619912.971075:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571386880 00000100:00100000:0.0:1788619912.971084:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571386880 00010000:00010000:0.0:1788619912.971130:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e800/0x1cd7fe536873437a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b88e expref: 12 pid: 5846 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619912.971173:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 283 00000100:00100000:0.0:1788619912.971182:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed62300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571386880:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619912.971200:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619912.971225:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e800/0x1cd7fe536873437a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b88e expref: 12 pid: 5846 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.971238:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e800/0x1cd7fe536873437a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b88e expref: 12 pid: 5846 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.971258:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e800/0x1cd7fe536873437a lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b88e expref: 12 pid: 5846 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.971274:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619912.971292:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe53687343e3 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b8e9 expref: 17 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.971303:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe53687343e3 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b8e9 expref: 17 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.971313:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe53687343e3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b8e9 expref: 17 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.971395:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe53687343e3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b8e9 expref: 17 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619912.971409:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5ff5880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589082240:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619912.971483:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaab40 going to sleep for 11 seconds 00000100:00100000:1.0:1788619912.974192:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619912.974275:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5ff5880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589082240:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619912.974298:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619912.974369:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed62300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571386880:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3188us (3407us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619912.974380:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 283 00000100:00100000:3.0:1788619912.974683:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571387008 00000100:00100000:3.0:1788619912.974690:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571387008 00002000:00010000:3.0:1788619912.974719:0:5857:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:3.0:1788619912.974723:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5d59f80 with x1875503571387008 00000100:00100000:3.0:1788619912.974750:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 468 00000100:00100000:3.0:1788619912.974757:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d59f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2354:x1875503571387008:12345-192.168.201.5@tcp:10:fsx.0 00000004:00010000:3.0:1788619912.975044:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 962214 -> 240354 00000004:00010000:3.0:1788619912.975048:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619912 -> 1788619912 00000004:00010000:3.0:1788619912.975051:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619911 -> 1788619911 00000004:00010000:3.0:1788619912.975053:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619912 -> 1788619912 00000004:00010000:3.0:1788619912.975055:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 646 -> 646 00000100:00100000:3.0:1788619912.975102:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d59f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2354:x1875503571387008:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 345us (918us total) trans 4294967620 rc 0/0 00000100:00100000:3.0:1788619912.975109:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 468 00002000:00010000:3.0:1788619912.975112:0:5857:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:3.0:1788619912.975115:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5d59f80 with x1875503571387008 00000100:00100000:1.0:1788619912.976720:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619912.976791:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571387136 00000100:00100000:2.0:1788619912.976799:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571387136 00002000:00010000:2.0:1788619912.976832:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:2.0:1788619912.976837:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5d5b100 with x1875503571387136 ext(0->18446744073709551615) 00002000:00010000:2.0:1788619912.976847:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5d5b100. 00000100:00100000:2.0:1788619912.976884:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 354 00000100:00100000:2.0:1788619912.976893:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d5b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+11:2354:x1875503571387136:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:2.0:1788619912.977160:0:6702:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1624460 -> 0 00002000:00010000:2.0:1788619912.977164:0:6702:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619912 -> 1788619912 00002000:00010000:2.0:1788619912.977166:0:6702:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619911 -> 1788619911 00002000:00010000:2.0:1788619912.977168:0:6702:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619912 -> 1788619912 00002000:00010000:2.0:1788619912.977170:0:6702:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 135 -> 135 00000100:00100000:2.0:1788619912.977224:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d5b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571387136:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 332us (509us total) trans 4294967398 rc 0/0 00000100:00100000:2.0:1788619912.977243:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 354 00002000:00010000:2.0:1788619912.977247:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:2.0:1788619912.977249:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5d5b100 with x1875503571387136 ext(0->18446744073709551615) 00002000:00010000:2.0:1788619912.977254:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5d5b100. 00000100:00100000:2.0:1788619912.980734:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619912.980838:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571387264 00000100:00100000:0.0:1788619912.980848:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571387264 00000100:00100000:0.0:1788619912.980927:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 834 00000100:00100000:0.0:1788619912.980936:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d54a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571387264:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619912.980959:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619912.980979:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe53687343ea lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b8f0 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.981084:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe53687343f1 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.981100:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe53687343f1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.981112:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe53687343f1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619912.981150:0:9417:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 240354 00000004:00200000:0.0:1788619912.981154:0:9417:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619912.981171:0:9417:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe53687343f1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.981195:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe53687343ea lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b8f0 expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.981213:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe53687343f1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b8f0 expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619912.981222:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619912.981225:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619912.981229:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6ee00, rc 0) 00000100:00100000:0.0:1788619912.981375:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d54a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+12:14908:x1875503571387264:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 440us (650us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619912.981382:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 834 00000100:00100000:0.0:1788619912.984376:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619912.984423:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619912.984528:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571387392 00000100:00100000:2.0:1788619912.984538:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571387392 00000100:00100000:2.0:1788619912.984593:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 835 00000100:00100000:2.0:1788619912.984603:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571387392:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619912.984623:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619912.984643:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe53687343f8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b8f7 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.984691:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe53687343e3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b8e9 expref: 18 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619912.984729:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d56d80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589082368:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619912.984798:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619912.987013:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d56d80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589082368:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619912.987040:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 240354 00010000:00010000:2.0:1788619912.987071:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe53687343f8 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b8f7 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.987093:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619912.987097:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619912.987116:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe53687343f8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b8f7 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.987140:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa07e00, rc 0) 00000100:00100000:2.0:1788619912.987190:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571387392:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2587us (2824us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619912.987198:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 835 00000100:00100000:1.0:1788619912.990451:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571387520 00000100:00100000:1.0:1788619912.990465:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571387520 00000100:00100000:1.0:1788619912.990573:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 643 00000100:00100000:1.0:1788619912.990584:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cd3c9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571387520:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619912.990602:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619912.990621:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d124dcf7/0x1cd7fe53687343ff lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b8fe expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619912.990692:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d58700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589082496:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619912.990781:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd9639c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619912.993358:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d58700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589082496:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619912.993410:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d124dcf7/0x1cd7fe53687343ff lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b8fe expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619912.993435:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d124dcf7/0x1cd7fe53687343ff lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b8fe expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619912.993448:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f4a200, rc 0) 00000100:00100000:1.0:1788619912.993502:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cd3c9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571387520:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2920us (9078us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619912.993510:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 643 00000100:00100000:0.0:1788619912.996114:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619912.996166:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619912.996281:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571387776 00000100:00100000:3.0:1788619912.996289:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571387776 00000100:00100000:3.0:1788619912.996385:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 644 00000100:00100000:3.0:1788619912.996395:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571387776:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619912.996412:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619912.996440:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001fc792ba/0x1cd7fe5368734406 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b90c expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619912.996522:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d5d180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589082624:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619912.996594:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164600 going to sleep for 11 seconds 00000100:00100000:2.0:1788619912.997011:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571387648 00000100:00100000:2.0:1788619912.997016:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571387648 00000100:00100000:2.0:1788619912.997046:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 836 00000100:00100000:2.0:1788619912.997055:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c0b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571387648:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619912.997065:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619912.997076:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe536873440d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b905 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619912.997105:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe53687343e3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b8e9 expref: 18 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619912.997142:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d56d80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589082752:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619912.997182:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc9c0 going to sleep for 11 seconds 00010000:00010000:3.0:1788619912.997683:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788619913.003095:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d5d180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589082624:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619913.003189:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001fc792ba/0x1cd7fe5368734406 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b90c expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.003214:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001fc792ba/0x1cd7fe5368734406 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b90c expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.003227:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d800, rc 0) 00000100:00100000:3.0:1788619913.003287:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571387776:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6893us (7120us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619913.003296:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 644 00000100:00100000:2.0:1788619913.004658:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d56d80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589082752:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619913.004769:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 240354 00010000:00010000:2.0:1788619913.004788:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe536873440d lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b905 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.004805:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619913.004808:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619913.004830:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe536873440d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b905 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.004842:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa06400, rc 0) 00000100:00100000:2.0:1788619913.004981:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c0b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571387648:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7926us (8875us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619913.004989:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 836 00000100:00100000:3.0:1788619913.008504:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.008666:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571387904 00000100:00100000:1.0:1788619913.008683:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571387904 00000100:00100000:1.0:1788619913.008766:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 837 00000100:00100000:1.0:1788619913.008776:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d5d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:14908:x1875503571387904:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619913.008795:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619913.008813:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a200/0x1cd7fe5368734414 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b913 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.008839:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe53687343e3 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b8e9 expref: 18 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.008849:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a200/0x1cd7fe5368734414 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b913 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.008892:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe53687343e3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b8e9 expref: 18 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.008933:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe53687343e3 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b8e9 expref: 18 pid: 11397 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619913.008946:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d58700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589082880:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619913.009021:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd9639c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619913.012165:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619913.012239:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571388032 00000100:00100000:3.0:1788619913.012247:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571388032 00010000:00010000:3.0:1788619913.012284:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe53687343e3 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b8e9 expref: 19 pid: 11397 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619913.012355:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 284 00000100:00100000:3.0:1788619913.012366:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2354:x1875503571388032:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619913.012379:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619913.012418:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe53687343e3 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b8e9 expref: 19 pid: 11397 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.012432:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe53687343e3 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b8e9 expref: 19 pid: 11397 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.012454:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619913.012461:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a200/0x1cd7fe5368734414 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b913 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.012488:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a200/0x1cd7fe5368734414 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b913 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.012497:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a200/0x1cd7fe5368734414 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b913 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.012535:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a200/0x1cd7fe5368734414 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b913 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619913.012549:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d5f800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589083008:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619913.012627:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164600 going to sleep for 11 seconds 00000100:00100000:3.0:1788619913.015225:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d5f800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589083008:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619913.015251:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619913.015301:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2354:x1875503571388032:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 2936us (3144us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619913.015311:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 284 00010000:00010000:1.0:1788619913.015487:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe53687343e3 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b8e9 expref: 18 pid: 11397 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619913.015505:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d58700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589082880:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619913.015526:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a200/0x1cd7fe5368734414 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b913 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.015542:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619913.015545:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619913.015548:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f4a200, rc 0) 00000100:00100000:1.0:1788619913.015575:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d5d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+13:14908:x1875503571387904:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6801us (7081us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619913.015581:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 837 00000100:00100000:0.0:1788619913.017982:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.018178:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571388160 00000100:00100000:1.0:1788619913.018188:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571388160 00000100:00100000:1.0:1788619913.018293:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 469 00000100:00100000:1.0:1788619913.018303:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6531c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571388160:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:1.0:1788619913.018666:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 240354 -> 240354 00000004:00010000:1.0:1788619913.018670:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619912 -> 1788619912 00000004:00010000:1.0:1788619913.018746:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619911 -> 1788619911 00000004:00010000:1.0:1788619913.018748:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619912 -> 1788619912 00000004:00010000:1.0:1788619913.018751:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 646 -> 646 00000100:00100000:1.0:1788619913.018870:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6531c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+14:2354:x1875503571388160:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 567us (894us total) trans 4294967621 rc 0/0 00000100:00100000:1.0:1788619913.018877:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 469 00000100:00100000:0.0:1788619913.021499:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619913.021597:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571388288 00000100:00100000:2.0:1788619913.021607:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571388288 00000100:00100000:2.0:1788619913.021697:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 838 00000100:00100000:2.0:1788619913.021708:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6340380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+18:14908:x1875503571388288:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619913.021731:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619913.021752:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe536873441b lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b91a expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.021835:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe5368734422 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.021861:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe5368734422 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.021874:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe5368734422 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788619913.021901:0:5846:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 240354 00000004:00200000:2.0:1788619913.021905:0:5846:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788619913.021922:0:5846:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe5368734422 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.021947:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe536873441b lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b91a expref: 20 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.021972:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe5368734422 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b91a expref: 19 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.021981:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619913.021983:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788619913.021987:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa07e00, rc 0) 00000100:00100000:2.0:1788619913.022046:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6340380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+19:14908:x1875503571388288:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 340us (555us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619913.022053:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 838 00000100:00100000:3.0:1788619913.024233:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619913.024989:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571388416 00000100:00100000:0.0:1788619913.024998:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571388416 00000100:00100000:0.0:1788619913.025050:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 839 00000100:00100000:0.0:1788619913.025060:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d5f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2354:x1875503571388416:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619913.025077:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619913.025095:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e800/0x1cd7fe5368734429 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b921 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.025230:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a200/0x1cd7fe5368734414 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b913 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619913.025271:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6341880 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589083136:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619913.025456:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaab40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619913.027774:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6341880 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589083136:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619913.027801:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 240354 00010000:00010000:0.0:1788619913.027819:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e800/0x1cd7fe5368734429 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b921 expref: 20 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.027838:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619913.027841:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619913.027847:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e800/0x1cd7fe5368734429 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b921 expref: 20 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.027889:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6e800, rc 0) 00000100:00100000:0.0:1788619913.027981:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d5f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2354:x1875503571388416:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2922us (3754us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619913.027989:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 839 00000100:00100000:0.0:1788619913.031051:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.031189:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571388544 00000100:00100000:1.0:1788619913.031199:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571388544 00000100:00100000:1.0:1788619913.031287:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 840 00000100:00100000:1.0:1788619913.031298:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6341880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2355:x1875503571388544:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619913.031318:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619913.031365:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4aa00/0x1cd7fe5368734430 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b928 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.031401:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a200/0x1cd7fe5368734414 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b913 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619913.031439:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d58700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589083264:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619913.031514:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd9639c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619913.033742:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d58700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589083264:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619913.033766:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 240354 00010000:00010000:1.0:1788619913.033784:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4aa00/0x1cd7fe5368734430 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b928 expref: 20 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.033804:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619913.033806:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619913.033826:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4aa00/0x1cd7fe5368734430 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b928 expref: 20 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.033841:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f4aa00, rc 0) 00000100:00100000:1.0:1788619913.034006:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6341880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2355:x1875503571388544:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2709us (2963us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619913.034015:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 840 00000100:00100000:0.0:1788619913.037889:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619913.038092:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571388800 00000100:00100000:2.0:1788619913.038102:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571388800 00000100:00100000:2.0:1788619913.038254:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 841 00000100:00100000:2.0:1788619913.038265:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6342a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+14:14908:x1875503571388800:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:2.0:1788619913.038356:0:11397:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619913.038384:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe53687343f1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b8f0 expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.038402:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619913.038407:0:11397:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788619913.038595:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe5368734437 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.038610:0:11397:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe5368734422 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b91a expref: 18 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.038619:0:11397:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe5368734437 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.038665:0:11397:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe5368734422 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b91a expref: 18 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.038788:0:11397:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe5368734422 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b91a expref: 18 pid: 5846 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619913.038804:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d56d80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589083392:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619913.038895:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619913.041699:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d56d80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589083392:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619913.041729:0:11397:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe5368734437 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619913.042177:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.042768:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571388928 00000100:00100000:1.0:1788619913.042776:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571388928 00010000:00010000:1.0:1788619913.042813:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe5368734422 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b91a expref: 19 pid: 5846 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619913.043082:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 285 00000100:00100000:1.0:1788619913.043092:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8020050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2354:x1875503571388928:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619913.043108:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619913.043115:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe5368734422 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b91a expref: 19 pid: 5846 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.043136:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe5368734422 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b91a expref: 19 pid: 5846 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.043157:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe5368734422 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b91a expref: 19 pid: 5846 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.043173:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619913.043177:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe5368734437 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.043204:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe5368734437 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.043212:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe5368734437 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.043239:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619913.043295:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8020050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2354:x1875503571388928:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 203us (1118us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619913.043303:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 285 00010000:00010000:2.0:1788619913.043556:0:11397:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe5368734437 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.043563:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe5368734437 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788619913.043700:0:11397:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967622 00000004:00010000:2.0:1788619913.043747:0:11397:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 240354 -> 240354 00000004:00010000:2.0:1788619913.043751:0:11397:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619912 -> 1788619912 00000004:00010000:2.0:1788619913.043753:0:11397:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619911 -> 1788619911 00000004:00010000:2.0:1788619913.043755:0:11397:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619912 -> 1788619912 00000004:00010000:2.0:1788619913.043758:0:11397:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 646 -> 646 00000004:00010000:2.0:1788619913.043764:0:11397:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000a745c603 reply state 00000000222a7897 transno 4294967623 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe5368734437 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619913.043783:0:11397:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788619913.043867:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6342a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+16:14908:x1875503571388800:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 5604us (5988us total) trans 4294967623 rc 0/0 00000100:00100000:2.0:1788619913.043875:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 841 00000020:00080000:2.0:1788619913.043878:0:11397:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1788619913.046988:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619913.047056:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571389056 00000100:00100000:0.0:1788619913.047065:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571389056 00000100:00100000:0.0:1788619913.047112:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 645 00000100:00100000:0.0:1788619913.047127:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d56d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:14908:x1875503571389056:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619913.047143:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619913.047160:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000daa9fe76/0x1cd7fe536873443e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b92f expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.047189:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000244b3d37/0x1cd7fe5368734311 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1269760->1294335) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b833 expref: 11 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.047197:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(4 8) 00010000:00010000:0.0:1788619913.047202:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(5 8) 00010000:00010000:0.0:1788619913.047207:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000daa9fe76/0x1cd7fe536873443e lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b92f expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.047240:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000244b3d37/0x1cd7fe5368734311 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1269760->1294335) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b833 expref: 11 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.047273:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000244b3d37/0x1cd7fe5368734311 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1269760->1294335) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b833 expref: 11 pid: 6698 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619913.047287:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6343b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589083520:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619913.047385:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaab40 going to sleep for 11 seconds 00010000:00010000:3.0:1788619913.049522:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe5368734437 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.049537:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe5368734437 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.049544:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe5368734437 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.049550:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78dfa06400 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:3.0:1788619913.049555:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe5368734437 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.049563:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe5368734437 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.049579:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619913.049582:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788619913.049587:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe5368734437 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.049594:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe5368734437 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619913.049883:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6343b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589083520:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619913.049918:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000daa9fe76/0x1cd7fe536873443e lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b92f expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.049934:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6f600, rc 0) 00000100:00100000:1.0:1788619913.049982:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d56d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:14908:x1875503571389056:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2861us (3003us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619913.049990:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 645 00000100:00100000:1.0:1788619913.051798:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619913.051873:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571389184 00000100:00100000:2.0:1788619913.051880:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571389184 00010000:00010000:2.0:1788619913.051915:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000244b3d37/0x1cd7fe5368734311 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1269760->1294335) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b833 expref: 12 pid: 6698 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619913.051955:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 286 00000100:00100000:2.0:1788619913.051964:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d58700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2355:x1875503571389184:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619913.051978:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619913.052000:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000244b3d37/0x1cd7fe5368734311 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1269760->1294335) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b833 expref: 12 pid: 6698 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.052014:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000244b3d37/0x1cd7fe5368734311 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1269760->1294335) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b833 expref: 12 pid: 6698 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.052034:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000244b3d37/0x1cd7fe5368734311 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1269760->1294335) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b833 expref: 12 pid: 6698 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.052048:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(5 8) 00010000:00010000:2.0:1788619913.052052:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(5 8) 00010000:00010000:2.0:1788619913.052075:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000daa9fe76/0x1cd7fe536873443e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b92f expref: 5 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.052084:0:5829:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000daa9fe76/0x1cd7fe536873443e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b92f expref: 5 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.052133:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000daa9fe76/0x1cd7fe536873443e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b92f expref: 5 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619913.052147:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d56300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1875503589083648:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619913.052355:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619913.055262:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619913.055384:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d56300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1875503589083648:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619913.055408:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619913.055459:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d58700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+11:2355:x1875503571389184:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3496us (3666us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619913.055470:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 286 00000100:00100000:0.0:1788619913.055653:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571389312 00000100:00100000:0.0:1788619913.055660:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571389312 00002000:00010000:0.0:1788619913.055703:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (2073839->18446744073709551615) 00002000:00100000:0.0:1788619913.055708:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5d5ca80 with x1875503571389312 ext(2073839->18446744073709551615) 00002000:00010000:0.0:1788619913.055717:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5d5ca80. 00000100:00100000:0.0:1788619913.055744:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 355 00000100:00100000:0.0:1788619913.055753:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d5ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571389312:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:0.0:1788619913.055976:0:6703:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 0 -> 2073839 00002000:00010000:0.0:1788619913.055980:0:6703:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619912 -> 1788619912 00002000:00010000:0.0:1788619913.055983:0:6703:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619911 -> 1788619911 00002000:00010000:0.0:1788619913.055984:0:6703:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619912 -> 1788619912 00002000:00010000:0.0:1788619913.055987:0:6703:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 135 -> 135 00000100:00100000:0.0:1788619913.056011:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d5ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571389312:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 259us (756us total) trans 4294967399 rc 0/0 00000100:00100000:0.0:1788619913.056018:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 355 00002000:00010000:0.0:1788619913.056021:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (2073839->18446744073709551615) 00002000:00100000:0.0:1788619913.056023:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5d5ca80 with x1875503571389312 ext(2073839->18446744073709551615) 00002000:00010000:0.0:1788619913.056027:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5d5ca80. 00000100:00100000:0.0:1788619913.059529:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619913.059603:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571389440 00000100:00100000:2.0:1788619913.059612:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571389440 00000100:00100000:2.0:1788619913.059669:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 842 00000100:00100000:2.0:1788619913.059689:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6342680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+18:14908:x1875503571389440:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619913.059709:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619913.059727:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06600/0x1cd7fe5368734445 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b936 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.059809:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06000/0x1cd7fe536873444c lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.059841:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06000/0x1cd7fe536873444c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.059862:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06000/0x1cd7fe536873444c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788619913.059890:0:9417:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 2073839 00000004:00200000:2.0:1788619913.059894:0:9417:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788619913.059912:0:9417:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06000/0x1cd7fe536873444c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.059936:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06600/0x1cd7fe5368734445 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b936 expref: 20 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.059952:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06000/0x1cd7fe536873444c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b936 expref: 19 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.059960:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619913.059963:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788619913.059966:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa06000, rc 0) 00000100:00100000:2.0:1788619913.060029:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6342680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+19:14908:x1875503571389440:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 351us (509us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619913.060037:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 842 00000100:00100000:3.0:1788619913.065284:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.065421:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571389696 00000100:00100000:1.0:1788619913.065430:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571389696 00000100:00100000:1.0:1788619913.065495:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 646 00000100:00100000:1.0:1788619913.065505:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d5ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+11:2354:x1875503571389696:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619913.065523:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619913.065541:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b15229b7/0x1cd7fe5368734453 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b944 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619913.065617:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d58000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589083776:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619913.065703:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963300 going to sleep for 11 seconds 00000100:00100000:3.0:1788619913.066113:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.066238:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571389568 00000100:00100000:1.0:1788619913.066243:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571389568 00000100:00100000:1.0:1788619913.066284:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 843 00000100:00100000:1.0:1788619913.066291:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d5e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2354:x1875503571389568:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619913.066304:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619913.066318:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4b200/0x1cd7fe536873445a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b93d expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.066465:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a200/0x1cd7fe5368734414 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b913 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619913.066495:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d58380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589083904:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619913.066548:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963d80 going to sleep for 11 seconds 00000100:00100000:0.0:1788619913.069520:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d58380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589083904:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619913.069554:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 240354 00010000:00010000:0.0:1788619913.069573:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4b200/0x1cd7fe536873445a lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b93d expref: 20 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.069590:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619913.069592:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619913.069596:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4b200/0x1cd7fe536873445a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b93d expref: 20 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.069607:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f4b200, rc 0) 00000100:00100000:0.0:1788619913.069667:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d5e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2354:x1875503571389568:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3376us (3554us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619913.069682:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 843 00000100:00100000:1.0:1788619913.070726:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d58000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589083776:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619913.070749:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b15229b7/0x1cd7fe5368734453 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b944 expref: 12 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.070763:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b15229b7/0x1cd7fe5368734453 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b944 expref: 12 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.070772:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f4a400, rc 0) 00000100:00100000:1.0:1788619913.070809:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d5ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+11:2354:x1875503571389696:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5306us (5536us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619913.070826:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 646 00000100:00100000:2.0:1788619913.072776:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619913.072810:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619913.072819:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571389824 00000100:00100000:0.0:1788619913.072827:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571389824 00000100:00100000:3.0:1788619913.072892:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571389952 00000100:00100000:3.0:1788619913.072897:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571389952 00000100:00100000:3.0:1788619913.072936:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 647 00000100:00100000:0.0:1788619913.072936:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 844 00000100:00100000:3.0:1788619913.072946:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d55f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+11:2355:x1875503571389952:12345-192.168.201.5@tcp:101:fsx.0 00000100:00100000:0.0:1788619913.072946:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d56300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2355:x1875503571389824:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619913.072966:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619913.072966:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619913.072980:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001fc792ba/0x1cd7fe5368734461 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b952 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.072981:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fa00/0x1cd7fe5368734468 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b94b expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.073023:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a200/0x1cd7fe5368734414 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b913 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619913.073034:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d5df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589084032:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619913.073044:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6341c00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589084160:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619913.073151:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164780 going to sleep for 11 seconds 00000100:00100000:0.0:1788619913.073933:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa180 going to sleep for 11 seconds 00000100:00100000:3.0:1788619913.077036:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d5df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589084032:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619913.077075:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001fc792ba/0x1cd7fe5368734461 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b952 expref: 12 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.077100:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001fc792ba/0x1cd7fe5368734461 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b952 expref: 12 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.077114:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d800, rc 0) 00000100:00100000:3.0:1788619913.077203:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d55f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+11:2355:x1875503571389952:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4258us (4390us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619913.077214:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 647 00000100:00100000:0.0:1788619913.077848:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6341c00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589084160:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619913.077881:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 240354 00010000:00010000:0.0:1788619913.077895:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fa00/0x1cd7fe5368734468 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b94b expref: 20 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.077909:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619913.077913:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619913.077918:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fa00/0x1cd7fe5368734468 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b94b expref: 20 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.077930:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6fa00, rc 0) 00000100:00100000:0.0:1788619913.077969:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d56300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2355:x1875503571389824:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5025us (5199us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619913.077977:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 844 00000100:00100000:2.0:1788619913.081494:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619913.081538:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619913.081541:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571390208 00000100:00100000:0.0:1788619913.081551:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571390208 00000100:00100000:3.0:1788619913.081602:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571390080 00000100:00100000:0.0:1788619913.081606:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 648 00000100:00100000:3.0:1788619913.081607:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571390080 00000100:00100000:0.0:1788619913.081626:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d57100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+11:2354:x1875503571390208:12345-192.168.201.5@tcp:101:fsx.0 00000100:00100000:3.0:1788619913.081642:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 845 00010000:00010000:0.0:1788619913.081643:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1788619913.081649:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d54700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2354:x1875503571390080:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619913.081657:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619913.081659:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b431d3d/0x1cd7fe536873446f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b960 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.081668:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368734476 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b959 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619913.081718:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6341c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589084288:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619913.081721:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a200/0x1cd7fe5368734414 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b913 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619913.081741:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d5df80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589084416:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619913.081794:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa180 going to sleep for 11 seconds 00000100:00100000:3.0:1788619913.084480:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6341c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589084288:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619913.084517:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002b431d3d/0x1cd7fe536873446f lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b960 expref: 12 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.084540:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b431d3d/0x1cd7fe536873446f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b960 expref: 12 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.084551:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6e400, rc 0) 00000100:00100000:3.0:1788619913.084593:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d57100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+11:2354:x1875503571390208:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2979us (3110us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619913.084601:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 648 00000100:00100000:0.0:1788619913.085164:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164780 going to sleep for 11 seconds 00000100:00100000:0.0:1788619913.088870:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d5df80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589084416:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619913.088921:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 240354 00010000:00010000:0.0:1788619913.088940:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368734476 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b959 expref: 20 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.088962:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619913.088965:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619913.088971:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368734476 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b959 expref: 20 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.088984:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c600, rc 0) 00000100:00100000:0.0:1788619913.089040:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d54700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2354:x1875503571390080:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7391us (7499us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619913.089049:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 845 00000100:00100000:1.0:1788619913.093024:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.093080:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619913.093219:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571390464 00000100:00100000:2.0:1788619913.093240:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571390464 00000100:00100000:2.0:1788619913.093298:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 649 00000100:00100000:2.0:1788619913.093308:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d59880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+11:2353:x1875503571390464:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619913.093356:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619913.093374:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cb4402d/0x1cd7fe536873447d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b96e expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619913.093433:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d54000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589084544:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619913.093509:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619913.093625:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571390336 00000100:00100000:0.0:1788619913.093629:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571390336 00000100:00100000:0.0:1788619913.093658:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 846 00000100:00100000:0.0:1788619913.093665:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d58000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2353:x1875503571390336:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619913.093685:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619913.093699:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fa00/0x1cd7fe5368734484 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b967 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.093727:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a200/0x1cd7fe5368734414 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b913 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619913.093752:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6340e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589084672:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619913.093782:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa180 going to sleep for 11 seconds 00000100:00100000:0.0:1788619913.096212:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6340e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589084672:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619913.096248:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 240354 00010000:00010000:0.0:1788619913.096265:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fa00/0x1cd7fe5368734484 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b967 expref: 20 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.096284:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619913.096287:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619913.096292:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fa00/0x1cd7fe5368734484 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b967 expref: 20 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.096306:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6fa00, rc 0) 00000100:00100000:0.0:1788619913.096386:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d58000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2353:x1875503571390336:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2720us (3371us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619913.096404:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 846 00000100:00100000:2.0:1788619913.100012:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d54000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589084544:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619913.100049:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005cb4402d/0x1cd7fe536873447d lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b96e expref: 12 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.100519:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cb4402d/0x1cd7fe536873447d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b96e expref: 12 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.100535:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa06200, rc 0) 00000100:00100000:2.0:1788619913.100628:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d59880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+11:2353:x1875503571390464:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7321us (7546us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619913.100637:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 649 00000100:00100000:2.0:1788619913.104449:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619913.104478:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619913.104543:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571390592 00000100:00100000:0.0:1788619913.104552:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571390592 00000100:00100000:0.0:1788619913.104603:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 847 00000100:00100000:0.0:1788619913.104613:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d54000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2355:x1875503571390592:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619913.104635:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619913.104657:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fc00/0x1cd7fe536873448b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b975 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.104702:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a200/0x1cd7fe5368734414 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b913 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619913.104737:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6340e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589084800:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619913.104812:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa180 going to sleep for 11 seconds 00000100:00100000:1.0:1788619913.105282:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571390720 00000100:00100000:1.0:1788619913.105287:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571390720 00000100:00100000:1.0:1788619913.105320:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 650 00000100:00100000:1.0:1788619913.105356:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d54380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+11:2355:x1875503571390720:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619913.105366:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619913.105376:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000afbf5f1/0x1cd7fe5368734492 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b97c expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619913.105416:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d5aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589084928:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619913.105452:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963300 going to sleep for 11 seconds 00000100:00100000:1.0:1788619913.109757:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d5aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589084928:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619913.109795:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000afbf5f1/0x1cd7fe5368734492 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b97c expref: 12 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.109818:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000afbf5f1/0x1cd7fe5368734492 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b97c expref: 12 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.109831:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f4ae00, rc 0) 00000100:00100000:1.0:1788619913.109938:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d54380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+11:2355:x1875503571390720:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4610us (5458us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619913.109946:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 650 00000100:00100000:2.0:1788619913.112067:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6340e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589084800:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619913.112093:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 240354 00010000:00010000:2.0:1788619913.112109:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fc00/0x1cd7fe536873448b lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b975 expref: 20 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.112152:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619913.112155:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619913.112161:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fc00/0x1cd7fe536873448b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b975 expref: 20 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.112175:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6fc00, rc 0) 00000100:00100000:2.0:1788619913.112229:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d54000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2355:x1875503571390592:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7616us (7787us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619913.112237:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 847 00000100:00100000:2.0:1788619913.116071:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619913.116309:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571390848 00000100:00100000:3.0:1788619913.116319:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571390848 00000100:00100000:3.0:1788619913.116400:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 848 00000100:00100000:3.0:1788619913.116410:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d55880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2355:x1875503571390848:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619913.116428:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619913.116446:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe5368734499 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b983 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.116482:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a200/0x1cd7fe5368734414 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b913 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619913.116536:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d5f100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589085056:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619913.116605:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164600 going to sleep for 11 seconds 00000100:00100000:3.0:1788619913.118796:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d5f100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589085056:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619913.118821:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 240354 00010000:00010000:3.0:1788619913.118837:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe5368734499 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b983 expref: 20 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.118866:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619913.118869:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619913.118875:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe5368734499 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b983 expref: 20 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.118889:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d800, rc 0) 00000100:00100000:3.0:1788619913.118949:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d55880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2355:x1875503571390848:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2540us (2885us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619913.118958:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 848 00000100:00100000:0.0:1788619913.120320:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619913.120507:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571390976 00000100:00100000:3.0:1788619913.120514:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571390976 00000100:00100000:3.0:1788619913.120563:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 651 00000100:00100000:3.0:1788619913.120573:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6341500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+11:2355:x1875503571390976:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619913.120588:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619913.120604:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a72c5200/0x1cd7fe53687344a0 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b98a expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619913.120660:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d5f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589085184:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619913.120746:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164600 going to sleep for 11 seconds 00000100:00100000:3.0:1788619913.123419:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d5f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589085184:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619913.123455:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a72c5200/0x1cd7fe53687344a0 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b98a expref: 12 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.123479:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a72c5200/0x1cd7fe53687344a0 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b98a expref: 12 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.123491:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c400, rc 0) 00000100:00100000:3.0:1788619913.123536:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6341500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+11:2355:x1875503571390976:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2965us (3221us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619913.123544:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 651 00000100:00100000:1.0:1788619913.126531:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619913.126627:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571391104 00000100:00100000:0.0:1788619913.126636:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571391104 00000100:00100000:0.0:1788619913.126694:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 849 00000100:00100000:0.0:1788619913.126703:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d5aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+15:14908:x1875503571391104:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619913.126727:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619913.126746:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f400/0x1cd7fe53687344a7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b991 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.126827:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe53687344ae lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.126842:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe53687344ae lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.126865:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe53687344ae lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619913.126893:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 2073839 00000004:00200000:0.0:1788619913.126897:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619913.126926:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe53687344ae lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.126952:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f400/0x1cd7fe53687344a7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b991 expref: 17 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.126969:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe53687344ae lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b991 expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.126977:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619913.126980:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619913.126998:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6e000, rc 0) 00000100:00100000:0.0:1788619913.127061:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d5aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+16:14908:x1875503571391104:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 360us (539us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619913.127069:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 849 00000100:00100000:0.0:1788619913.135096:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619913.135148:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619913.135232:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571391232 00000100:00100000:2.0:1788619913.135242:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571391232 00000100:00100000:2.0:1788619913.135301:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 850 00000100:00100000:2.0:1788619913.135312:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6343480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2353:x1875503571391232:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619913.135367:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619913.135389:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07200/0x1cd7fe53687344b5 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b998 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.135427:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a200/0x1cd7fe5368734414 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b913 expref: 15 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619913.135467:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d56a00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589085312:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619913.135536:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619913.135741:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571391360 00000100:00100000:3.0:1788619913.135745:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571391360 00000100:00100000:3.0:1788619913.135782:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 652 00000100:00100000:3.0:1788619913.135789:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6341180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+11:2353:x1875503571391360:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619913.135797:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619913.135808:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a72c5200/0x1cd7fe53687344bc lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b99f expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619913.135842:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d5f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589085440:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619913.135892:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164600 going to sleep for 11 seconds 00000100:00100000:3.0:1788619913.140097:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d5f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589085440:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619913.140145:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a72c5200/0x1cd7fe53687344bc lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b99f expref: 12 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.140198:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a72c5200/0x1cd7fe53687344bc lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b99f expref: 12 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.140211:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c400, rc 0) 00000100:00100000:3.0:1788619913.140268:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6341180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+11:2353:x1875503571391360:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4479us (5118us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619913.140277:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 652 00000100:00100000:2.0:1788619913.140801:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d56a00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589085312:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619913.140817:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 240354 00010000:00010000:2.0:1788619913.140836:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07200/0x1cd7fe53687344b5 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b998 expref: 20 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.140858:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619913.140862:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619913.140866:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07200/0x1cd7fe53687344b5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b998 expref: 20 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.140878:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa07200, rc 0) 00000100:00100000:2.0:1788619913.140914:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6343480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2353:x1875503571391232:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5604us (5827us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619913.140921:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 850 00000100:00100000:0.0:1788619913.144428:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619913.144613:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571391488 00000100:00100000:3.0:1788619913.144623:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571391488 00000100:00100000:3.0:1788619913.144669:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 851 00000100:00100000:3.0:1788619913.144687:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6343800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2355:x1875503571391488:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619913.144706:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619913.144725:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe53687344c3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b9a6 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.144761:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a200/0x1cd7fe5368734414 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b913 expref: 15 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619913.144799:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d5f100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589085568:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619913.144880:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164600 going to sleep for 11 seconds 00000100:00100000:1.0:1788619913.146994:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619913.147097:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571391616 00000100:00100000:2.0:1788619913.147104:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571391616 00000100:00100000:2.0:1788619913.147164:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 653 00000100:00100000:2.0:1788619913.147174:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d5bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+11:2355:x1875503571391616:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619913.147187:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619913.147203:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ce26d4f7/0x1cd7fe53687344ca lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b9ad expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619913.147253:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d56a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589085696:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619913.147342:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619913.147868:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d5f100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589085568:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619913.147887:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 240354 00010000:00010000:3.0:1788619913.147902:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe53687344c3 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9a6 expref: 20 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.147938:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619913.147941:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619913.147947:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe53687344c3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9a6 expref: 20 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.147960:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c400, rc 0) 00000100:00100000:3.0:1788619913.147999:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6343800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2355:x1875503571391488:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3313us (3578us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619913.148006:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 851 00000100:00100000:2.0:1788619913.151029:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d56a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589085696:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619913.151067:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ce26d4f7/0x1cd7fe53687344ca lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9ad expref: 12 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.151089:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ce26d4f7/0x1cd7fe53687344ca lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9ad expref: 12 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.151126:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa07800, rc 0) 00000100:00100000:2.0:1788619913.151184:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d5bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+11:2355:x1875503571391616:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4012us (4195us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619913.151192:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 653 00000100:00100000:1.0:1788619913.154357:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.154410:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619913.154818:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571391872 00000100:00100000:2.0:1788619913.154828:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571391872 00000100:00100000:2.0:1788619913.154887:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 654 00000100:00100000:2.0:1788619913.154897:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+11:2354:x1875503571391872:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619913.154925:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619913.154941:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001b18afa8/0x1cd7fe53687344d1 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b9bb expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619913.154998:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d56a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589085824:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619913.155077:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619913.155183:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571391744 00000100:00100000:3.0:1788619913.155187:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571391744 00000100:00100000:3.0:1788619913.155226:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 852 00000100:00100000:3.0:1788619913.155234:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d59c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2354:x1875503571391744:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619913.155243:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619913.155256:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe53687344d8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b9b4 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.155300:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a200/0x1cd7fe5368734414 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b913 expref: 15 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619913.155349:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d5f100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589085952:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619913.155377:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164600 going to sleep for 11 seconds 00000100:00100000:2.0:1788619913.158500:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d56a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589085824:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619913.158537:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001b18afa8/0x1cd7fe53687344d1 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9bb expref: 12 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.158563:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001b18afa8/0x1cd7fe53687344d1 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9bb expref: 12 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.158574:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa07200, rc 0) 00000100:00100000:2.0:1788619913.158631:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+11:2354:x1875503571391872:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3734us (4312us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619913.158654:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 654 00000100:00100000:3.0:1788619913.158814:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d5f100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589085952:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619913.158827:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 240354 00010000:00010000:3.0:1788619913.158841:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe53687344d8 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9b4 expref: 20 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.158872:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619913.158875:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619913.158879:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe53687344d8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9b4 expref: 20 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.158891:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c400, rc 0) 00000100:00100000:3.0:1788619913.158913:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d59c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2354:x1875503571391744:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3682us (4504us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619913.158920:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 852 00000100:00100000:1.0:1788619913.162071:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619913.162200:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571392000 00000100:00100000:2.0:1788619913.162211:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571392000 00000100:00100000:2.0:1788619913.162273:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 853 00000100:00100000:2.0:1788619913.162283:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d58e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2353:x1875503571392000:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619913.162303:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619913.162323:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07800/0x1cd7fe53687344df lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b9c2 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.162394:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a200/0x1cd7fe5368734414 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b913 expref: 15 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619913.162433:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d56a00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589086080:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619913.162504:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619913.164844:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619913.164943:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571392128 00000100:00100000:0.0:1788619913.164951:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571392128 00000100:00100000:0.0:1788619913.165006:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 655 00000100:00100000:0.0:1788619913.165016:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+11:2353:x1875503571392128:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619913.165032:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619913.165048:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000244b3d37/0x1cd7fe53687344e6 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b9c9 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619913.165101:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6340a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589086208:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619913.165184:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaab40 going to sleep for 11 seconds 00000100:00100000:2.0:1788619913.167698:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d56a00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589086080:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619913.167725:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 240354 00010000:00010000:2.0:1788619913.167744:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07800/0x1cd7fe53687344df lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9c2 expref: 20 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.167764:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619913.167767:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619913.167772:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07800/0x1cd7fe53687344df lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9c2 expref: 20 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.167785:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa07800, rc 0) 00000100:00100000:2.0:1788619913.167838:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d58e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2353:x1875503571392000:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5556us (5775us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619913.167889:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 853 00000100:00100000:0.0:1788619913.168436:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6340a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589086208:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619913.168461:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000244b3d37/0x1cd7fe53687344e6 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9c9 expref: 12 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.168478:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000244b3d37/0x1cd7fe53687344e6 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9c9 expref: 12 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.168488:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6f400, rc 0) 00000100:00100000:0.0:1788619913.168529:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+11:2353:x1875503571392128:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3513us (3692us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619913.168546:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 655 00000100:00100000:1.0:1788619913.172240:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619913.172368:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571392384 00000100:00100000:3.0:1788619913.172380:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571392384 00000100:00100000:3.0:1788619913.172437:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 854 00000100:00100000:3.0:1788619913.172448:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d5b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+19:14908:x1875503571392384:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:3.0:1788619913.172503:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619913.172530:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06000/0x1cd7fe536873444c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b936 expref: 19 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.172548:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619913.172552:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788619913.172769:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687344ed lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.172785:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe53687344ae lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b991 expref: 15 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.172794:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687344ed lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.172840:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe53687344ae lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b991 expref: 15 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.172863:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe53687344ae lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b991 expref: 15 pid: 5844 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619913.172877:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d5f100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589086336:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619913.172966:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164600 going to sleep for 11 seconds 00000100:00100000:3.0:1788619913.175696:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619913.175807:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d5f100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589086336:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619913.175831:0:5845:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687344ed lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619913.176667:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571392512 00000100:00100000:1.0:1788619913.176729:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571392512 00010000:00010000:1.0:1788619913.176767:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe53687344ae lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b991 expref: 16 pid: 5844 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619913.176846:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 287 00000100:00100000:1.0:1788619913.176864:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d5c700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571392512:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619913.176879:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619913.176886:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe53687344ae lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b991 expref: 16 pid: 5844 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.176899:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe53687344ae lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b991 expref: 16 pid: 5844 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.177003:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe53687344ae lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b991 expref: 16 pid: 5844 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.177020:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619913.177025:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687344ed lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.177052:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687344ed lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.177059:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687344ed lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.177129:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619913.177188:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d5c700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+15:2355:x1875503571392512:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 325us (1507us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619913.177195:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 287 00010000:00010000:0.0:1788619913.178241:0:5845:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687344ed lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.178252:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687344ed lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788619913.180468:0:5845:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967624 00000004:00010000:0.0:1788619913.180533:0:5845:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 240354 -> 240354 00000004:00010000:0.0:1788619913.180538:0:5845:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619912 -> 1788619913 00000004:00010000:0.0:1788619913.180540:0:5845:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619911 -> 1788619911 00000004:00010000:0.0:1788619913.180543:0:5845:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619912 -> 1788619913 00000004:00010000:0.0:1788619913.180545:0:5845:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 646 -> 646 00000004:00010000:0.0:1788619913.180555:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000f1b626fd reply state 00000000e33a98ba transno 4294967625 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687344ed lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619913.180567:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788619913.181018:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d5b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+21:14908:x1875503571392384:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 8567us (8784us total) trans 4294967625 rc 0/0 00000100:00100000:0.0:1788619913.181026:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 854 00000020:00080000:0.0:1788619913.181032:0:5845:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:1.0:1788619913.183883:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619913.184008:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571392640 00000100:00100000:3.0:1788619913.184018:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571392640 00000100:00100000:3.0:1788619913.184077:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 855 00000100:00100000:3.0:1788619913.184087:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d59180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+21:14908:x1875503571392640:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619913.184111:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619913.184136:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe53687344f4 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b9d0 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.184165:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a200/0x1cd7fe5368734414 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b913 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.184174:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe53687344f4 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9d0 expref: 22 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.184206:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a200/0x1cd7fe5368734414 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b913 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.184240:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a200/0x1cd7fe5368734414 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b913 expref: 14 pid: 5844 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619913.184254:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d5c000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589086464:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619913.184360:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164e40 going to sleep for 11 seconds 00000100:00100000:2.0:1788619913.187280:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.187369:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571392768 00000100:00100000:1.0:1788619913.187378:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571392768 00010000:00010000:1.0:1788619913.187412:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a200/0x1cd7fe5368734414 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b913 expref: 15 pid: 5844 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619913.187442:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 288 00000100:00100000:1.0:1788619913.187452:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d56a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+15:2355:x1875503571392768:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619913.187465:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619913.187490:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a200/0x1cd7fe5368734414 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b913 expref: 15 pid: 5844 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.187504:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a200/0x1cd7fe5368734414 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b913 expref: 15 pid: 5844 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.187525:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619913.187531:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe53687344f4 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9d0 expref: 22 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.187556:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe53687344f4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9d0 expref: 22 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.187565:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe53687344f4 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9d0 expref: 22 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.187603:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe53687344f4 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9d0 expref: 22 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619913.187615:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d5b800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589086592:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619913.187695:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963300 going to sleep for 11 seconds 00010000:00010000:3.0:1788619913.187891:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a200/0x1cd7fe5368734414 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b913 expref: 15 pid: 5844 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619913.187910:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d5c000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589086464:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619913.187933:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe53687344f4 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9d0 expref: 22 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.187944:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619913.187947:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619913.187950:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c400, rc 0) 00000100:00100000:3.0:1788619913.187995:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d59180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+22:14908:x1875503571392640:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3908us (4120us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619913.188003:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 855 00010000:00010000:0.0:1788619913.190503:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687344ed lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.190519:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687344ed lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.190539:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687344ed lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.190545:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c683d800 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788619913.190550:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687344ed lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.190558:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687344ed lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.190574:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619913.190577:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788619913.190582:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687344ed lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.190588:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687344ed lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619913.192645:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d5b800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589086592:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619913.192687:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619913.192747:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d56a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+14:2355:x1875503571392768:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 5295us (5474us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619913.192758:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 288 00000100:00100000:1.0:1788619913.195516:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619913.195591:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571392896 00000100:00100000:0.0:1788619913.195600:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571392896 00000100:00100000:0.0:1788619913.195657:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 656 00000100:00100000:0.0:1788619913.195668:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+11:14908:x1875503571392896:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619913.195696:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619913.195715:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000244b3d37/0x1cd7fe53687344fb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b9d7 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.195747:0:6696:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000daa9fe76/0x1cd7fe536873443e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b92f expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.195774:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(5 8) 00010000:00010000:0.0:1788619913.195778:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(6 8) 00010000:00010000:0.0:1788619913.195783:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000244b3d37/0x1cd7fe53687344fb lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9d7 expref: 12 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.195819:0:6696:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000daa9fe76/0x1cd7fe536873443e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b92f expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.195833:0:6696:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000daa9fe76/0x1cd7fe536873443e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b92f expref: 6 pid: 6697 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619913.195848:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6340a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589086720:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619913.195935:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaab40 going to sleep for 11 seconds 00000100:00100000:2.0:1788619913.198887:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619913.198995:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6340a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589086720:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619913.199027:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000244b3d37/0x1cd7fe53687344fb lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9d7 expref: 12 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.199046:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6f400, rc 0) 00000100:00100000:0.0:1788619913.199104:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:14908:x1875503571392896:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3438us (3597us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619913.199113:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 656 00000100:00100000:1.0:1788619913.199203:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571393024 00000100:00100000:1.0:1788619913.199210:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571393024 00010000:00010000:1.0:1788619913.199240:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000daa9fe76/0x1cd7fe536873443e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b92f expref: 7 pid: 6697 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619913.199275:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 289 00000100:00100000:1.0:1788619913.199283:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d57b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571393024:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619913.199311:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619913.199365:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000daa9fe76/0x1cd7fe536873443e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b92f expref: 7 pid: 6697 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.199379:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000daa9fe76/0x1cd7fe536873443e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b92f expref: 7 pid: 6697 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.199401:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000daa9fe76/0x1cd7fe536873443e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b92f expref: 7 pid: 6697 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.199414:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(6 8) 00010000:00010000:1.0:1788619913.199418:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(6 8) 00010000:00010000:1.0:1788619913.199427:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000244b3d37/0x1cd7fe53687344fb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9d7 expref: 11 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.199437:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000244b3d37/0x1cd7fe53687344fb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9d7 expref: 11 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.199469:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000244b3d37/0x1cd7fe53687344fb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9d7 expref: 11 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619913.199483:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d58a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589086848:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619913.199532:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963300 going to sleep for 11 seconds 00000100:00100000:1.0:1788619913.203308:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.203370:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619913.203452:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571393152 00000100:00100000:0.0:1788619913.203461:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571393152 00002000:00010000:0.0:1788619913.203508:0:5858:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:0.0:1788619913.203513:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5d5ad80 with x1875503571393152 00000100:00100000:0.0:1788619913.203556:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 470 00000100:00100000:0.0:1788619913.203565:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d5ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2354:x1875503571393152:12345-192.168.201.5@tcp:10:fsx.0 00000004:00010000:0.0:1788619913.203868:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 240354 -> 142359 00000004:00010000:0.0:1788619913.203872:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619913 -> 1788619913 00000004:00010000:0.0:1788619913.203874:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619911 -> 1788619911 00000004:00010000:0.0:1788619913.203877:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619913 -> 1788619913 00000004:00010000:0.0:1788619913.203879:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 646 -> 646 00000100:00100000:0.0:1788619913.203939:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d5ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2354:x1875503571393152:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 375us (641us total) trans 4294967626 rc 0/0 00000100:00100000:0.0:1788619913.203946:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 470 00002000:00010000:0.0:1788619913.203949:0:5858:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:0.0:1788619913.203951:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5d5ad80 with x1875503571393152 00000100:00100000:3.0:1788619913.204022:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571393280 00000100:00100000:3.0:1788619913.204027:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571393280 00002000:00010000:3.0:1788619913.204046:0:6701:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:3.0:1788619913.204050:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5d58380 with x1875503571393280 ext(0->18446744073709551615) 00002000:00010000:3.0:1788619913.204058:0:6701:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5d58380. 00000100:00100000:3.0:1788619913.204081:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 356 00000100:00100000:3.0:1788619913.204088:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571393280:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:3.0:1788619913.204261:0:6701:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 2073839 -> 0 00002000:00010000:3.0:1788619913.204265:0:6701:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619912 -> 1788619913 00002000:00010000:3.0:1788619913.204268:0:6701:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619911 -> 1788619911 00002000:00010000:3.0:1788619913.204270:0:6701:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619912 -> 1788619913 00002000:00010000:3.0:1788619913.204271:0:6701:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 135 -> 135 00000100:00100000:3.0:1788619913.204293:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571393280:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 205us (922us total) trans 4294967400 rc 0/0 00000100:00100000:3.0:1788619913.204299:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 356 00002000:00010000:3.0:1788619913.204302:0:6701:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:3.0:1788619913.204304:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5d58380 with x1875503571393280 ext(0->18446744073709551615) 00002000:00010000:3.0:1788619913.204308:0:6701:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5d58380. 00000100:00100000:2.0:1788619913.204555:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d58a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589086848:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619913.204575:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619913.204596:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d57b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571393024:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 5315us (5719us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619913.204604:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 289 00000100:00100000:2.0:1788619913.207432:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619913.207528:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571393408 00000100:00100000:0.0:1788619913.207538:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571393408 00000100:00100000:0.0:1788619913.207593:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 856 00000100:00100000:0.0:1788619913.207603:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c3255c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+22:14908:x1875503571393408:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619913.207624:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619913.207644:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe5368734502 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b9de expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.207738:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fe00/0x1cd7fe5368734509 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.207753:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fe00/0x1cd7fe5368734509 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.207782:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fe00/0x1cd7fe5368734509 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619913.207810:0:11397:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 142359 00000004:00200000:0.0:1788619913.207814:0:11397:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619913.207831:0:11397:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fe00/0x1cd7fe5368734509 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.207866:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe5368734502 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9de expref: 24 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.207884:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fe00/0x1cd7fe5368734509 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9de expref: 23 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.207892:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619913.207895:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619913.207899:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6fe00, rc 0) 00000100:00100000:0.0:1788619913.207954:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c3255c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+23:14908:x1875503571393408:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 353us (531us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619913.207961:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 856 00000100:00100000:2.0:1788619913.212765:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619913.213038:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571393536 00000100:00100000:0.0:1788619913.213048:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571393536 00000100:00100000:0.0:1788619913.213096:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 857 00000100:00100000:0.0:1788619913.213107:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d56680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+14:2354:x1875503571393536:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619913.213137:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619913.213156:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe5368734510 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b9e5 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.213210:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe53687344f4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9d0 expref: 22 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619913.213250:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6340380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589086976:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619913.213975:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa840 going to sleep for 11 seconds 00000100:00100000:1.0:1788619913.216015:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619913.216158:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571393664 00000100:00100000:2.0:1788619913.216166:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571393664 00000100:00100000:2.0:1788619913.216217:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 657 00000100:00100000:2.0:1788619913.216227:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d5a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571393664:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619913.216243:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619913.216258:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cb4402d/0x1cd7fe5368734517 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b9ec expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619913.216311:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d55c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589087104:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619913.216407:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619913.217194:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6340380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589086976:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619913.217214:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 142359 00010000:00010000:0.0:1788619913.217230:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe5368734510 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9e5 expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.217248:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619913.217251:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619913.217257:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe5368734510 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9e5 expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.217271:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6ee00, rc 0) 00000100:00100000:0.0:1788619913.217316:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d56680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+14:2354:x1875503571393536:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4210us (4560us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619913.217402:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 857 00000100:00100000:3.0:1788619913.219916:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d55c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589087104:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619913.219951:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005cb4402d/0x1cd7fe5368734517 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9ec expref: 7 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.219973:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cb4402d/0x1cd7fe5368734517 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9ec expref: 7 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.219985:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa06200, rc 0) 00000100:00100000:3.0:1788619913.220037:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d5a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571393664:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3811us (4029us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619913.220045:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 657 00000100:00100000:3.0:1788619913.222749:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619913.222814:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571393792 00000100:00100000:0.0:1788619913.222822:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571393792 00000100:00100000:0.0:1788619913.222886:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 858 00000100:00100000:0.0:1788619913.222895:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d5c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+14:14908:x1875503571393792:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619913.222920:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619913.222939:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e800/0x1cd7fe536873451e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b9f3 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.222970:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe53687344f4 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9d0 expref: 22 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.222980:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e800/0x1cd7fe536873451e lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9f3 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.223009:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe53687344f4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9d0 expref: 22 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.223020:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe53687344f4 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9d0 expref: 22 pid: 5846 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619913.223033:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6340380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589087232:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619913.223100:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa840 going to sleep for 11 seconds 00000100:00100000:1.0:1788619913.225372:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619913.225457:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571393920 00000100:00100000:2.0:1788619913.225465:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571393920 00010000:00010000:2.0:1788619913.225501:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe53687344f4 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9d0 expref: 23 pid: 5846 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619913.225543:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 290 00000100:00100000:2.0:1788619913.225566:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d58000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+23:2354:x1875503571393920:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619913.225579:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619913.225604:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe53687344f4 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9d0 expref: 23 pid: 5846 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.225616:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe53687344f4 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9d0 expref: 23 pid: 5846 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.225635:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619913.225642:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e800/0x1cd7fe536873451e lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9f3 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.225653:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e800/0x1cd7fe536873451e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9f3 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.225662:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e800/0x1cd7fe536873451e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9f3 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.225708:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e800/0x1cd7fe536873451e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9f3 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619913.225722:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d54e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589087360:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619913.225789:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc300 going to sleep for 11 seconds 00010000:00010000:0.0:1788619913.226263:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe53687344f4 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9d0 expref: 23 pid: 5846 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619913.226279:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6340380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589087232:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619913.226301:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e800/0x1cd7fe536873451e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9f3 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.226311:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619913.226314:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619913.226317:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6e800, rc 0) 00000100:00100000:0.0:1788619913.226390:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d5c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+15:14908:x1875503571393792:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3496us (3648us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619913.226405:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 858 00000100:00100000:1.0:1788619913.230157:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619913.230234:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571394048 00000100:00100000:0.0:1788619913.230243:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571394048 00002000:00100000:0.0:1788619913.230290:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5d5aa00 x1875503571394048/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619913.230301:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5d5aa00 with x1875503571394048 00000100:00100000:0.0:1788619913.230364:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 471 00000100:00100000:0.0:1788619913.230373:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d5aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+15:2354:x1875503571394048:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619913.230587:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d5aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+15:2354:x1875503571394048:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 215us (438us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619913.230596:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 471 00002000:00100000:0.0:1788619913.230603:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5d5aa00 x1875503571394048/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:9/0 lens 488/4536 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619913.230610:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5d5aa00 with x1875503571394048 00000100:00100000:2.0:1788619913.231230:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d54e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589087360:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619913.231251:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619913.231290:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d58000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2354:x1875503571393920:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 5725us (5928us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619913.231306:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 290 00000100:00100000:1.0:1788619913.234372:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619913.234522:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571394176 00000100:00100000:3.0:1788619913.234545:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571394176 00002000:00100000:3.0:1788619913.234593:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5d59c00 x1875503571394176/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619913.234604:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5d59c00 with x1875503571394176 00000100:00100000:3.0:1788619913.234638:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 472 00000100:00100000:3.0:1788619913.234647:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d59c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+15:2352:x1875503571394176:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619913.234878:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d59c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+15:2352:x1875503571394176:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 231us (553us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619913.234886:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 472 00002000:00100000:3.0:1788619913.234892:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5d59c00 x1875503571394176/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:9/0 lens 488/4536 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619913.234899:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5d59c00 with x1875503571394176 00000100:00100000:3.0:1788619913.238175:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.238690:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571394304 00000100:00100000:1.0:1788619913.238700:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571394304 00002000:00100000:1.0:1788619913.238748:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5d5c380 x1875503571394304/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619913.238760:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5d5c380 with x1875503571394304 00000100:00100000:1.0:1788619913.238796:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 473 00000100:00100000:1.0:1788619913.238806:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d5c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+15:2354:x1875503571394304:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619913.239020:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d5c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+15:2354:x1875503571394304:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 215us (853us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619913.239027:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 473 00002000:00100000:1.0:1788619913.239034:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5d5c380 x1875503571394304/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:9/0 lens 488/4536 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619913.239041:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5d5c380 with x1875503571394304 00000100:00100000:1.0:1788619913.246285:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619913.246603:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571394432 00000100:00100000:2.0:1788619913.246613:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571394432 00000100:00100000:2.0:1788619913.246764:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 658 00000100:00100000:2.0:1788619913.246774:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d58e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571394432:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619913.246796:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619913.246816:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000034a58768/0x1cd7fe5368734525 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1b9fa expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619913.246890:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d54e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589087488:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619913.247019:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc300 going to sleep for 11 seconds 00000100:00100000:1.0:1788619913.249128:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d54e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589087488:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619913.249164:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000034a58768/0x1cd7fe5368734525 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9fa expref: 7 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.249189:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000034a58768/0x1cd7fe5368734525 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9fa expref: 7 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.249221:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa06600, rc 0) 00000100:00100000:1.0:1788619913.249437:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d58e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571394432:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2663us (3159us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619913.249446:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 658 00000100:00100000:3.0:1788619913.252018:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.252300:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571394560 00000100:00100000:1.0:1788619913.252310:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571394560 00002000:00100000:1.0:1788619913.252423:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5d5d880 x1875503571394560/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619913.252567:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5d5d880 with x1875503571394560 00000100:00100000:1.0:1788619913.252640:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 474 00000100:00100000:1.0:1788619913.252650:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d5d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+15:2354:x1875503571394560:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619913.252859:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d5d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+15:2354:x1875503571394560:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 199us (838us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619913.252867:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 474 00002000:00100000:1.0:1788619913.252874:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5d5d880 x1875503571394560/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:9/0 lens 488/4536 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619913.252881:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5d5d880 with x1875503571394560 00000100:00100000:2.0:1788619913.256207:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619913.256368:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571394688 00000100:00100000:3.0:1788619913.256378:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571394688 00000100:00100000:3.0:1788619913.256441:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 859 00000100:00100000:3.0:1788619913.256452:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d55500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2354:x1875503571394688:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619913.256472:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619913.256492:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe536873452c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ba01 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.256525:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe536873452c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba01 expref: 23 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788619913.256536:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 142359 00010000:00010000:3.0:1788619913.256558:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe536873452c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba01 expref: 23 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.256570:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619913.256573:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619913.256576:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d800, rc 0) 00000100:00100000:3.0:1788619913.256627:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d55500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+23:2354:x1875503571394688:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 177us (429us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619913.256633:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 859 00000100:00100000:2.0:1788619913.261561:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619913.261974:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571394816 00000100:00100000:0.0:1788619913.261985:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571394816 00000100:00100000:0.0:1788619913.262514:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 860 00000100:00100000:0.0:1788619913.262526:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d54a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+23:14908:x1875503571394816:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619913.262546:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619913.262550:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:0.0:1788619913.262595:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe536873452c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba01 expref: 23 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.262614:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619913.262619:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788619913.262640:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe5368734533 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ba08 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.262665:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e800/0x1cd7fe536873451e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9f3 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.262685:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe5368734533 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba08 expref: 23 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.262718:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e800/0x1cd7fe536873451e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9f3 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.262733:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e800/0x1cd7fe536873451e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9f3 expref: 14 pid: 5846 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619913.262745:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6340380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589087616:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619913.262830:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaab40 going to sleep for 11 seconds 00000100:00100000:3.0:1788619913.271078:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.271233:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571394944 00000100:00100000:1.0:1788619913.271245:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571394944 00010000:00010000:1.0:1788619913.271287:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e800/0x1cd7fe536873451e lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9f3 expref: 15 pid: 5846 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619913.271365:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 291 00000100:00100000:1.0:1788619913.271375:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d5ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+15:2354:x1875503571394944:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619913.271406:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619913.271437:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e800/0x1cd7fe536873451e lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9f3 expref: 15 pid: 5846 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.271450:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e800/0x1cd7fe536873451e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9f3 expref: 15 pid: 5846 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.271474:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619913.271480:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe5368734533 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba08 expref: 23 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.271493:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe5368734533 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba08 expref: 23 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.271504:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe5368734533 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba08 expref: 23 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.271552:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe5368734533 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba08 expref: 23 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619913.271566:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d5b480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589087744:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619913.271651:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963f00 going to sleep for 11 seconds 00000100:00100000:1.0:1788619913.274740:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d5b480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589087744:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619913.274768:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619913.274827:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d5ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+15:2354:x1875503571394944:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3452us (3757us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619913.274866:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 291 00010000:00010000:0.0:1788619913.276018:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e800/0x1cd7fe536873451e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9f3 expref: 14 pid: 5846 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619913.276039:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6340380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589087616:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619913.276062:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe5368734533 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba08 expref: 23 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.276077:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619913.276080:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619913.276083:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6ee00, rc 0) 00000100:00100000:0.0:1788619913.276243:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d54a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+23:14908:x1875503571394816:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 13718us (14690us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619913.276251:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 860 00000100:00100000:3.0:1788619913.282966:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.283074:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571395072 00000100:00100000:1.0:1788619913.283084:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571395072 00000100:00100000:1.0:1788619913.283212:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 475 00000100:00100000:1.0:1788619913.283223:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d5fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+23:2355:x1875503571395072:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:1.0:1788619913.283539:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 142359 -> 142359 00000004:00010000:1.0:1788619913.283543:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619913 -> 1788619913 00000004:00010000:1.0:1788619913.283546:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619911 -> 1788619911 00000004:00010000:1.0:1788619913.283547:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619913 -> 1788619913 00000004:00010000:1.0:1788619913.283549:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 646 -> 646 00000100:00100000:1.0:1788619913.283624:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d5fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2355:x1875503571395072:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 402us (668us total) trans 4294967627 rc 0/0 00000100:00100000:1.0:1788619913.283632:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 475 00000100:00100000:3.0:1788619913.286180:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619913.286387:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571395200 00000100:00100000:0.0:1788619913.286395:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571395200 00000100:00100000:0.0:1788619913.286453:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 861 00000100:00100000:0.0:1788619913.286463:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d5e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+14:14908:x1875503571395200:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619913.286492:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619913.286511:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e800/0x1cd7fe536873453a lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ba0f expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.286586:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e200/0x1cd7fe5368734541 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.286603:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e200/0x1cd7fe5368734541 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.286613:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e200/0x1cd7fe5368734541 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619913.286639:0:5846:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 142359 00000004:00200000:0.0:1788619913.286644:0:5846:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619913.286662:0:5846:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e200/0x1cd7fe5368734541 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.286699:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e800/0x1cd7fe536873453a lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba0f expref: 16 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.286718:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e200/0x1cd7fe5368734541 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba0f expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.286743:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619913.286746:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619913.286750:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6e200, rc 0) 00000100:00100000:0.0:1788619913.286810:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d5e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+15:14908:x1875503571395200:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 348us (639us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619913.286817:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 861 00000100:00100000:3.0:1788619913.289041:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619913.289074:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619913.289150:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571395328 00000100:00100000:2.0:1788619913.289157:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571395328 00000100:00100000:2.0:1788619913.289207:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 862 00000100:00100000:2.0:1788619913.289216:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d5dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+15:2354:x1875503571395328:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619913.289233:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619913.289249:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe5368734548 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ba16 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.289280:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe5368734533 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba08 expref: 23 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619913.289318:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d56300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589087872:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619913.289415:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc000 going to sleep for 11 seconds 00000100:00100000:0.0:1788619913.290790:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571395456 00000100:00100000:0.0:1788619913.290799:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571395456 00000100:00100000:0.0:1788619913.290862:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 659 00000100:00100000:0.0:1788619913.290872:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571395456:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619913.290884:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619913.290899:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4614856/0x1cd7fe536873454f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ba1d expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619913.290947:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6340380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589088000:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619913.291011:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaab40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619913.293726:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6340380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589088000:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619913.293766:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a4614856/0x1cd7fe536873454f lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba1d expref: 7 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.293792:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4614856/0x1cd7fe536873454f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba1d expref: 7 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.293805:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6f000, rc 0) 00000100:00100000:0.0:1788619913.293889:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571395456:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3018us (4812us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619913.293898:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 659 00000100:00100000:2.0:1788619913.294592:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d56300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589087872:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619913.294612:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 142359 00010000:00010000:2.0:1788619913.294627:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe5368734548 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba16 expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.294642:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619913.294645:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619913.294649:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe5368734548 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba16 expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.294660:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa07a00, rc 0) 00000100:00100000:2.0:1788619913.294714:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d5dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+15:2354:x1875503571395328:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5499us (5678us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619913.294721:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 862 00000100:00100000:3.0:1788619913.298767:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619913.298810:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619913.299035:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571395584 00000100:00100000:0.0:1788619913.299044:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571395584 00000100:00100000:0.0:1788619913.299106:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 863 00000100:00100000:0.0:1788619913.299126:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d5d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+15:2352:x1875503571395584:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619913.299148:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619913.299170:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f800/0x1cd7fe5368734556 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ba24 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.299206:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe5368734533 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba08 expref: 23 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619913.299247:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6340380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589088128:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619913.299322:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaab40 going to sleep for 11 seconds 00000100:00100000:1.0:1788619913.299927:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571395712 00000100:00100000:1.0:1788619913.299932:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571395712 00000100:00100000:1.0:1788619913.299973:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 660 00000100:00100000:1.0:1788619913.299980:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe13a450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571395712:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619913.299989:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619913.300000:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3a77911/0x1cd7fe536873455d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ba2b expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619913.300038:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d5b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589088256:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619913.300080:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963f00 going to sleep for 11 seconds 00000100:00100000:0.0:1788619913.304080:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6340380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589088128:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619913.304105:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 142359 00010000:00010000:0.0:1788619913.304137:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f800/0x1cd7fe5368734556 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba24 expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.304159:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619913.304162:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619913.304167:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f800/0x1cd7fe5368734556 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba24 expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.304181:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6f800, rc 0) 00000100:00100000:0.0:1788619913.304256:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d5d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+15:2352:x1875503571395584:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5140us (5496us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619913.304264:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 863 00000100:00100000:1.0:1788619913.304824:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d5b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589088256:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619913.304873:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d3a77911/0x1cd7fe536873455d lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba2b expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.304892:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3a77911/0x1cd7fe536873455d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba2b expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.304904:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f4aa00, rc 0) 00000100:00100000:1.0:1788619913.304947:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe13a450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571395712:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4967us (6136us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619913.304953:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 660 00000100:00100000:3.0:1788619913.310482:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.310663:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571395840 00000100:00100000:1.0:1788619913.310681:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571395840 00002000:00100000:1.0:1788619913.310729:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5d5f480 x1875503571395840/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619913.310739:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5d5f480 with x1875503571395840 00000100:00100000:1.0:1788619913.310765:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 476 00000100:00100000:1.0:1788619913.310775:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d5f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2353:x1875503571395840:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619913.311024:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d5f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2353:x1875503571395840:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 248us (551us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619913.311031:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 476 00002000:00100000:1.0:1788619913.311038:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5d5f480 x1875503571395840/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/4536 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619913.311046:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5d5f480 with x1875503571395840 00000100:00100000:3.0:1788619913.313984:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.314060:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571395968 00000100:00100000:1.0:1788619913.314080:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571395968 00002000:00100000:1.0:1788619913.314131:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5d5ed80 x1875503571395968/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619913.314142:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5d5ed80 with x1875503571395968 00000100:00100000:1.0:1788619913.314181:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 477 00000100:00100000:1.0:1788619913.314202:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d5ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2353:x1875503571395968:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619913.314428:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d5ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2353:x1875503571395968:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 237us (452us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619913.314436:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 477 00002000:00100000:1.0:1788619913.314442:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5d5ed80 x1875503571395968/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/4536 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619913.314449:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5d5ed80 with x1875503571395968 00000100:00100000:2.0:1788619913.319499:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619913.319581:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571396096 00000100:00100000:0.0:1788619913.319590:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571396096 00002000:00100000:0.0:1788619913.319640:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5d56300 x1875503571396096/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619913.319649:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5d56300 with x1875503571396096 00000100:00100000:0.0:1788619913.319695:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 478 00000100:00100000:0.0:1788619913.319704:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d56300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2352:x1875503571396096:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619913.319925:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d56300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2352:x1875503571396096:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 221us (434us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619913.319933:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 478 00002000:00100000:0.0:1788619913.319940:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5d56300 x1875503571396096/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/4536 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619913.319947:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5d56300 with x1875503571396096 00000100:00100000:3.0:1788619913.323388:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619913.323635:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571396224 00000100:00100000:0.0:1788619913.323646:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571396224 00002000:00100000:0.0:1788619913.323697:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5d5ce00 x1875503571396224/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619913.323708:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5d5ce00 with x1875503571396224 00000100:00100000:0.0:1788619913.323748:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 479 00000100:00100000:0.0:1788619913.323758:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d5ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2353:x1875503571396224:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619913.323977:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d5ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2353:x1875503571396224:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 219us (596us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619913.324001:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 479 00002000:00100000:0.0:1788619913.324008:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5d5ce00 x1875503571396224/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/4536 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619913.324015:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5d5ce00 with x1875503571396224 00000100:00100000:3.0:1788619913.327526:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.327620:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571396352 00000100:00100000:1.0:1788619913.327630:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571396352 00002000:00100000:1.0:1788619913.327686:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5d5d180 x1875503571396352/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619913.327711:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5d5d180 with x1875503571396352 00000100:00100000:1.0:1788619913.327747:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 480 00000100:00100000:1.0:1788619913.327756:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d5d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2352:x1875503571396352:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619913.327969:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d5d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2352:x1875503571396352:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 213us (451us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619913.327976:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 480 00002000:00100000:1.0:1788619913.327983:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5d5d180 x1875503571396352/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/4536 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619913.327990:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5d5d180 with x1875503571396352 00000100:00100000:2.0:1788619913.333176:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.333506:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571396480 00000100:00100000:1.0:1788619913.333516:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571396480 00002000:00100000:1.0:1788619913.333570:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5d54700 x1875503571396480/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619913.333582:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5d54700 with x1875503571396480 00000100:00100000:1.0:1788619913.333623:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 481 00000100:00100000:1.0:1788619913.333633:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d54700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2352:x1875503571396480:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619913.334174:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d54700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2352:x1875503571396480:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 540us (1004us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619913.334185:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 481 00002000:00100000:1.0:1788619913.334193:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5d54700 x1875503571396480/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/4536 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619913.334201:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5d54700 with x1875503571396480 00000100:00100000:3.0:1788619913.339232:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619913.339391:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571396608 00000100:00100000:0.0:1788619913.339401:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571396608 00002000:00100000:0.0:1788619913.339452:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5d5f800 x1875503571396608/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619913.339462:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5d5f800 with x1875503571396608 00000100:00100000:0.0:1788619913.339504:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 482 00000100:00100000:0.0:1788619913.339513:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d5f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2354:x1875503571396608:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619913.339744:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d5f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2354:x1875503571396608:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 231us (521us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619913.339751:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 482 00002000:00100000:0.0:1788619913.339758:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5d5f800 x1875503571396608/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/4536 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619913.339766:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5d5f800 with x1875503571396608 00000100:00100000:2.0:1788619913.344922:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619913.345342:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571396736 00000100:00100000:3.0:1788619913.345353:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571396736 00002000:00100000:3.0:1788619913.345400:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5d54000 x1875503571396736/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619913.345410:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5d54000 with x1875503571396736 00000100:00100000:3.0:1788619913.345485:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 483 00000100:00100000:3.0:1788619913.345495:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d54000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2355:x1875503571396736:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619913.345836:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d54000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2355:x1875503571396736:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 342us (925us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619913.345845:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 483 00002000:00100000:3.0:1788619913.345864:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5d54000 x1875503571396736/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/4536 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619913.345871:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5d54000 with x1875503571396736 00000100:00100000:2.0:1788619913.348926:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.349038:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571396864 00000100:00100000:1.0:1788619913.349049:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571396864 00002000:00100000:1.0:1788619913.349096:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5d55880 x1875503571396864/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619913.349107:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5d55880 with x1875503571396864 00000100:00100000:1.0:1788619913.349241:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 484 00000100:00100000:1.0:1788619913.349251:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d55880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2355:x1875503571396864:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619913.349491:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d55880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2355:x1875503571396864:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 241us (646us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619913.349499:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 484 00002000:00100000:1.0:1788619913.349505:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5d55880 x1875503571396864/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/4536 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619913.349525:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5d55880 with x1875503571396864 00000100:00100000:3.0:1788619913.352369:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619913.353602:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571396992 00000100:00100000:0.0:1788619913.353615:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571396992 00002000:00100000:0.0:1788619913.353677:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5d5e300 x1875503571396992/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619913.353690:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5d5e300 with x1875503571396992 00000100:00100000:0.0:1788619913.353716:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 485 00000100:00100000:0.0:1788619913.353726:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d5e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2352:x1875503571396992:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619913.354107:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d5e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2352:x1875503571396992:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 381us (1778us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619913.354124:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 485 00002000:00100000:0.0:1788619913.354131:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5d5e300 x1875503571396992/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/4536 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619913.354139:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5d5e300 with x1875503571396992 00000100:00100000:2.0:1788619913.357560:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619913.357656:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571397120 00000100:00100000:0.0:1788619913.357665:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571397120 00002000:00100000:0.0:1788619913.357774:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5d57480 x1875503571397120/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619913.357785:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5d57480 with x1875503571397120 00000100:00100000:0.0:1788619913.357927:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 486 00000100:00100000:0.0:1788619913.357936:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d57480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2353:x1875503571397120:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619913.358207:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d57480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2353:x1875503571397120:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 271us (656us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619913.358215:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 486 00002000:00100000:0.0:1788619913.358222:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5d57480 x1875503571397120/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/4536 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619913.358230:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5d57480 with x1875503571397120 00000100:00100000:2.0:1788619913.361440:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619913.361535:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571397248 00000100:00100000:3.0:1788619913.361544:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571397248 00002000:00100000:3.0:1788619913.361588:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5d55180 x1875503571397248/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619913.361599:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5d55180 with x1875503571397248 00000100:00100000:3.0:1788619913.361692:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 487 00000100:00100000:3.0:1788619913.361704:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d55180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2354:x1875503571397248:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619913.361912:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d55180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2354:x1875503571397248:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 211us (481us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619913.361920:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 487 00002000:00100000:3.0:1788619913.361926:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5d55180 x1875503571397248/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/4536 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619913.361933:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5d55180 with x1875503571397248 00000100:00100000:2.0:1788619913.365523:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619913.365624:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571397376 00000100:00100000:0.0:1788619913.365634:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571397376 00002000:00100000:0.0:1788619913.365741:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5d56d80 x1875503571397376/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619913.365753:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5d56d80 with x1875503571397376 00000100:00100000:0.0:1788619913.365804:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 488 00000100:00100000:0.0:1788619913.365815:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d56d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2355:x1875503571397376:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619913.366237:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d56d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2355:x1875503571397376:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 423us (720us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619913.366245:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 488 00002000:00100000:0.0:1788619913.366252:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5d56d80 x1875503571397376/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/4536 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619913.366259:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5d56d80 with x1875503571397376 00000100:00100000:2.0:1788619913.369412:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619913.369522:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571397504 00000100:00100000:3.0:1788619913.369531:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571397504 00002000:00100000:3.0:1788619913.369577:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5d55f80 x1875503571397504/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619913.369586:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5d55f80 with x1875503571397504 00000100:00100000:3.0:1788619913.369635:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 489 00000100:00100000:3.0:1788619913.369644:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d55f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2352:x1875503571397504:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619913.369884:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d55f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2352:x1875503571397504:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 240us (480us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619913.369893:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 489 00002000:00100000:3.0:1788619913.369899:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5d55f80 x1875503571397504/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/4536 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619913.369906:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5d55f80 with x1875503571397504 00000100:00100000:2.0:1788619913.373220:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619913.373318:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571397632 00000100:00100000:0.0:1788619913.373361:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571397632 00002000:00100000:0.0:1788619913.373410:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5d57100 x1875503571397632/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619913.373420:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5d57100 with x1875503571397632 00000100:00100000:0.0:1788619913.373455:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 490 00000100:00100000:0.0:1788619913.373464:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d57100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2355:x1875503571397632:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619913.373667:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d57100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2355:x1875503571397632:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 204us (457us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619913.373683:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 490 00002000:00100000:0.0:1788619913.373690:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5d57100 x1875503571397632/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/4536 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619913.373698:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5d57100 with x1875503571397632 00000100:00100000:2.0:1788619913.378495:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.378598:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571397760 00000100:00100000:1.0:1788619913.378608:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571397760 00002000:00100000:1.0:1788619913.378656:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5d54380 x1875503571397760/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 504/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619913.378669:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5d54380 with x1875503571397760 00000100:00100000:1.0:1788619913.378719:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 491 00000100:00100000:1.0:1788619913.378729:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d54380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2354:x1875503571397760:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619913.383531:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d54380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2354:x1875503571397760:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 4792us (5032us total) trans 0 rc 81920/81920 00000100:00100000:1.0:1788619913.383545:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 491 00002000:00100000:1.0:1788619913.383558:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5d54380 x1875503571397760/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 504/440 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 81920/81920 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619913.383568:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5d54380 with x1875503571397760 00000100:00100000:2.0:1788619913.386595:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619913.386692:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619913.386792:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571397888 00000100:00100000:0.0:1788619913.386802:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571397888 00000100:00100000:0.0:1788619913.386875:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 864 00000100:00100000:0.0:1788619913.386886:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d55c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+15:2354:x1875503571397888:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619913.386920:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619913.386941:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe5368734564 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ba32 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.386977:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe5368734533 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba08 expref: 23 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619913.387014:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6340380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589088384:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619913.387099:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619913.389301:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571398016 00000100:00100000:3.0:1788619913.389317:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571398016 00000100:00100000:3.0:1788619913.389565:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 661 00000100:00100000:3.0:1788619913.389574:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d54e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571398016:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619913.389591:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619913.389608:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000095a50c3e/0x1cd7fe536873456b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ba39 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619913.389663:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d5f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589088512:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619913.389960:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164300 going to sleep for 11 seconds 00000004:00010000:2.0:1788619913.392662:0:9417:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619911 -> 1788619913 00000100:00100000:2.0:1788619913.392706:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6340380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589088384:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619913.392744:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 142359 00010000:00010000:2.0:1788619913.392763:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe5368734564 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba32 expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.392783:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619913.392786:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619913.392791:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe5368734564 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba32 expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.392806:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6ea00, rc 0) 00000100:00100000:2.0:1788619913.393011:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d55c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+15:2354:x1875503571397888:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6125us (6317us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619913.393020:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 864 00002000:00010000:3.0:1788619913.394651:0:6697:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619911 -> 1788619913 00000100:00100000:3.0:1788619913.394750:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d5f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589088512:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619913.394787:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000095a50c3e/0x1cd7fe536873456b lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba39 expref: 7 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.394810:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000095a50c3e/0x1cd7fe536873456b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba39 expref: 7 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.394844:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d600, rc 0) 00000100:00100000:3.0:1788619913.395066:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d54e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571398016:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5491us (8481us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619913.395073:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 661 00000100:00100000:0.0:1788619913.397803:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619913.397945:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619913.398465:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571398272 00000100:00100000:3.0:1788619913.398475:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571398272 00000100:00100000:3.0:1788619913.398536:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 662 00000100:00100000:3.0:1788619913.398546:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6342a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571398272:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619913.398563:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619913.398581:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe5368734572 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ba47 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619913.398637:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d5f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589088640:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619913.398720:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164300 going to sleep for 11 seconds 00000100:00100000:1.0:1788619913.400768:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d5f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589088640:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619913.400805:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe5368734572 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba47 expref: 7 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.400832:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe5368734572 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba47 expref: 7 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.400844:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c600, rc 0) 00000100:00100000:1.0:1788619913.400920:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6342a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571398272:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2376us (2975us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619913.400944:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 662 00000100:00100000:2.0:1788619913.402450:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571398144 00000100:00100000:2.0:1788619913.402461:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571398144 00000100:00100000:2.0:1788619913.402545:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 865 00000100:00100000:2.0:1788619913.402557:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6341880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+15:2352:x1875503571398144:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619913.402575:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619913.402595:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07800/0x1cd7fe5368734579 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ba40 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.402628:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe5368734533 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba08 expref: 23 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619913.402678:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501f480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589088768:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619913.402750:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc240 going to sleep for 11 seconds 00000100:00100000:1.0:1788619913.405853:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501f480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589088768:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619913.405878:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 142359 00010000:00010000:1.0:1788619913.405896:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07800/0x1cd7fe5368734579 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba40 expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.405984:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619913.405988:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619913.405995:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07800/0x1cd7fe5368734579 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba40 expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.406008:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa07800, rc 0) 00000100:00100000:1.0:1788619913.406121:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6341880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+15:2352:x1875503571398144:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3558us (8319us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619913.406131:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 865 00000100:00100000:3.0:1788619913.408880:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619913.408924:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619913.409173:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571398400 00000100:00100000:2.0:1788619913.409198:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571398400 00000100:00100000:2.0:1788619913.409260:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 866 00000100:00100000:2.0:1788619913.409271:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d5f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+15:2354:x1875503571398400:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619913.409289:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619913.409310:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06c00/0x1cd7fe5368734580 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ba4e expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.409377:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe5368734533 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba08 expref: 23 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619913.409417:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501c700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589088896:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619913.409492:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc780 going to sleep for 11 seconds 00000100:00100000:0.0:1788619913.413491:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571398528 00000100:00100000:0.0:1788619913.413504:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571398528 00000100:00100000:0.0:1788619913.413572:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 663 00000100:00100000:0.0:1788619913.413582:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d5df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571398528:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619913.413598:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619913.413616:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c2edd0f/0x1cd7fe5368734587 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ba55 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619913.413685:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6342680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589089024:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619913.413766:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa780 going to sleep for 11 seconds 00000100:00100000:0.0:1788619913.415826:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6342680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589089024:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619913.415872:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005c2edd0f/0x1cd7fe5368734587 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba55 expref: 7 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.415896:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c2edd0f/0x1cd7fe5368734587 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba55 expref: 7 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.415933:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6e600, rc 0) 00000100:00100000:0.0:1788619913.415986:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d5df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571398528:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2405us (7113us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619913.415994:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 663 00000100:00100000:1.0:1788619913.416604:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501c700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589088896:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619913.416622:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 142359 00010000:00010000:1.0:1788619913.416635:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06c00/0x1cd7fe5368734580 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba4e expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.416649:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619913.416653:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619913.416658:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06c00/0x1cd7fe5368734580 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba4e expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.416670:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa06c00, rc 0) 00000100:00100000:1.0:1788619913.416707:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d5f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+15:2354:x1875503571398400:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7440us (7784us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619913.416714:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 866 00000100:00100000:3.0:1788619913.420445:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619913.420635:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571398656 00000100:00100000:2.0:1788619913.420645:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571398656 00000100:00100000:2.0:1788619913.420708:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 867 00000100:00100000:2.0:1788619913.420719:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7581c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+15:14908:x1875503571398656:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619913.420739:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619913.420760:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06a00/0x1cd7fe536873458e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ba5c expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.420790:0:9417:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe5368734533 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba08 expref: 23 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.420800:0:9417:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06a00/0x1cd7fe536873458e lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba5c expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.420833:0:9417:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe5368734533 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba08 expref: 23 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.420871:0:9417:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe5368734533 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba08 expref: 23 pid: 5845 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619913.420886:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501d880 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589089152:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619913.421268:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc600 going to sleep for 11 seconds 00000100:00100000:1.0:1788619913.427045:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619913.427414:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571398784 00000100:00100000:3.0:1788619913.427424:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571398784 00010000:00010000:3.0:1788619913.427466:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe5368734533 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba08 expref: 24 pid: 5845 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619913.427507:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 292 00000100:00100000:3.0:1788619913.427516:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8022850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2354:x1875503571398784:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619913.427547:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619913.427571:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe5368734533 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba08 expref: 24 pid: 5845 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.427584:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe5368734533 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba08 expref: 24 pid: 5845 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.427606:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619913.427612:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06a00/0x1cd7fe536873458e lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba5c expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.427624:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06a00/0x1cd7fe536873458e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba5c expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.427633:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06a00/0x1cd7fe536873458e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba5c expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.427669:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06a00/0x1cd7fe536873458e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba5c expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619913.427694:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c7583100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589089280:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619913.427764:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164600 going to sleep for 11 seconds 00010000:00010000:0.0:1788619913.428159:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe5368734533 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba08 expref: 24 pid: 5845 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619913.428176:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501d880 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589089152:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619913.428200:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06a00/0x1cd7fe536873458e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba5c expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.428211:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619913.428214:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619913.428217:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa06a00, rc 0) 00000100:00100000:0.0:1788619913.428255:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7581c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+16:14908:x1875503571398656:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7537us (7821us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619913.428262:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 867 00000100:00100000:0.0:1788619913.430285:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c7583100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589089280:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619913.430309:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619913.430380:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8022850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+23:2354:x1875503571398784:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 2864us (3342us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619913.430389:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 292 00000100:00100000:3.0:1788619913.432952:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619913.433036:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571398912 00000100:00100000:0.0:1788619913.433044:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571398912 00002000:00100000:0.0:1788619913.433103:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c7582300 x1875503571398912/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619913.433124:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c7582300 with x1875503571398912 00000100:00100000:0.0:1788619913.433159:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 492 00000100:00100000:0.0:1788619913.433168:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7582300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571398912:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619913.433405:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7582300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571398912:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 237us (461us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619913.433413:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 492 00002000:00100000:0.0:1788619913.433419:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c7582300 x1875503571398912/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:9/0 lens 488/4536 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619913.433426:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c7582300 with x1875503571398912 00000100:00100000:3.0:1788619913.436019:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.436089:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571399040 00000100:00100000:1.0:1788619913.436097:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571399040 00002000:00100000:1.0:1788619913.436144:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c7581500 x1875503571399040/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619913.436154:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c7581500 with x1875503571399040 00000100:00100000:1.0:1788619913.436187:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 493 00000100:00100000:1.0:1788619913.436196:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7581500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+16:2353:x1875503571399040:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619913.436399:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7581500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+16:2353:x1875503571399040:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 203us (387us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619913.436405:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 493 00002000:00100000:1.0:1788619913.436412:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c7581500 x1875503571399040/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:9/0 lens 488/4536 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619913.436418:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c7581500 with x1875503571399040 00000100:00100000:3.0:1788619913.439528:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.439656:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571399168 00000100:00100000:1.0:1788619913.439665:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571399168 00002000:00100000:1.0:1788619913.439721:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c7583100 x1875503571399168/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619913.439732:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c7583100 with x1875503571399168 00000100:00100000:1.0:1788619913.439771:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 494 00000100:00100000:1.0:1788619913.439782:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7583100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571399168:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619913.439989:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7583100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571399168:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 207us (470us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619913.440009:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 494 00002000:00100000:1.0:1788619913.440015:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c7583100 x1875503571399168/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:9/0 lens 488/4536 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619913.440023:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c7583100 with x1875503571399168 00000100:00100000:3.0:1788619913.443063:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619913.443421:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571399296 00000100:00100000:0.0:1788619913.443430:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571399296 00002000:00100000:0.0:1788619913.443479:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5694a80 x1875503571399296/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619913.443489:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5694a80 with x1875503571399296 00000100:00100000:0.0:1788619913.443514:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 495 00000100:00100000:0.0:1788619913.443522:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5694a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+16:2354:x1875503571399296:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619913.444092:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5694a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+16:2354:x1875503571399296:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 569us (1036us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619913.444101:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 495 00002000:00100000:0.0:1788619913.444109:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5694a80 x1875503571399296/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:9/0 lens 488/4536 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619913.444173:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5694a80 with x1875503571399296 00000100:00100000:3.0:1788619913.447054:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619913.447152:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571399424 00000100:00100000:2.0:1788619913.447161:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571399424 00002000:00100000:2.0:1788619913.447209:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5696d80 x1875503571399424/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619913.447220:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5696d80 with x1875503571399424 00000100:00100000:2.0:1788619913.447257:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 496 00000100:00100000:2.0:1788619913.447267:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5696d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571399424:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619913.447523:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5696d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571399424:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 257us (476us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619913.447531:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 496 00002000:00100000:2.0:1788619913.447538:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5696d80 x1875503571399424/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:9/0 lens 488/4536 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619913.447546:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5696d80 with x1875503571399424 00000100:00100000:3.0:1788619913.450153:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.450400:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571399552 00000100:00100000:1.0:1788619913.450410:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571399552 00000100:00100000:1.0:1788619913.450461:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 664 00000100:00100000:1.0:1788619913.450471:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5695180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571399552:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619913.450490:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619913.450507:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006443b428/0x1cd7fe5368734595 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ba63 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619913.450564:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d59880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589089408:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619913.450651:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963300 going to sleep for 11 seconds 00000100:00100000:1.0:1788619913.452886:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d59880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589089408:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619913.452926:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000006443b428/0x1cd7fe5368734595 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba63 expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.452948:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006443b428/0x1cd7fe5368734595 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba63 expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.452962:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f4b800, rc 0) 00000100:00100000:1.0:1788619913.453020:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5695180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571399552:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2548us (2873us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619913.453045:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 664 00000100:00100000:2.0:1788619913.456033:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619913.456232:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571399680 00000100:00100000:0.0:1788619913.456243:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571399680 00002000:00100000:0.0:1788619913.456292:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c501d180 x1875503571399680/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619913.456302:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c501d180 with x1875503571399680 00000100:00100000:0.0:1788619913.456370:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 497 00000100:00100000:0.0:1788619913.456380:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571399680:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619913.456582:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571399680:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 203us (557us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619913.456591:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 497 00002000:00100000:0.0:1788619913.456598:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c501d180 x1875503571399680/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:9/0 lens 488/4536 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619913.456606:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c501d180 with x1875503571399680 00000100:00100000:3.0:1788619913.464062:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619913.464356:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571399808 00000100:00100000:2.0:1788619913.464368:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571399808 00000100:00100000:2.0:1788619913.464431:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 665 00000100:00100000:2.0:1788619913.464442:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5694e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571399808:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619913.464459:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619913.464491:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0b04224/0x1cd7fe536873459c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ba6a expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619913.464553:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589089536:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619913.464631:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccc00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619913.472997:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589089536:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619913.473036:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e0b04224/0x1cd7fe536873459c lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba6a expref: 7 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.473061:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0b04224/0x1cd7fe536873459c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba6a expref: 7 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.473073:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa06800, rc 0) 00000100:00100000:2.0:1788619913.473238:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5694e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571399808:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8796us (9185us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619913.473247:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 665 00000100:00100000:3.0:1788619913.476920:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.477028:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571399936 00000100:00100000:1.0:1788619913.477040:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571399936 00000100:00100000:1.0:1788619913.477093:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 666 00000100:00100000:1.0:1788619913.477100:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5695f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571399936:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619913.477121:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619913.477136:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006443b428/0x1cd7fe53687345a3 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ba71 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619913.477181:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d59880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589089664:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619913.477266:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963300 going to sleep for 11 seconds 00000100:00100000:1.0:1788619913.480167:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d59880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589089664:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619913.480209:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000006443b428/0x1cd7fe53687345a3 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba71 expref: 7 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.480233:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006443b428/0x1cd7fe53687345a3 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba71 expref: 7 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.480247:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f4b800, rc 0) 00000100:00100000:1.0:1788619913.480306:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5695f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571399936:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3205us (3394us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619913.480315:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 666 00000100:00100000:3.0:1788619913.483149:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619913.483212:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571400064 00000100:00100000:0.0:1788619913.483222:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571400064 00000100:00100000:0.0:1788619913.483281:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 667 00000100:00100000:0.0:1788619913.484451:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5697b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571400064:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619913.484475:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619913.484495:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7f5b2cb/0x1cd7fe53687345aa lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ba78 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619913.484555:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6342680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589089792:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619913.484634:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:0.0:1788619913.487694:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6342680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589089792:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619913.487739:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c7f5b2cb/0x1cd7fe53687345aa lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba78 expref: 7 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.487782:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7f5b2cb/0x1cd7fe53687345aa lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba78 expref: 7 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.487795:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6ee00, rc 0) 00000100:00100000:0.0:1788619913.487868:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5697b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571400064:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4578us (4727us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619913.487878:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 667 00000100:00100000:3.0:1788619913.490350:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619913.490405:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571400192 00000100:00100000:0.0:1788619913.490414:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571400192 00000100:00100000:1.0:1788619913.490573:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 668 00000100:00100000:1.0:1788619913.490584:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5694380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571400192:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619913.490614:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619913.490633:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006443b428/0x1cd7fe53687345b1 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ba7f expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619913.490699:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d59880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589089920:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619913.490777:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963300 going to sleep for 11 seconds 00000100:00100000:1.0:1788619913.493047:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d59880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589089920:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619913.493088:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000006443b428/0x1cd7fe53687345b1 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba7f expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.493112:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006443b428/0x1cd7fe53687345b1 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba7f expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.493136:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f4b800, rc 0) 00000100:00100000:1.0:1788619913.493189:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5694380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571400192:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2608us (2872us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619913.493198:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 668 00000100:00100000:2.0:1788619913.495877:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.495969:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571400320 00000100:00100000:1.0:1788619913.495978:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571400320 00000100:00100000:1.0:1788619913.496032:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 868 00000100:00100000:1.0:1788619913.496042:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+23:2354:x1875503571400320:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619913.496061:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619913.496081:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a400/0x1cd7fe53687345b8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ba86 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.496149:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a400/0x1cd7fe53687345b8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba86 expref: 24 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788619913.496163:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 142359 00010000:00010000:1.0:1788619913.496170:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a400/0x1cd7fe53687345b8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba86 expref: 24 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.496182:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619913.496185:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619913.496189:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6f4a400, rc 0) 00000100:00100000:1.0:1788619913.496248:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2354:x1875503571400320:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 208us (381us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619913.496255:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 868 00000100:00100000:3.0:1788619913.499922:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.500071:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571400576 00000100:00100000:1.0:1788619913.500080:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571400576 00000100:00100000:1.0:1788619913.500143:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 869 00000100:00100000:1.0:1788619913.500155:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5697100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+24:14908:x1875503571400576:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:1.0:1788619913.500207:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619913.500233:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fe00/0x1cd7fe5368734509 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9de expref: 24 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.500251:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619913.500256:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788619913.500514:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a800/0x1cd7fe53687345bf lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.500528:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e200/0x1cd7fe5368734541 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba0f expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.500544:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a800/0x1cd7fe53687345bf lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.500577:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e200/0x1cd7fe5368734541 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba0f expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.500590:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e200/0x1cd7fe5368734541 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba0f expref: 15 pid: 5846 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619913.500602:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d59880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589090048:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619913.500676:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963300 going to sleep for 11 seconds 00000100:00100000:1.0:1788619913.503430:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d59880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589090048:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619913.503456:0:5845:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a800/0x1cd7fe53687345bf lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619913.504808:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619913.504915:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571400704 00000100:00100000:2.0:1788619913.504923:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571400704 00010000:00010000:2.0:1788619913.504965:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e200/0x1cd7fe5368734541 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba0f expref: 16 pid: 5846 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619913.505012:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 293 00000100:00100000:2.0:1788619913.505023:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5695c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571400704:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619913.505037:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619913.505044:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e200/0x1cd7fe5368734541 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba0f expref: 16 pid: 5846 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.505071:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e200/0x1cd7fe5368734541 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba0f expref: 16 pid: 5846 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.505094:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e200/0x1cd7fe5368734541 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba0f expref: 16 pid: 5846 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.505112:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619913.505118:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a800/0x1cd7fe53687345bf lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.505126:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a800/0x1cd7fe53687345bf lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.505134:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a800/0x1cd7fe53687345bf lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.505164:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788619913.505213:0:5845:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a800/0x1cd7fe53687345bf lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.505222:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a800/0x1cd7fe53687345bf lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788619913.505366:0:5845:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967628 00000100:00100000:2.0:1788619913.505379:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5695c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+15:2355:x1875503571400704:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 357us (575us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619913.505386:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 293 00000004:00010000:1.0:1788619913.505414:0:5845:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 142359 -> 142359 00000004:00010000:1.0:1788619913.505418:0:5845:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619913 -> 1788619913 00000004:00010000:1.0:1788619913.505421:0:5845:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619913 -> 1788619911 00000004:00010000:1.0:1788619913.505423:0:5845:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619913 -> 1788619913 00000004:00010000:1.0:1788619913.505425:0:5845:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 646 -> 646 00000004:00010000:1.0:1788619913.505440:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000003481a86a reply state 00000000d9d3c3a1 transno 4294967629 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a800/0x1cd7fe53687345bf lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619913.505448:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788619913.505515:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5697100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+26:14908:x1875503571400576:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 5363us (5613us total) trans 4294967629 rc 0/0 00000100:00100000:1.0:1788619913.505522:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 869 00000020:00080000:1.0:1788619913.505525:0:5845:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:1.0:1788619913.508082:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00010000:00010000:3.0:1788619913.508363:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a800/0x1cd7fe53687345bf lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.508375:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a800/0x1cd7fe53687345bf lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.508382:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a800/0x1cd7fe53687345bf lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.508388:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c6f4a800 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:3.0:1788619913.508392:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a800/0x1cd7fe53687345bf lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.508399:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a800/0x1cd7fe53687345bf lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.508414:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619913.508417:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788619913.508422:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a800/0x1cd7fe53687345bf lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.508429:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a800/0x1cd7fe53687345bf lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619913.509115:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571400832 00000100:00100000:2.0:1788619913.509127:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571400832 00000100:00100000:2.0:1788619913.509192:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 870 00000100:00100000:2.0:1788619913.509214:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d59880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+25:14908:x1875503571400832:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619913.509237:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619913.509240:0:9417:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1788619913.509277:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6f4a400/0x1cd7fe53687345b8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba86 expref: 25 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.509315:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619913.509319:0:9417:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788619913.509350:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06600/0x1cd7fe53687345c6 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ba8d expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.509374:0:9417:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06a00/0x1cd7fe536873458e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba5c expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.509383:0:9417:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06600/0x1cd7fe53687345c6 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba8d expref: 25 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.509427:0:9417:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06a00/0x1cd7fe536873458e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba5c expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.509440:0:9417:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06a00/0x1cd7fe536873458e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba5c expref: 14 pid: 9417 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619913.509454:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501e680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589090176:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619913.509530:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccc00 going to sleep for 11 seconds 00000100:00100000:3.0:1788619913.512196:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.512282:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571400960 00000100:00100000:1.0:1788619913.512292:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571400960 00010000:00010000:1.0:1788619913.512376:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06a00/0x1cd7fe536873458e lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba5c expref: 15 pid: 9417 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619913.512410:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 294 00000100:00100000:1.0:1788619913.512419:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5696a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+15:2354:x1875503571400960:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619913.512434:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619913.512464:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06a00/0x1cd7fe536873458e lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba5c expref: 15 pid: 9417 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.512477:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06a00/0x1cd7fe536873458e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba5c expref: 15 pid: 9417 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.512500:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619913.512506:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06600/0x1cd7fe53687345c6 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba8d expref: 25 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.512518:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06600/0x1cd7fe53687345c6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba8d expref: 25 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.512534:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06600/0x1cd7fe53687345c6 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba8d expref: 25 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.512550:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06a00/0x1cd7fe536873458e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba5c expref: 15 pid: 9417 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619913.512565:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501e680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589090176:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619913.512570:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06600/0x1cd7fe53687345c6 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba8d expref: 25 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619913.512583:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d5bb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589090304:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619913.512593:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06600/0x1cd7fe53687345c6 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba8d expref: 25 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.512626:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619913.512629:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619913.512632:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa06600, rc 0) 00000100:00100000:2.0:1788619913.512657:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d59880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+25:14908:x1875503571400832:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3447us (4586us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619913.512658:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963300 going to sleep for 11 seconds 00000100:00100000:2.0:1788619913.512664:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 870 00000100:00100000:1.0:1788619913.515383:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d5bb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589090304:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619913.515408:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619913.515453:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5696a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+14:2354:x1875503571400960:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3035us (3268us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619913.515463:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 294 00000100:00100000:3.0:1788619913.517661:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.518050:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571401088 00000100:00100000:1.0:1788619913.518060:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571401088 00002000:00010000:1.0:1788619913.518100:0:5859:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:1.0:1788619913.518105:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e647ad80 with x1875503571401088 00000100:00100000:1.0:1788619913.518152:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 498 00000100:00100000:1.0:1788619913.518162:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e647ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+25:2355:x1875503571401088:12345-192.168.201.5@tcp:10:fsx.0 00000004:00010000:1.0:1788619913.518377:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 142359 -> 945165 00000004:00010000:1.0:1788619913.518381:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619913 -> 1788619913 00000004:00010000:1.0:1788619913.518384:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619911 -> 1788619911 00000004:00010000:1.0:1788619913.518386:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619913 -> 1788619913 00000004:00010000:1.0:1788619913.518389:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 646 -> 646 00000100:00100000:1.0:1788619913.518460:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e647ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+26:2355:x1875503571401088:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 299us (808us total) trans 4294967630 rc 0/0 00000100:00100000:1.0:1788619913.518468:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 498 00002000:00010000:1.0:1788619913.518472:0:5859:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:1.0:1788619913.518474:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e647ad80 with x1875503571401088 00000100:00100000:3.0:1788619913.520591:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619913.523157:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571401216 00000100:00100000:2.0:1788619913.523172:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571401216 00002000:00010000:2.0:1788619913.523217:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:2.0:1788619913.523221:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78e6479f80 with x1875503571401216 ext(0->18446744073709551615) 00002000:00010000:2.0:1788619913.523232:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78e6479f80. 00000100:00100000:2.0:1788619913.523267:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 357 00000100:00100000:2.0:1788619913.523277:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e6479f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571401216:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:2.0:1788619913.523482:0:6702:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:2.0:1788619913.523485:0:6702:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619913 -> 1788619913 00002000:00010000:2.0:1788619913.523487:0:6702:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619913 -> 1788619911 00002000:00010000:2.0:1788619913.523488:0:6702:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619913 -> 1788619913 00002000:00010000:2.0:1788619913.523489:0:6702:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 135 -> 135 00000100:00100000:2.0:1788619913.523543:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e6479f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571401216:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 268us (2957us total) trans 4294967401 rc 0/0 00000100:00100000:2.0:1788619913.523551:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 357 00002000:00010000:2.0:1788619913.523554:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:2.0:1788619913.523557:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78e6479f80 with x1875503571401216 ext(0->18446744073709551615) 00002000:00010000:2.0:1788619913.523561:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78e6479f80. 00000100:00100000:1.0:1788619913.527049:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619913.527141:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571401344 00000100:00100000:3.0:1788619913.527150:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571401344 00000100:00100000:3.0:1788619913.527204:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 871 00000100:00100000:3.0:1788619913.527214:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d5bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+26:14908:x1875503571401344:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619913.527238:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619913.527257:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe53687345cd lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ba94 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.527366:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687345d4 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.527381:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687345d4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.527393:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687345d4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788619913.527418:0:11397:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 945165 00000004:00200000:3.0:1788619913.527422:0:11397:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788619913.527439:0:11397:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687345d4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.527463:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe53687345cd lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba94 expref: 28 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.527480:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687345d4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba94 expref: 27 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.527504:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619913.527507:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788619913.527511:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d600, rc 0) 00000100:00100000:3.0:1788619913.527615:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d5bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+27:14908:x1875503571401344:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 402us (574us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619913.527622:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 871 00000100:00100000:3.0:1788619913.534923:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619913.535111:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571401472 00000100:00100000:2.0:1788619913.535130:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571401472 00002000:00100000:2.0:1788619913.535180:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e647b100 x1875503571401472/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619913.535191:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e647b100 with x1875503571401472 00000100:00100000:2.0:1788619913.535230:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 499 00000100:00100000:2.0:1788619913.535240:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e647b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+27:2354:x1875503571401472:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619913.551870:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e647b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+27:2354:x1875503571401472:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 16621us (16947us total) trans 0 rc 946176/946176 00000100:00100000:2.0:1788619913.551884:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 499 00002000:00100000:2.0:1788619913.551897:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e647b100 x1875503571401472/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/440 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 946176/946176 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619913.551908:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e647b100 with x1875503571401472 00000100:00100000:2.0:1788619913.555367:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.555500:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571401600 00000100:00100000:1.0:1788619913.555511:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571401600 00000100:00100000:1.0:1788619913.555574:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 872 00000100:00100000:1.0:1788619913.555584:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+14:14908:x1875503571401600:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619913.555604:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619913.555623:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7c00/0x1cd7fe53687345db lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ba9b expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.555718:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7e00/0x1cd7fe53687345e2 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.555734:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7e00/0x1cd7fe53687345e2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.555746:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7e00/0x1cd7fe53687345e2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788619913.555773:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 945165 00000004:00200000:1.0:1788619913.555777:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788619913.555794:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7e00/0x1cd7fe53687345e2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.555821:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7c00/0x1cd7fe53687345db lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba9b expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.555874:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7e00/0x1cd7fe53687345e2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba9b expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.555885:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619913.555888:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788619913.555891:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78ccca7e00, rc 0) 00000100:00100000:1.0:1788619913.555949:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+15:14908:x1875503571401600:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 366us (630us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619913.555956:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 872 00000100:00100000:1.0:1788619913.559146:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.559189:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619913.559262:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571401856 00000100:00100000:3.0:1788619913.559271:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571401856 00000100:00100000:3.0:1788619913.559356:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 669 00000100:00100000:3.0:1788619913.559366:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571401856:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619913.559381:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619913.559397:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003b84bbc9/0x1cd7fe53687345e9 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1baa9 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619913.559451:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e6478700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589090432:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619913.559523:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164780 going to sleep for 11 seconds 00000100:00100000:2.0:1788619913.559961:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571401728 00000100:00100000:2.0:1788619913.559967:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571401728 00000100:00100000:2.0:1788619913.559998:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 873 00000100:00100000:2.0:1788619913.560005:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d5a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+15:2352:x1875503571401728:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619913.560015:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619913.560028:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe53687345f0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1baa2 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.560060:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06600/0x1cd7fe53687345c6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba8d expref: 26 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619913.560084:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501c380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589090560:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619913.560145:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccc00 going to sleep for 11 seconds 00000004:00010000:2.0:1788619913.563916:0:9417:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619911 -> 1788619913 00000100:00100000:2.0:1788619913.563948:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501c380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589090560:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619913.563969:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 945165 00010000:00010000:2.0:1788619913.563988:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe53687345f0 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1baa2 expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.564008:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619913.564011:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619913.564016:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe53687345f0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1baa2 expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.564031:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa07e00, rc 0) 00000100:00100000:2.0:1788619913.564089:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d5a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+15:2352:x1875503571401728:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4084us (4954us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619913.564097:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 873 00002000:00010000:3.0:1788619913.566126:0:6697:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619911 -> 1788619913 00000100:00100000:3.0:1788619913.566153:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e6478700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589090432:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619913.566184:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003b84bbc9/0x1cd7fe53687345e9 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1baa9 expref: 7 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.566207:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b84bbc9/0x1cd7fe53687345e9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1baa9 expref: 7 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.566240:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683de00, rc 0) 00000100:00100000:3.0:1788619913.566292:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571401856:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6928us (7101us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619913.566301:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 669 00000100:00100000:1.0:1788619913.569963:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.570007:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619913.570158:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571402112 00000100:00100000:3.0:1788619913.570168:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571402112 00000100:00100000:3.0:1788619913.570221:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 670 00000100:00100000:3.0:1788619913.570231:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d5a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571402112:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619913.570248:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619913.570265:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000485aa315/0x1cd7fe53687345f7 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bab7 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619913.570322:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e6478700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589090688:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619913.570430:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164780 going to sleep for 11 seconds 00000100:00100000:0.0:1788619913.570767:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571401984 00000100:00100000:0.0:1788619913.570772:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571401984 00000100:00100000:0.0:1788619913.570815:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 874 00000100:00100000:0.0:1788619913.570823:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d5b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+15:2354:x1875503571401984:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619913.570835:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619913.570863:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe53687345fe lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bab0 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.570896:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06600/0x1cd7fe53687345c6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba8d expref: 26 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619913.570923:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6342680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589090816:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619913.570962:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:0.0:1788619913.578025:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6342680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589090816:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619913.578052:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 945165 00010000:00010000:0.0:1788619913.578086:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe53687345fe lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bab0 expref: 16 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.578107:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619913.578120:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619913.578126:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe53687345fe lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bab0 expref: 16 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.578141:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6ee00, rc 0) 00000100:00100000:0.0:1788619913.578204:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d5b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+15:2354:x1875503571401984:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7381us (8251us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619913.578213:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 874 00000100:00100000:1.0:1788619913.580699:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e6478700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589090688:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619913.580735:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000485aa315/0x1cd7fe53687345f7 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bab7 expref: 7 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.580758:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000485aa315/0x1cd7fe53687345f7 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bab7 expref: 7 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.580770:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683cc00, rc 0) 00000100:00100000:1.0:1788619913.580820:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d5a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571402112:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 10591us (10812us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619913.580828:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 670 00000100:00100000:2.0:1788619913.583178:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619913.583241:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571402240 00000100:00100000:0.0:1788619913.583250:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571402240 00000100:00100000:0.0:1788619913.583309:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 500 00000100:00100000:0.0:1788619913.583319:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+27:2354:x1875503571402240:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:0.0:1788619913.583619:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 945165 -> 945165 00000004:00010000:0.0:1788619913.583624:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619913 -> 1788619913 00000004:00010000:0.0:1788619913.583626:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619913 -> 1788619913 00000004:00010000:0.0:1788619913.583629:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619913 -> 1788619913 00000004:00010000:0.0:1788619913.583631:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 646 -> 646 00000100:00100000:0.0:1788619913.583860:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+28:2354:x1875503571402240:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 540us (687us total) trans 4294967631 rc 0/0 00000100:00100000:0.0:1788619913.583867:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 500 00000100:00100000:2.0:1788619913.588690:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.588766:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571402368 00000100:00100000:1.0:1788619913.588774:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571402368 00000100:00100000:1.0:1788619913.588855:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 501 00000100:00100000:1.0:1788619913.588866:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+28:2353:x1875503571402368:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:1.0:1788619913.589095:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 945165 -> 945165 00000004:00010000:1.0:1788619913.589099:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619913 -> 1788619913 00000004:00010000:1.0:1788619913.589102:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619913 -> 1788619913 00000004:00010000:1.0:1788619913.589104:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619913 -> 1788619913 00000004:00010000:1.0:1788619913.589106:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 646 -> 646 00000100:00100000:1.0:1788619913.589183:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+29:2353:x1875503571402368:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 319us (502us total) trans 4294967632 rc 0/0 00000100:00100000:1.0:1788619913.589191:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 501 00000100:00100000:2.0:1788619913.596208:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.596314:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571402624 00000100:00100000:1.0:1788619913.596359:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571402624 00000100:00100000:1.0:1788619913.596415:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 875 00000100:00100000:1.0:1788619913.596424:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+29:14908:x1875503571402624:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:1.0:1788619913.596484:0:5844:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619913.596511:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687345d4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba94 expref: 29 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.596530:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619913.596535:0:5844:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788619913.596746:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7c00/0x1cd7fe5368734605 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.596760:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7e00/0x1cd7fe53687345e2 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba9b expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.596783:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7c00/0x1cd7fe5368734605 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.596813:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7e00/0x1cd7fe53687345e2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba9b expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.596826:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7e00/0x1cd7fe53687345e2 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba9b expref: 14 pid: 5845 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619913.596855:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d58e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589090944:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619913.597142:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963840 going to sleep for 11 seconds 00000100:00100000:2.0:1788619913.602302:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619913.602391:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571402752 00000100:00100000:3.0:1788619913.602400:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571402752 00010000:00010000:3.0:1788619913.602439:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7e00/0x1cd7fe53687345e2 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba9b expref: 15 pid: 5845 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619913.602481:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 295 00000100:00100000:3.0:1788619913.602491:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501d880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+15:2354:x1875503571402752:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619913.602506:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619913.602512:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7e00/0x1cd7fe53687345e2 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba9b expref: 15 pid: 5845 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.602545:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7e00/0x1cd7fe53687345e2 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba9b expref: 15 pid: 5845 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.602567:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619913.602573:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7c00/0x1cd7fe5368734605 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.602584:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7c00/0x1cd7fe5368734605 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.602592:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7c00/0x1cd7fe5368734605 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.602611:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619913.602680:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501d880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+15:2354:x1875503571402752:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 189us (386us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619913.602688:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 295 00010000:00010000:1.0:1788619913.603827:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7e00/0x1cd7fe53687345e2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba9b expref: 14 pid: 5845 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619913.604241:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d58e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589090944:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619913.604264:0:5844:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7c00/0x1cd7fe5368734605 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.604275:0:5844:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7c00/0x1cd7fe5368734605 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.604281:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7c00/0x1cd7fe5368734605 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788619913.604461:0:5844:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967633 00000004:00010000:1.0:1788619913.604510:0:5844:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 945165 -> 945165 00000004:00010000:1.0:1788619913.604514:0:5844:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619913 -> 1788619913 00000004:00010000:1.0:1788619913.604528:0:5844:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619913 -> 1788619913 00000004:00010000:1.0:1788619913.604531:0:5844:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619913 -> 1788619913 00000004:00010000:1.0:1788619913.604533:0:5844:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 646 -> 646 00000004:00010000:1.0:1788619913.604541:0:5844:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000dc019f66 reply state 00000000c95a1e38 transno 4294967634 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7c00/0x1cd7fe5368734605 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619913.604549:0:5844:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788619913.604698:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+31:14908:x1875503571402624:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 8274us (8500us total) trans 4294967634 rc 0/0 00000100:00100000:1.0:1788619913.604707:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 875 00000020:00080000:1.0:1788619913.604711:0:5844:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1788619913.608637:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.608755:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571402880 00000100:00100000:1.0:1788619913.608766:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571402880 00002000:00010000:1.0:1788619913.608805:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1165723->18446744073709551615) 00002000:00100000:1.0:1788619913.608810:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c501c700 with x1875503571402880 ext(1165723->18446744073709551615) 00002000:00010000:1.0:1788619913.608821:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c501c700. 00000100:00100000:1.0:1788619913.608863:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 358 00000100:00100000:1.0:1788619913.608873:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571402880:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:1.0:1788619913.609098:0:6703:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 0 -> 1165723 00002000:00010000:1.0:1788619913.609103:0:6703:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619913 -> 1788619913 00002000:00010000:1.0:1788619913.609105:0:6703:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619913 -> 1788619911 00002000:00010000:1.0:1788619913.609107:0:6703:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619913 -> 1788619913 00002000:00010000:1.0:1788619913.609121:0:6703:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 135 -> 135 00000100:00100000:1.0:1788619913.609181:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571402880:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 309us (553us total) trans 4294967402 rc 0/0 00000100:00100000:1.0:1788619913.609189:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 358 00002000:00010000:1.0:1788619913.609193:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1165723->18446744073709551615) 00002000:00100000:1.0:1788619913.609210:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c501c700 with x1875503571402880 ext(1165723->18446744073709551615) 00002000:00010000:1.0:1788619913.609214:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c501c700. 00000100:00100000:2.0:1788619913.611976:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619913.612039:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571403008 00000100:00100000:3.0:1788619913.612048:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571403008 00000100:00100000:3.0:1788619913.612104:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 876 00000100:00100000:3.0:1788619913.612221:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+31:14908:x1875503571403008:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619913.612251:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619913.612274:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe536873460c lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1babe expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.612457:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368734613 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.612485:0:9417:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7c00/0x1cd7fe5368734605 lrc: 2/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.612491:0:9417:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368734613 lrc: 4/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x13/0x8 rrc: 7 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.612508:0:9417:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7c00/0x1cd7fe5368734605 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000020 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.612520:0:9417:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368734613 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x13/0x8 rrc: 7 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.612632:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7c00/0x1cd7fe5368734605 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000020 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.612641:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7c00/0x1cd7fe5368734605 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000020 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.612655:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7c00/0x1cd7fe5368734605 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000020 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.612661:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78ccca7c00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788619913.612734:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7c00/0x1cd7fe5368734605 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000020 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.612744:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7c00/0x1cd7fe5368734605 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000020 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.612759:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619913.612762:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788619913.612764:0:3303:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368734613 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x13/0x8 rrc: 7 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.612773:0:3303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368734613 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40290000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.612781:0:3303:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368734613 lrc: 4/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40290000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.612860:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7c00/0x1cd7fe5368734605 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000020 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.612867:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7c00/0x1cd7fe5368734605 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000020 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.612976:0:9417:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368734613 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.612983:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368734613 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788619913.613014:0:9417:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1165723 00000004:00200000:3.0:1788619913.613018:0:9417:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788619913.613037:0:9417:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368734613 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.613061:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe536873460c lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1babe expref: 32 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.613075:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368734613 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1babe expref: 31 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.613082:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619913.613089:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788619913.613092:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683de00, rc 0) 00000100:00100000:3.0:1788619913.613277:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+31:14908:x1875503571403008:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 1060us (1307us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619913.613286:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 876 00000100:00100000:2.0:1788619913.619725:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619913.619761:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.619911:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571403136 00000100:00100000:1.0:1788619913.619921:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571403136 00002000:00100000:1.0:1788619913.619972:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1167359) req@ffff8e77c5cd7480 x1875503571403136/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619913.619983:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5cd7480 with x1875503571403136 ext(1048576->1167359) 00002000:00010000:1.0:1788619913.619990:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5cd7480 00000100:00100000:1.0:1788619913.620023:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 359 00000100:00100000:1.0:1788619913.620033:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571403136:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619913.620449:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571403264 00000100:00100000:3.0:1788619913.620454:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571403264 00002000:00100000:3.0:1788619913.620481:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5cd6680 x1875503571403264/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619913.620489:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5cd6680 with x1875503571403264 00000100:00100000:3.0:1788619913.620516:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 502 00000100:00100000:3.0:1788619913.620523:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+31:2355:x1875503571403264:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619913.623992:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+31:2355:x1875503571403264:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 3459us (4220us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619913.624005:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 502 00002000:00100000:3.0:1788619913.624018:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5cd6680 x1875503571403264/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/440 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619913.624029:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5cd6680 with x1875503571403264 00000100:00100000:1.0:1788619913.626424:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571403136:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 6384us (6702us total) trans 0 rc 118784/118784 00000100:00100000:1.0:1788619913.626438:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 359 00002000:00100000:1.0:1788619913.626449:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1167359) req@ffff8e77c5cd7480 x1875503571403136/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/440 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 118784/118784 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619913.626460:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5cd7480 with x1875503571403136 ext(1048576->1167359) 00002000:00010000:1.0:1788619913.626465:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5cd7480 00000100:00100000:3.0:1788619913.630259:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619913.630357:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571403392 00000100:00100000:0.0:1788619913.630368:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571403392 00000100:00100000:0.0:1788619913.630426:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 877 00000100:00100000:0.0:1788619913.630437:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e6478380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+14:14908:x1875503571403392:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619913.630472:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619913.630495:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f600/0x1cd7fe536873461a lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bac5 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.630577:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368734621 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.630595:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368734621 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.630606:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368734621 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619913.630634:0:5846:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1165723 00000004:00200000:0.0:1788619913.630639:0:5846:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619913.630656:0:5846:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368734621 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.630693:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f600/0x1cd7fe536873461a lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bac5 expref: 16 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.630710:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368734621 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bac5 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.630718:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619913.630720:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619913.630724:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6e000, rc 0) 00000100:00100000:0.0:1788619913.630777:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e6478380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+15:14908:x1875503571403392:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 343us (527us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619913.630784:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 877 00000100:00100000:2.0:1788619913.633520:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.633616:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571403520 00000100:00100000:1.0:1788619913.633625:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571403520 00000100:00100000:1.0:1788619913.633687:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 878 00000100:00100000:1.0:1788619913.633697:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+15:2355:x1875503571403520:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619913.633716:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619913.633737:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7e00/0x1cd7fe5368734628 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bacc expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.633797:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06600/0x1cd7fe53687345c6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba8d expref: 30 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619913.633836:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d58e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589091072:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619913.634036:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963840 going to sleep for 11 seconds 00000100:00100000:3.0:1788619913.636279:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619913.636504:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571403648 00000100:00100000:0.0:1788619913.636511:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571403648 00000100:00100000:0.0:1788619913.636565:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 671 00000100:00100000:0.0:1788619913.636574:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e6479880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571403648:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619913.636590:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619913.636606:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005414d418/0x1cd7fe536873462f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bad3 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619913.636661:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6342680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589091200:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619913.636741:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:1.0:1788619913.636873:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d58e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589091072:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619913.636894:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 945165 00010000:00010000:1.0:1788619913.636909:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7e00/0x1cd7fe5368734628 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bacc expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.636925:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619913.636928:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619913.636933:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7e00/0x1cd7fe5368734628 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bacc expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.636947:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78ccca7e00, rc 0) 00000100:00100000:1.0:1788619913.636974:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+15:2355:x1875503571403520:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3279us (3465us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619913.636980:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 878 00002000:00010000:0.0:1788619913.639784:0:7654:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619911 -> 1788619913 00000100:00100000:0.0:1788619913.639815:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6342680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589091200:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619913.639864:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005414d418/0x1cd7fe536873462f lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bad3 expref: 7 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.639887:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005414d418/0x1cd7fe536873462f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bad3 expref: 7 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.639899:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6f200, rc 0) 00000100:00100000:0.0:1788619913.639954:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e6479880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571403648:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3379us (3681us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619913.639962:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 671 00000100:00100000:2.0:1788619913.643078:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619913.643127:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.643215:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571403904 00000100:00100000:1.0:1788619913.643224:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571403904 00000100:00100000:1.0:1788619913.643280:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 672 00000100:00100000:1.0:1788619913.643291:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571403904:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619913.643306:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619913.643321:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001eabdd3d/0x1cd7fe5368734636 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bae1 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619913.643412:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d58e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589091328:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619913.643479:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963840 going to sleep for 11 seconds 00000100:00100000:3.0:1788619913.643982:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571403776 00000100:00100000:3.0:1788619913.643987:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571403776 00000100:00100000:3.0:1788619913.644024:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 879 00000100:00100000:3.0:1788619913.644032:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+15:2355:x1875503571403776:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619913.644043:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619913.644055:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe536873463d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bada expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.644091:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06600/0x1cd7fe53687345c6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba8d expref: 30 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619913.644127:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e647a680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589091456:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619913.644173:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164900 going to sleep for 11 seconds 00000100:00100000:1.0:1788619913.646802:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d58e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589091328:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619913.646836:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001eabdd3d/0x1cd7fe5368734636 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bae1 expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.646872:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001eabdd3d/0x1cd7fe5368734636 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bae1 expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.646885:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78ccca7e00, rc 0) 00000100:00100000:1.0:1788619913.646939:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571403904:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3650us (3810us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619913.646948:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 672 00000100:00100000:3.0:1788619913.647421:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e647a680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589091456:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619913.647436:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 945165 00010000:00010000:3.0:1788619913.647447:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe536873463d lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bada expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.647460:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619913.647463:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619913.647468:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe536873463d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bada expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.647479:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683da00, rc 0) 00000100:00100000:3.0:1788619913.647533:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+15:2355:x1875503571403776:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3502us (4463us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619913.647540:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 879 00000100:00100000:2.0:1788619913.653927:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619913.654057:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571404160 00000100:00100000:3.0:1788619913.654066:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571404160 00000100:00100000:3.0:1788619913.654208:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 880 00000100:00100000:3.0:1788619913.654219:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+15:14908:x1875503571404160:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:3.0:1788619913.654274:0:11397:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619913.654300:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368734621 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bac5 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.654362:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619913.654368:0:11397:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788619913.654563:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368734644 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.654578:0:11397:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368734613 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1babe expref: 30 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.654586:0:11397:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368734644 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.654618:0:11397:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368734613 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1babe expref: 30 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.654632:0:11397:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368734613 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1babe expref: 30 pid: 9417 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619913.654645:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e647a680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589091584:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619913.654829:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164900 going to sleep for 11 seconds 00000100:00100000:0.0:1788619913.657183:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e647a680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589091584:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619913.657211:0:11397:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368734644 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619913.659217:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619913.659445:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571404288 00000100:00100000:0.0:1788619913.659455:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571404288 00010000:00010000:0.0:1788619913.659497:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368734613 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1babe expref: 31 pid: 9417 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619913.659542:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 296 00000100:00100000:0.0:1788619913.659553:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd4000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+31:2355:x1875503571404288:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619913.659567:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619913.659573:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368734613 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1babe expref: 31 pid: 9417 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.659586:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368734613 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1babe expref: 31 pid: 9417 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.659609:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368734613 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1babe expref: 31 pid: 9417 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.659626:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619913.659631:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368734644 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.659643:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368734644 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.659651:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368734644 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.659687:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619913.659759:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd4000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+30:2355:x1875503571404288:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 207us (556us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619913.659767:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 296 00010000:00010000:3.0:1788619913.660767:0:11397:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368734644 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.660778:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368734644 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788619913.661047:0:11397:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967635 00000004:00010000:3.0:1788619913.661096:0:11397:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 945165 -> 945165 00000004:00010000:3.0:1788619913.661100:0:11397:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619913 -> 1788619913 00000004:00010000:3.0:1788619913.661185:0:11397:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619913 -> 1788619913 00000004:00010000:3.0:1788619913.661187:0:11397:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619913 -> 1788619913 00000004:00010000:3.0:1788619913.661190:0:11397:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 646 -> 646 00000004:00010000:3.0:1788619913.661197:0:11397:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000003b72f1a1 reply state 00000000e33a98ba transno 4294967636 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368734644 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619913.661206:0:11397:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788619913.661294:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+17:14908:x1875503571404160:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 7075us (7450us total) trans 4294967636 rc 0/0 00000100:00100000:3.0:1788619913.661302:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 880 00000020:00080000:3.0:1788619913.661306:0:11397:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1788619913.665469:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00010000:00010000:2.0:1788619913.666056:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368734644 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.666069:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368734644 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.666077:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368734644 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.666083:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c683d400 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:2.0:1788619913.666087:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368734644 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.666096:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368734644 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.666165:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619913.666169:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788619913.666174:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368734644 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.666181:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368734644 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619913.666236:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571404416 00000100:00100000:1.0:1788619913.666246:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571404416 00000100:00100000:1.0:1788619913.666414:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 881 00000100:00100000:1.0:1788619913.666424:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e647bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+16:14908:x1875503571404416:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619913.666444:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619913.666461:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7e00/0x1cd7fe536873464b lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bae8 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.666486:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06600/0x1cd7fe53687345c6 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba8d expref: 29 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.666515:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7e00/0x1cd7fe536873464b lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bae8 expref: 17 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.666548:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06600/0x1cd7fe53687345c6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba8d expref: 29 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.666561:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06600/0x1cd7fe53687345c6 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba8d expref: 29 pid: 9417 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619913.666575:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d58e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589091712:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619913.666799:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963840 going to sleep for 11 seconds 00000100:00100000:1.0:1788619913.671060:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d58e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589091712:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619913.671096:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7e00/0x1cd7fe536873464b lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bae8 expref: 17 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.671113:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619913.671115:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619913.671131:0:5844:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7e00/0x1cd7fe536873464b lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bae8 expref: 17 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.671141:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78ccca7e00, rc 0) 00000100:00100000:1.0:1788619913.671311:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e647bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+17:14908:x1875503571404416:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4889us (5852us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619913.671321:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 881 00000100:00100000:2.0:1788619913.672428:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619913.672501:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571404544 00000100:00100000:3.0:1788619913.672509:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571404544 00010000:00010000:3.0:1788619913.672546:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06600/0x1cd7fe53687345c6 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba8d expref: 30 pid: 9417 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619913.672585:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 297 00000100:00100000:3.0:1788619913.672595:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+30:2355:x1875503571404544:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619913.672608:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619913.672632:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06600/0x1cd7fe53687345c6 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba8d expref: 30 pid: 9417 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.672644:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06600/0x1cd7fe53687345c6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba8d expref: 30 pid: 9417 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.672675:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06600/0x1cd7fe53687345c6 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ba8d expref: 30 pid: 9417 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.672708:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619913.672715:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7e00/0x1cd7fe536873464b lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bae8 expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.672725:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7e00/0x1cd7fe536873464b lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bae8 expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.672735:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7e00/0x1cd7fe536873464b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bae8 expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.672770:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7e00/0x1cd7fe536873464b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bae8 expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619913.672783:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e6479c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589091840:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619913.672859:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164900 going to sleep for 11 seconds 00000100:00100000:3.0:1788619913.677095:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.677270:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e6479c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589091840:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619913.677305:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619913.677385:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+29:2355:x1875503571404544:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 4791us (4971us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619913.677396:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 297 00000100:00100000:0.0:1788619913.678535:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571404672 00000100:00100000:0.0:1788619913.678546:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571404672 00000100:00100000:2.0:1788619913.678760:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 673 00000100:00100000:2.0:1788619913.678771:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e647aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:14908:x1875503571404672:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619913.678791:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619913.678807:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001e71691f/0x1cd7fe5368734652 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1baef expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.678916:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000244b3d37/0x1cd7fe53687344fb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9d7 expref: 14 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.678926:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=1s v=(6 8) 00010000:00010000:2.0:1788619913.678930:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(7 8) 00010000:00010000:2.0:1788619913.678935:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001e71691f/0x1cd7fe5368734652 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1baef expref: 7 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.678969:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000244b3d37/0x1cd7fe53687344fb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9d7 expref: 14 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.678982:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000244b3d37/0x1cd7fe53687344fb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9d7 expref: 14 pid: 6696 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619913.678995:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cd7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589091968:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619913.679072:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccc00 going to sleep for 11 seconds 00000100:00100000:3.0:1788619913.681858:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.682044:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571404800 00000100:00100000:1.0:1788619913.682052:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571404800 00002000:00100000:1.0:1788619913.682096:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1073152->1126399) req@ffff8e78e6479c00 x1875503571404800/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619913.682106:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78e6479c00 with x1875503571404800 ext(1073152->1126399) 00010000:00010000:1.0:1788619913.682115:0:6701:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000244b3d37/0x1cd7fe53687344fb lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9d7 expref: 15 pid: 6696 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.682136:0:6701:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000244b3d37/0x1cd7fe53687344fb lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9d7 expref: 15 pid: 6696 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788619913.682143:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff8e78e6479c00 00000100:00100000:1.0:1788619913.682178:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 13 00000100:00100000:1.0:1788619913.682187:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e6479c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571404800:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:2.0:1788619913.682452:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cd7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589091968:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619913.682480:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001e71691f/0x1cd7fe5368734652 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1baef expref: 7 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.682497:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa06a00, rc 0) 00000100:00100000:2.0:1788619913.682525:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e647aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:14908:x1875503571404672:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3756us (5439us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619913.682542:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 673 00000100:00100000:1.0:1788619913.685889:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e6479c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2354:x1875503571404800:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 3693us (4041us total) trans 4294967403 rc 0/0 00000100:00100000:1.0:1788619913.685902:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 13 00002000:00100000:1.0:1788619913.685915:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1073152->1126399) req@ffff8e78e6479c00 x1875503571404800/t4294967403(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/448 e 0 to 0 dl 1788619924 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619913.685925:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78e6479c00 with x1875503571404800 ext(1073152->1126399) 00010000:00010000:1.0:1788619913.685944:0:6701:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000244b3d37/0x1cd7fe53687344fb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9d7 expref: 17 pid: 6696 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.685960:0:6701:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000244b3d37/0x1cd7fe53687344fb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9d7 expref: 17 pid: 6696 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788619913.685968:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff8e78e6479c00 00000100:00100000:3.0:1788619913.689265:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619913.689603:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571404928 00000100:00100000:0.0:1788619913.689613:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571404928 00010000:00010000:0.0:1788619913.689651:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000244b3d37/0x1cd7fe53687344fb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9d7 expref: 17 pid: 6696 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619913.689707:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 298 00000100:00100000:0.0:1788619913.689718:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e647a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2354:x1875503571404928:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619913.689732:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619913.689755:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000244b3d37/0x1cd7fe53687344fb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9d7 expref: 17 pid: 6696 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.689769:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000244b3d37/0x1cd7fe53687344fb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9d7 expref: 17 pid: 6696 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.689794:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000244b3d37/0x1cd7fe53687344fb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1b9d7 expref: 17 pid: 6696 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.689810:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(7 8) 00010000:00010000:0.0:1788619913.689815:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(7 8) 00010000:00010000:0.0:1788619913.689824:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000001e71691f/0x1cd7fe5368734652 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1baef expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.689865:0:5831:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001e71691f/0x1cd7fe5368734652 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1baef expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.689914:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000001e71691f/0x1cd7fe5368734652 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1baef expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619913.689931:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6342680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5831:1875503589092096:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619913.690012:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:0.0:1788619913.692935:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6342680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5831:1875503589092096:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619913.692960:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619913.693013:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e647a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571404928:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3296us (3757us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619913.693024:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 298 00000100:00100000:3.0:1788619913.695013:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619913.695057:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.695210:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571405184 00000100:00100000:1.0:1788619913.695219:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571405184 00002000:00010000:1.0:1788619913.695258:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:1.0:1788619913.695263:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5c31500 with x1875503571405184 ext(0->18446744073709551615) 00002000:00010000:1.0:1788619913.695274:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5c31500. 00000100:00100000:1.0:1788619913.695305:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 360 00000100:00100000:1.0:1788619913.695314:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c31500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571405184:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:1.0:1788619913.695628:0:6702:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1165723 -> 0 00002000:00010000:1.0:1788619913.695632:0:6702:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619913 -> 1788619913 00002000:00010000:1.0:1788619913.695635:0:6702:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619913 -> 1788619911 00002000:00010000:1.0:1788619913.695637:0:6702:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619913 -> 1788619913 00002000:00010000:1.0:1788619913.695639:0:6702:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 135 -> 135 00000100:00100000:1.0:1788619913.695700:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c31500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+8:2354:x1875503571405184:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 386us (641us total) trans 4294967404 rc 0/0 00000100:00100000:1.0:1788619913.695707:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 360 00002000:00010000:1.0:1788619913.695711:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:1.0:1788619913.695713:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5c31500 with x1875503571405184 ext(0->18446744073709551615) 00002000:00010000:1.0:1788619913.695717:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5c31500. 00000100:00100000:0.0:1788619913.696121:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571405056 00000100:00100000:0.0:1788619913.696127:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571405056 00002000:00010000:0.0:1788619913.696149:0:5859:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:0.0:1788619913.696154:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e6478700 with x1875503571405056 00000100:00100000:0.0:1788619913.696172:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 503 00000100:00100000:0.0:1788619913.696179:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e6478700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+17:2354:x1875503571405056:12345-192.168.201.5@tcp:10:fsx.0 00000004:00010000:0.0:1788619913.696361:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 945165 -> 913332 00000004:00010000:0.0:1788619913.696365:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619913 -> 1788619913 00000004:00010000:0.0:1788619913.696368:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619913 -> 1788619913 00000004:00010000:0.0:1788619913.696371:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619913 -> 1788619913 00000004:00010000:0.0:1788619913.696386:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 646 -> 646 00000100:00100000:0.0:1788619913.696424:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e6478700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+18:2354:x1875503571405056:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 245us (1419us total) trans 4294967637 rc 0/0 00000100:00100000:0.0:1788619913.696432:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 503 00002000:00010000:0.0:1788619913.696435:0:5859:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:0.0:1788619913.696438:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e6478700 with x1875503571405056 00000100:00100000:3.0:1788619913.699450:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619913.699542:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571405312 00000100:00100000:2.0:1788619913.699551:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571405312 00000100:00100000:2.0:1788619913.699608:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 882 00000100:00100000:2.0:1788619913.699618:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c30380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+18:14908:x1875503571405312:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619913.699640:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619913.699660:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07400/0x1cd7fe5368734659 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1baf6 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.699755:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06c00/0x1cd7fe5368734660 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.699769:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06c00/0x1cd7fe5368734660 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.699779:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06c00/0x1cd7fe5368734660 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788619913.699802:0:9417:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 913332 00000004:00200000:2.0:1788619913.699806:0:9417:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788619913.699823:0:9417:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06c00/0x1cd7fe5368734660 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.699923:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07400/0x1cd7fe5368734659 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1baf6 expref: 20 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.699940:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06c00/0x1cd7fe5368734660 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1baf6 expref: 19 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.699964:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619913.699967:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788619913.699971:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa06c00, rc 0) 00000100:00100000:2.0:1788619913.700061:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c30380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+19:14908:x1875503571405312:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 444us (620us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619913.700068:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 882 00000100:00100000:2.0:1788619913.703776:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619913.703921:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571405440 00000100:00100000:3.0:1788619913.703931:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571405440 00000100:00100000:3.0:1788619913.704035:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 504 00000100:00100000:3.0:1788619913.704046:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+19:2355:x1875503571405440:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:2.0:1788619913.708273:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 913332 -> 913332 00000004:00010000:2.0:1788619913.708284:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619913 -> 1788619913 00000004:00010000:2.0:1788619913.708287:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619913 -> 1788619913 00000004:00010000:2.0:1788619913.708289:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619913 -> 1788619913 00000004:00010000:2.0:1788619913.708292:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 646 -> 646 00000100:00100000:2.0:1788619913.708395:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+20:2355:x1875503571405440:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 4346us (4623us total) trans 4294967638 rc 0/0 00000100:00100000:2.0:1788619913.708406:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 504 00000100:00100000:2.0:1788619913.713637:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.713782:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571405568 00000100:00100000:1.0:1788619913.713806:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571405568 00000100:00100000:1.0:1788619913.713969:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 505 00000100:00100000:1.0:1788619913.713980:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+20:2354:x1875503571405568:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:1.0:1788619913.714242:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 913332 -> 913332 00000004:00010000:1.0:1788619913.714246:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619913 -> 1788619913 00000004:00010000:1.0:1788619913.714249:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619913 -> 1788619913 00000004:00010000:1.0:1788619913.714252:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619913 -> 1788619913 00000004:00010000:1.0:1788619913.714254:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 646 -> 646 00000100:00100000:1.0:1788619913.714364:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+21:2354:x1875503571405568:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 349us (699us total) trans 4294967639 rc 0/0 00000100:00100000:1.0:1788619913.714372:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 505 00000100:00100000:3.0:1788619913.718118:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619913.718308:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571405696 00000100:00100000:0.0:1788619913.718319:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571405696 00000100:00100000:0.0:1788619913.718406:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 883 00000100:00100000:0.0:1788619913.718416:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c32d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+29:14908:x1875503571405696:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619913.718437:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619913.718458:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f400/0x1cd7fe5368734667 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bafd expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.718489:0:11397:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7e00/0x1cd7fe536873464b lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bae8 expref: 20 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.718499:0:11397:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f400/0x1cd7fe5368734667 lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bafd expref: 30 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.718533:0:11397:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7e00/0x1cd7fe536873464b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bae8 expref: 20 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.718545:0:11397:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7e00/0x1cd7fe536873464b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bae8 expref: 20 pid: 5844 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619913.718558:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6342680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589092224:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619913.718651:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:2.0:1788619913.721419:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.721512:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571405824 00000100:00100000:1.0:1788619913.721522:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571405824 00010000:00010000:1.0:1788619913.721558:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7e00/0x1cd7fe536873464b lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bae8 expref: 21 pid: 5844 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619913.721599:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 299 00000100:00100000:1.0:1788619913.721609:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8021450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+21:2354:x1875503571405824:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619913.721623:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619913.721651:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7e00/0x1cd7fe536873464b lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bae8 expref: 21 pid: 5844 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.721665:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7e00/0x1cd7fe536873464b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bae8 expref: 21 pid: 5844 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.721687:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619913.721705:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f400/0x1cd7fe5368734667 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bafd expref: 30 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.721716:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f400/0x1cd7fe5368734667 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bafd expref: 30 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.721724:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f400/0x1cd7fe5368734667 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bafd expref: 30 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.721758:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f400/0x1cd7fe5368734667 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bafd expref: 30 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619913.721771:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d58e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589092352:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619913.722258:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963840 going to sleep for 11 seconds 00010000:00010000:0.0:1788619913.722420:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7e00/0x1cd7fe536873464b lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bae8 expref: 21 pid: 5844 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619913.722437:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6342680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589092224:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619913.722461:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f400/0x1cd7fe5368734667 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bafd expref: 30 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.722474:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619913.722476:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619913.722479:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6f400, rc 0) 00000100:00100000:0.0:1788619913.722523:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c32d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+30:14908:x1875503571405696:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4108us (4424us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619913.722531:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 883 00000100:00100000:1.0:1788619913.727276:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d58e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589092352:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619913.727304:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619913.727428:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8021450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+20:2354:x1875503571405824:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 5819us (6016us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619913.727441:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 299 00000100:00100000:2.0:1788619913.729379:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619913.729423:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571405952 00000100:00100000:0.0:1788619913.729433:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571405952 00002000:00100000:0.0:1788619913.729480:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5cd4a80 x1875503571405952/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619913.729492:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5cd4a80 with x1875503571405952 00000100:00100000:0.0:1788619913.729527:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 506 00000100:00100000:0.0:1788619913.729537:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+30:2354:x1875503571405952:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619913.740275:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+30:2354:x1875503571405952:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 10728us (10890us total) trans 0 rc 913408/913408 00000100:00100000:2.0:1788619913.740291:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 506 00002000:00100000:2.0:1788619913.740304:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5cd4a80 x1875503571405952/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/440 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 913408/913408 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619913.740315:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5cd4a80 with x1875503571405952 00000100:00100000:3.0:1788619913.744585:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.744697:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571406080 00000100:00100000:1.0:1788619913.744708:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571406080 00000100:00100000:1.0:1788619913.744768:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 884 00000100:00100000:1.0:1788619913.744778:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c32a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+20:2355:x1875503571406080:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619913.744805:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619913.744825:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7000/0x1cd7fe536873466e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bb04 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.744887:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7000/0x1cd7fe536873466e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb04 expref: 21 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788619913.744899:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 913332 00010000:00010000:1.0:1788619913.744907:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7000/0x1cd7fe536873466e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb04 expref: 21 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.744920:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619913.744924:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619913.744928:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78ccca7000, rc 0) 00000100:00100000:1.0:1788619913.744988:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c32a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+21:2355:x1875503571406080:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 212us (414us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619913.744994:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 884 00000100:00100000:3.0:1788619913.748815:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.748935:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571406208 00000100:00100000:1.0:1788619913.748944:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571406208 00000100:00100000:1.0:1788619913.749007:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 885 00000100:00100000:1.0:1788619913.749017:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c33800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+21:14908:x1875503571406208:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619913.749037:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619913.749040:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:1.0:1788619913.749082:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7000/0x1cd7fe536873466e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb04 expref: 21 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.749102:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619913.749108:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788619913.749120:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6200/0x1cd7fe5368734675 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bb0b expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.749164:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f400/0x1cd7fe5368734667 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bafd expref: 29 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.749172:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6200/0x1cd7fe5368734675 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb0b expref: 21 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.749198:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f400/0x1cd7fe5368734667 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bafd expref: 29 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.749212:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f400/0x1cd7fe5368734667 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bafd expref: 29 pid: 11397 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619913.749224:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d58e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589092480:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619913.749304:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963a80 going to sleep for 11 seconds 00000100:00100000:1.0:1788619913.752478:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d58e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589092480:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619913.752515:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6200/0x1cd7fe5368734675 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb0b expref: 21 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.752532:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619913.752535:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619913.752540:0:5845:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6200/0x1cd7fe5368734675 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb0b expref: 21 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.752550:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78ccca6200, rc 0) 00000100:00100000:1.0:1788619913.752609:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c33800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+21:14908:x1875503571406208:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3593us (3801us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619913.752632:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 885 00000100:00100000:2.0:1788619913.757437:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.757716:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571406336 00000100:00100000:1.0:1788619913.757726:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571406336 00010000:00010000:1.0:1788619913.757765:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f400/0x1cd7fe5368734667 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bafd expref: 30 pid: 11397 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619913.757805:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 300 00000100:00100000:1.0:1788619913.757815:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+30:2354:x1875503571406336:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619913.757829:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619913.757863:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f400/0x1cd7fe5368734667 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bafd expref: 30 pid: 11397 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.757877:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f400/0x1cd7fe5368734667 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bafd expref: 30 pid: 11397 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.757900:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f400/0x1cd7fe5368734667 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bafd expref: 30 pid: 11397 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.757917:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619913.757923:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6200/0x1cd7fe5368734675 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb0b expref: 20 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.757933:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6200/0x1cd7fe5368734675 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb0b expref: 20 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.757942:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6200/0x1cd7fe5368734675 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb0b expref: 20 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.757989:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6200/0x1cd7fe5368734675 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb0b expref: 20 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619913.758002:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d58e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589092608:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619913.758083:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963a80 going to sleep for 11 seconds 00000100:00100000:3.0:1788619913.761633:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.761734:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d58e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589092608:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619913.761755:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619913.761806:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+29:2354:x1875503571406336:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3992us (4379us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619913.761816:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 300 00000100:00100000:2.0:1788619913.761920:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571406464 00000100:00100000:2.0:1788619913.761928:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571406464 00002000:00100000:2.0:1788619913.761970:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5c30000 x1875503571406464/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619913.761981:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5c30000 with x1875503571406464 00000100:00100000:2.0:1788619913.762011:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 507 00000100:00100000:2.0:1788619913.762020:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c30000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+21:2354:x1875503571406464:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619913.762250:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c30000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+21:2354:x1875503571406464:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 230us (625us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619913.762257:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 507 00002000:00100000:2.0:1788619913.762263:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5c30000 x1875503571406464/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:9/0 lens 488/4536 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619913.762271:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5c30000 with x1875503571406464 00000100:00100000:3.0:1788619913.767067:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619913.767227:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571406592 00000100:00100000:2.0:1788619913.767239:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571406592 00002000:00100000:2.0:1788619913.767294:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5c31180 x1875503571406592/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619913.767306:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5c31180 with x1875503571406592 00000100:00100000:2.0:1788619913.767440:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 508 00000100:00100000:2.0:1788619913.767450:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c31180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+21:2352:x1875503571406592:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619913.767735:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c31180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+21:2352:x1875503571406592:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 285us (678us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619913.767742:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 508 00002000:00100000:2.0:1788619913.767749:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5c31180 x1875503571406592/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:9/0 lens 488/4536 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619913.767774:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5c31180 with x1875503571406592 00000100:00100000:3.0:1788619913.771895:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.772133:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571406720 00000100:00100000:1.0:1788619913.772144:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571406720 00000100:00100000:1.0:1788619913.772211:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 509 00000100:00100000:1.0:1788619913.772221:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c31f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+21:2353:x1875503571406720:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:1.0:1788619913.772500:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 913332 -> 913332 00000004:00010000:1.0:1788619913.772504:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619913 -> 1788619913 00000004:00010000:1.0:1788619913.772506:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619913 -> 1788619913 00000004:00010000:1.0:1788619913.772509:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619913 -> 1788619913 00000004:00010000:1.0:1788619913.772511:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 646 -> 646 00000100:00100000:1.0:1788619913.772576:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c31f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+22:2353:x1875503571406720:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 357us (691us total) trans 4294967640 rc 0/0 00000100:00100000:1.0:1788619913.772584:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 509 00000100:00100000:0.0:1788619913.775870:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619913.775949:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571406848 00000100:00100000:3.0:1788619913.775959:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571406848 00000100:00100000:3.0:1788619913.776017:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 886 00000100:00100000:3.0:1788619913.776027:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6342680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+29:14908:x1875503571406848:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619913.776047:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619913.776069:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe536873467c lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bb12 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.776102:0:11397:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6200/0x1cd7fe5368734675 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb0b expref: 21 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.776121:0:11397:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe536873467c lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb12 expref: 30 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.776159:0:11397:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6200/0x1cd7fe5368734675 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb0b expref: 21 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.776173:0:11397:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6200/0x1cd7fe5368734675 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb0b expref: 21 pid: 5845 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619913.776186:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c31c00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589092736:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619913.776301:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164240 going to sleep for 11 seconds 00000100:00100000:0.0:1788619913.778143:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c31c00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589092736:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619913.778174:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe536873467c lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb12 expref: 30 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.778208:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619913.778211:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619913.778216:0:11397:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe536873467c lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb12 expref: 30 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.778226:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d000, rc 0) 00000100:00100000:0.0:1788619913.778278:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6342680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+30:14908:x1875503571406848:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2251us (2416us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619913.778286:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 886 00000100:00100000:2.0:1788619913.780465:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619913.780638:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571406976 00000100:00100000:0.0:1788619913.780647:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571406976 00002000:00100000:0.0:1788619913.780704:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5cd4000 x1875503571406976/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619913.780714:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5cd4000 with x1875503571406976 00010000:00010000:0.0:1788619913.780722:0:5857:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6200/0x1cd7fe5368734675 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb0b expref: 22 pid: 5845 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.780737:0:5857:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6200/0x1cd7fe5368734675 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb0b expref: 22 pid: 5845 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788619913.780744:0:5857:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e77c5cd4000 00000100:00100000:0.0:1788619913.780779:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 21 00000100:00100000:0.0:1788619913.780788:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+22:2355:x1875503571406976:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:1.0:1788619913.784880:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 913332 -> 913332 00000004:00010000:1.0:1788619913.784889:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619913 -> 1788619913 00000004:00010000:1.0:1788619913.784903:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619913 -> 1788619913 00000004:00010000:1.0:1788619913.784905:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619913 -> 1788619913 00000004:00010000:1.0:1788619913.784907:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 646 -> 646 00000100:00100000:1.0:1788619913.785029:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+24:2355:x1875503571406976:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 4237us (4569us total) trans 4294967641 rc 0/0 00000100:00100000:1.0:1788619913.785042:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 21 00002000:00100000:1.0:1788619913.785054:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5cd4000 x1875503571406976/t4294967641(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:9/0 lens 488/448 e 0 to 0 dl 1788619924 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619913.785065:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5cd4000 with x1875503571406976 00010000:00010000:1.0:1788619913.785074:0:5857:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6200/0x1cd7fe5368734675 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb0b expref: 24 pid: 5845 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.785089:0:5857:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6200/0x1cd7fe5368734675 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb0b expref: 24 pid: 5845 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788619913.785097:0:5857:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e77c5cd4000 00000100:00100000:3.0:1788619913.787645:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.787733:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571407104 00000100:00100000:1.0:1788619913.787743:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571407104 00010000:00010000:1.0:1788619913.787781:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6200/0x1cd7fe5368734675 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb0b expref: 24 pid: 5845 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619913.787823:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 301 00000100:00100000:1.0:1788619913.787832:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c31880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+24:2354:x1875503571407104:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619913.787855:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619913.787880:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6200/0x1cd7fe5368734675 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb0b expref: 24 pid: 5845 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.787920:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6200/0x1cd7fe5368734675 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb0b expref: 24 pid: 5845 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.787940:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6200/0x1cd7fe5368734675 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb0b expref: 24 pid: 5845 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.787957:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619913.787963:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe536873467c lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb12 expref: 29 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.787975:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe536873467c lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb12 expref: 29 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.787984:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe536873467c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb12 expref: 29 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.788023:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe536873467c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb12 expref: 29 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619913.788037:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d58e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589092864:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619913.788445:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963a80 going to sleep for 11 seconds 00000100:00100000:1.0:1788619913.790610:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d58e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589092864:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619913.790636:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619913.790685:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c31880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+23:2354:x1875503571407104:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 2854us (3049us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619913.790710:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 301 00000100:00100000:2.0:1788619913.792543:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619913.792607:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571407232 00000100:00100000:3.0:1788619913.792614:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571407232 00000100:00100000:3.0:1788619913.792691:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 510 00000100:00100000:3.0:1788619913.792701:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+30:2354:x1875503571407232:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:3.0:1788619913.792942:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 913332 -> 913332 00000004:00010000:3.0:1788619913.792947:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619913 -> 1788619913 00000004:00010000:3.0:1788619913.792949:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619913 -> 1788619913 00000004:00010000:3.0:1788619913.792952:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619913 -> 1788619913 00000004:00010000:3.0:1788619913.792954:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 646 -> 646 00000100:00100000:3.0:1788619913.793043:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+31:2354:x1875503571407232:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 344us (506us total) trans 4294967642 rc 0/0 00000100:00100000:3.0:1788619913.793050:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 510 00000100:00100000:2.0:1788619913.795852:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619913.796020:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571407360 00000100:00100000:3.0:1788619913.796029:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571407360 00000100:00100000:3.0:1788619913.796190:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 887 00000100:00100000:3.0:1788619913.796202:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c77e3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+23:2354:x1875503571407360:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619913.796226:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619913.796246:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368734683 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bb19 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.796282:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe536873467c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb12 expref: 30 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619913.796352:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c32680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589092992:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619913.796446:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164d80 going to sleep for 11 seconds 00000100:00100000:3.0:1788619913.798607:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c32680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589092992:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619913.798632:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 913332 00010000:00010000:3.0:1788619913.798650:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368734683 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb19 expref: 24 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.798730:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619913.798734:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619913.798741:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368734683 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb19 expref: 24 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.798758:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683dc00, rc 0) 00000100:00100000:3.0:1788619913.798852:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c77e3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+23:2354:x1875503571407360:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2653us (3020us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619913.798860:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 887 00000100:00100000:2.0:1788619913.802292:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619913.802383:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571407488 00000100:00100000:3.0:1788619913.802393:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571407488 00000100:00100000:3.0:1788619913.802450:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 888 00000100:00100000:3.0:1788619913.802461:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c77e0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+23:2354:x1875503571407488:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619913.802481:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619913.802500:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe536873468a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bb20 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.802537:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe536873467c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb12 expref: 30 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619913.802574:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c32680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589093120:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619913.804942:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164d80 going to sleep for 11 seconds 00000100:00100000:3.0:1788619913.805739:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c32680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589093120:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619913.805763:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 913332 00010000:00010000:3.0:1788619913.805782:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe536873468a lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb20 expref: 24 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.805802:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619913.805805:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619913.805811:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe536873468a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb20 expref: 24 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.805825:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683da00, rc 0) 00000100:00100000:3.0:1788619913.805929:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c77e0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+23:2354:x1875503571407488:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3469us (3646us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619913.805937:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 888 00000100:00100000:0.0:1788619913.809812:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619913.809966:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571407616 00000100:00100000:3.0:1788619913.809975:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571407616 00000100:00100000:3.0:1788619913.810064:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 889 00000100:00100000:3.0:1788619913.810075:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6343480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+23:2354:x1875503571407616:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619913.810094:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619913.810177:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368734691 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bb27 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.810217:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe536873467c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb12 expref: 30 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619913.810261:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c32680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589093248:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619913.810391:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164d80 going to sleep for 11 seconds 00000100:00100000:3.0:1788619913.812804:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c32680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589093248:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619913.812829:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 913332 00010000:00010000:3.0:1788619913.812857:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368734691 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb27 expref: 24 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.812879:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619913.812882:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619913.812888:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368734691 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb27 expref: 24 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.813049:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d200, rc 0) 00000100:00100000:3.0:1788619913.813200:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6343480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+23:2354:x1875503571407616:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3125us (3394us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619913.813209:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 889 00000100:00100000:2.0:1788619913.815946:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619913.816027:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571407744 00000100:00100000:3.0:1788619913.816036:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571407744 00000100:00100000:3.0:1788619913.816093:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 890 00000100:00100000:3.0:1788619913.816103:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c77e2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+23:2354:x1875503571407744:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619913.816129:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619913.816148:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368734698 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bb2e expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.816182:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe536873467c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb12 expref: 30 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619913.816221:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c32680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589093376:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619913.816320:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164d80 going to sleep for 11 seconds 00000100:00100000:3.0:1788619913.819101:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c32680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589093376:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619913.819139:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 913332 00010000:00010000:3.0:1788619913.819160:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368734698 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb2e expref: 24 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.819179:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619913.819182:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619913.819187:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368734698 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb2e expref: 24 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.819200:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683dc00, rc 0) 00000100:00100000:3.0:1788619913.819258:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c77e2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+23:2354:x1875503571407744:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3156us (3320us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619913.819267:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 890 00000100:00100000:1.0:1788619913.822378:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619913.822482:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571407872 00000100:00100000:2.0:1788619913.822493:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571407872 00000100:00100000:2.0:1788619913.822551:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 674 00000100:00100000:2.0:1788619913.822560:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d58e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571407872:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619913.822579:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619913.822598:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000855993/0x1cd7fe536873469f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bb35 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619913.822672:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c77e1c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589093504:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619913.822757:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccc00 going to sleep for 11 seconds 00002000:00010000:2.0:1788619913.825954:0:6698:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619911 -> 1788619913 00000100:00100000:2.0:1788619913.825984:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c77e1c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589093504:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619913.826019:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000000855993/0x1cd7fe536873469f lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb35 expref: 17 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.826044:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000855993/0x1cd7fe536873469f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb35 expref: 17 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.826076:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079c00, rc 0) 00000100:00100000:2.0:1788619913.826148:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d58e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571407872:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3589us (3829us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619913.826159:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 674 00000100:00100000:2.0:1788619913.827721:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c33b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3316:1875503589093760:0@lo:400:kworker.0 00000100:00100000:2.0:1788619913.827783:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788619913.827820:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b12d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589094272:0@lo:400:kworker.0 00000100:00100000:2.0:1788619913.827855:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788619913.827878:0:3316:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1788619913.827883:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c31c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3316:1875503589093888:0@lo:400:kworker.0 00000100:00100000:2.0:1788619913.827904:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619913.827966:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b10a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589094016:0@lo:400:kworker.0 00000100:00100000:1.0:1788619913.827993:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619913.828018:0:3318:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:1.0:1788619913.828023:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c32680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:3318:1875503589093632:0@lo:400:kworker.0 00000100:00100000:1.0:1788619913.828050:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619913.828078:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b11500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3318:1875503589094144:0@lo:400:kworker.0 00000100:00100000:1.0:1788619913.828096:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619913.828131:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589093632 00000100:00100000:1.0:1788619913.828138:0:5836:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589093632 00000100:00100000:1.0:1788619913.828182:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 288 00000100:00100000:1.0:1788619913.828190:0:5836:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5ce3850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+19:3318:x1875503589093632:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788619913.828237:0:5836:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5ce3850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+19:3318:x1875503589093632:12345-0@lo:400:kworker.0 Request processed in 47us (186us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619913.828244:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 288 00000100:00100000:1.0:1788619913.828284:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c32680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:3318:1875503589093632:0@lo:400:kworker.0 00000100:00100000:0.0:1788619913.828370:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589094272 00000100:00100000:0.0:1788619913.828386:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589094272 00000100:00100000:0.0:1788619913.828434:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589094016 00000100:00100000:0.0:1788619913.828438:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589094016 00000100:00100000:0.0:1788619913.828458:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 124 00000100:00100000:0.0:1788619913.828465:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c77e3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503589094272:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788619913.828499:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c77e3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503589094272:12345-0@lo:400:kworker.0 Request processed in 35us (643us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619913.828505:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 124 00000100:00100000:0.0:1788619913.828517:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 125 00000100:00100000:0.0:1788619913.828522:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589094016:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788619913.828548:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589094016:12345-0@lo:400:kworker.0 Request processed in 27us (555us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619913.828553:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 125 00000100:00100000:0.0:1788619913.828572:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589093760 00000100:00100000:0.0:1788619913.828575:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589093760 00000100:00100000:0.0:1788619913.828594:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589093888 00000100:00100000:0.0:1788619913.828596:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589093888 00000100:00100000:0.0:1788619913.828614:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589094144 00000100:00100000:0.0:1788619913.828616:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589094144 00000100:00100000:0.0:1788619913.828630:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 183 00000100:00100000:0.0:1788619913.828635:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c77e1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3316:x1875503589093760:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788619913.828664:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c77e1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3316:x1875503589093760:12345-0@lo:400:kworker.0 Request processed in 28us (882us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619913.828669:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 183 00000100:00100000:0.0:1788619913.828680:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 184 00000100:00100000:0.0:1788619913.828685:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c77e0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3316:x1875503589093888:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788619913.828708:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c77e0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3316:x1875503589093888:12345-0@lo:400:kworker.0 Request processed in 24us (803us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619913.828722:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 184 00000100:00100000:0.0:1788619913.828733:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 185 00000100:00100000:0.0:1788619913.828738:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d59180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3318:x1875503589094144:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788619913.828763:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d59180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3318:x1875503589094144:12345-0@lo:400:kworker.0 Request processed in 26us (666us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619913.828767:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 185 00000100:00100000:2.0:1788619913.828879:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c31c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3316:1875503589093888:0@lo:400:kworker.0 00000100:00100000:2.0:1788619913.828890:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c33b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3316:1875503589093760:0@lo:400:kworker.0 00000100:00100000:2.0:1788619913.828898:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b12d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589094272:0@lo:400:kworker.0 00000100:00080000:1.0:1788619913.829238:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1788619913.829243:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1788619913.829245:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1788619913.829264:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b11500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3318:1875503589094144:0@lo:400:kworker.0 00000100:00100000:1.0:1788619913.829276:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b10a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589094016:0@lo:400:kworker.0 00000100:00100000:3.0:1788619913.831890:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.831982:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571408000 00000100:00100000:1.0:1788619913.831992:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571408000 00002000:00100000:1.0:1788619913.832053:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b10700 x1875503571408000/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619913.832064:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b10700 with x1875503571408000 00000100:00100000:1.0:1788619913.832099:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 511 00000100:00100000:1.0:1788619913.832116:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b10700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+31:2354:x1875503571408000:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619913.832351:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b10700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+31:2354:x1875503571408000:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 211us (443us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619913.832359:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 511 00002000:00100000:1.0:1788619913.832366:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b10700 x1875503571408000/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/4536 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619913.832374:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b10700 with x1875503571408000 00000100:00100000:1.0:1788619913.835797:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619913.836243:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571408128 00000100:00100000:0.0:1788619913.836253:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571408128 00002000:00100000:0.0:1788619913.836305:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5d58a80 x1875503571408128/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619913.836317:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5d58a80 with x1875503571408128 00000100:00100000:0.0:1788619913.836458:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 512 00000100:00100000:0.0:1788619913.836469:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d58a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+31:2353:x1875503571408128:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619913.836703:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d58a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+31:2353:x1875503571408128:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 236us (914us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619913.836710:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 512 00002000:00100000:0.0:1788619913.836716:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5d58a80 x1875503571408128/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/4536 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619913.836723:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5d58a80 with x1875503571408128 00000100:00100000:3.0:1788619913.839873:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.840319:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571408256 00000100:00100000:1.0:1788619913.840343:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571408256 00002000:00100000:1.0:1788619913.840389:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b11880 x1875503571408256/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619913.840400:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b11880 with x1875503571408256 00000100:00100000:1.0:1788619913.840425:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 513 00000100:00100000:1.0:1788619913.840434:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b11880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+31:2352:x1875503571408256:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619913.840797:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b11880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+31:2352:x1875503571408256:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 362us (931us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619913.840805:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 513 00002000:00100000:1.0:1788619913.840812:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b11880 x1875503571408256/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/4536 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619913.840833:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b11880 with x1875503571408256 00000100:00100000:3.0:1788619913.843980:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.844296:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571408384 00000100:00100000:1.0:1788619913.844306:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571408384 00002000:00100000:1.0:1788619913.844381:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b10e00 x1875503571408384/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619913.844392:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b10e00 with x1875503571408384 00000100:00100000:1.0:1788619913.844416:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 514 00000100:00100000:1.0:1788619913.844425:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b10e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+31:2352:x1875503571408384:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619913.844661:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b10e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+31:2352:x1875503571408384:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 236us (690us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619913.844676:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 514 00002000:00100000:1.0:1788619913.844682:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b10e00 x1875503571408384/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/4536 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619913.844689:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b10e00 with x1875503571408384 00000100:00100000:3.0:1788619913.848396:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619913.848614:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571408512 00000100:00100000:2.0:1788619913.848625:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571408512 00002000:00100000:2.0:1788619913.848740:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b12a00 x1875503571408512/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619913.848750:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b12a00 with x1875503571408512 00000100:00100000:2.0:1788619913.848836:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 515 00000100:00100000:2.0:1788619913.848877:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b12a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+31:2354:x1875503571408512:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619913.849096:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b12a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+31:2354:x1875503571408512:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 220us (707us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619913.849565:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 515 00002000:00100000:2.0:1788619913.849573:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b12a00 x1875503571408512/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/4536 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619913.849581:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b12a00 with x1875503571408512 00000100:00100000:2.0:1788619913.853059:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619913.853150:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571408640 00000100:00100000:0.0:1788619913.853160:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571408640 00002000:00100000:0.0:1788619913.853206:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c77e1180 x1875503571408640/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619913.853217:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c77e1180 with x1875503571408640 00000100:00100000:0.0:1788619913.853267:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 516 00000100:00100000:0.0:1788619913.853276:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c77e1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+31:2354:x1875503571408640:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619913.853518:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c77e1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+31:2354:x1875503571408640:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 243us (469us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619913.853526:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 516 00002000:00100000:0.0:1788619913.853533:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c77e1180 x1875503571408640/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/4536 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619913.853541:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c77e1180 with x1875503571408640 00000100:00100000:3.0:1788619913.856755:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619913.856863:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571408768 00000100:00100000:2.0:1788619913.856872:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571408768 00002000:00100000:2.0:1788619913.856921:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b10000 x1875503571408768/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619913.856932:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b10000 with x1875503571408768 00000100:00100000:2.0:1788619913.856967:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 517 00000100:00100000:2.0:1788619913.856977:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b10000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+31:2355:x1875503571408768:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619913.857187:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b10000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+31:2355:x1875503571408768:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 210us (439us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619913.857194:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 517 00002000:00100000:2.0:1788619913.857201:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b10000 x1875503571408768/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/4536 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619913.857208:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b10000 with x1875503571408768 00000100:00100000:3.0:1788619913.860043:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.860148:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571408896 00000100:00100000:1.0:1788619913.860157:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571408896 00002000:00100000:1.0:1788619913.860204:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b11f80 x1875503571408896/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619913.860215:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b11f80 with x1875503571408896 00000100:00100000:1.0:1788619913.860252:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 518 00000100:00100000:1.0:1788619913.860261:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b11f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+31:2354:x1875503571408896:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619913.860488:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b11f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+31:2354:x1875503571408896:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 227us (454us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619913.860494:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 518 00002000:00100000:1.0:1788619913.860514:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b11f80 x1875503571408896/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/4536 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619913.860522:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b11f80 with x1875503571408896 00000100:00100000:3.0:1788619913.863952:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619913.864246:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571409024 00000100:00100000:0.0:1788619913.864255:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571409024 00002000:00100000:0.0:1788619913.864304:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b13b80 x1875503571409024/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619913.864314:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b13b80 with x1875503571409024 00000100:00100000:0.0:1788619913.864474:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 519 00000100:00100000:0.0:1788619913.864484:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b13b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+31:2355:x1875503571409024:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619913.864700:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b13b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+31:2355:x1875503571409024:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 216us (755us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619913.864708:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 519 00002000:00100000:0.0:1788619913.864716:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b13b80 x1875503571409024/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/4536 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619913.864723:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b13b80 with x1875503571409024 00000100:00100000:3.0:1788619913.867363:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.868110:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571409152 00000100:00100000:1.0:1788619913.868120:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571409152 00002000:00100000:1.0:1788619913.868169:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b10a80 x1875503571409152/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619913.868179:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b10a80 with x1875503571409152 00000100:00100000:1.0:1788619913.868202:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 520 00000100:00100000:1.0:1788619913.868212:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b10a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+31:2354:x1875503571409152:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619913.868430:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b10a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+31:2354:x1875503571409152:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 219us (1101us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619913.868438:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 520 00002000:00100000:1.0:1788619913.868445:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b10a80 x1875503571409152/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/4536 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619913.868452:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b10a80 with x1875503571409152 00000100:00100000:3.0:1788619913.871057:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619913.871205:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571409280 00000100:00100000:0.0:1788619913.871216:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571409280 00002000:00100000:0.0:1788619913.871266:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b11500 x1875503571409280/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619913.871296:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b11500 with x1875503571409280 00000100:00100000:0.0:1788619913.871344:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 521 00000100:00100000:0.0:1788619913.871353:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b11500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+31:2353:x1875503571409280:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619913.871535:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b11500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+31:2353:x1875503571409280:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 182us (487us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619913.871542:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 521 00002000:00100000:0.0:1788619913.871548:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b11500 x1875503571409280/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/4536 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619913.871555:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b11500 with x1875503571409280 00000100:00100000:3.0:1788619913.874429:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619913.874765:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571409408 00000100:00100000:2.0:1788619913.874777:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571409408 00002000:00100000:2.0:1788619913.874833:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b12d80 x1875503571409408/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619913.874853:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b12d80 with x1875503571409408 00000100:00100000:2.0:1788619913.874898:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 522 00000100:00100000:2.0:1788619913.874925:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b12d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+31:2355:x1875503571409408:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619913.875137:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b12d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+31:2355:x1875503571409408:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 213us (716us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619913.875144:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 522 00002000:00100000:2.0:1788619913.875151:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b12d80 x1875503571409408/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/4536 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619913.875158:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b12d80 with x1875503571409408 00000100:00100000:3.0:1788619913.879674:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619913.879816:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571409536 00000100:00100000:2.0:1788619913.879827:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571409536 00002000:00100000:2.0:1788619913.879912:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5773b80 x1875503571409536/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619913.879922:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5773b80 with x1875503571409536 00000100:00100000:2.0:1788619913.879962:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 523 00000100:00100000:2.0:1788619913.879973:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5773b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+31:2354:x1875503571409536:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619913.880243:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5773b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+31:2354:x1875503571409536:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 271us (589us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619913.880250:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 523 00002000:00100000:2.0:1788619913.880257:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5773b80 x1875503571409536/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/4536 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619913.880264:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5773b80 with x1875503571409536 00000100:00100000:3.0:1788619913.883182:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.883504:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571409664 00000100:00100000:1.0:1788619913.883515:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571409664 00002000:00100000:1.0:1788619913.883562:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5772680 x1875503571409664/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619913.883573:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5772680 with x1875503571409664 00000100:00100000:1.0:1788619913.883615:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 524 00000100:00100000:1.0:1788619913.883625:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5772680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+31:2355:x1875503571409664:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619913.883911:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5772680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+31:2355:x1875503571409664:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 286us (737us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619913.883928:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 524 00002000:00100000:1.0:1788619913.883936:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5772680 x1875503571409664/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/4536 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619913.883943:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5772680 with x1875503571409664 00000100:00100000:3.0:1788619913.887245:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619913.887381:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571409792 00000100:00100000:0.0:1788619913.887391:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571409792 00002000:00100000:0.0:1788619913.887440:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5771500 x1875503571409792/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619913.887451:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5771500 with x1875503571409792 00000100:00100000:0.0:1788619913.887489:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 525 00000100:00100000:0.0:1788619913.887499:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5771500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+31:2354:x1875503571409792:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619913.887745:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5771500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+31:2354:x1875503571409792:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 247us (508us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619913.887753:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 525 00002000:00100000:0.0:1788619913.887759:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5771500 x1875503571409792/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/4536 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619913.887767:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5771500 with x1875503571409792 00000100:00100000:3.0:1788619913.890419:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619913.890699:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571409920 00000100:00100000:2.0:1788619913.890709:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571409920 00002000:00100000:2.0:1788619913.890758:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5770e00 x1875503571409920/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619913.890768:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5770e00 with x1875503571409920 00000100:00100000:2.0:1788619913.890820:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 526 00000100:00100000:2.0:1788619913.890829:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5770e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+31:2354:x1875503571409920:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:0.0:1788619913.893446:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 913332 -> 913332 00000004:00010000:0.0:1788619913.893461:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619913 -> 1788619913 00000004:00010000:0.0:1788619913.893464:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619913 -> 1788619913 00000004:00010000:0.0:1788619913.893466:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619913 -> 1788619913 00000004:00010000:0.0:1788619913.893467:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 646 -> 646 00000100:00100000:0.0:1788619913.893536:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5770e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+33:2354:x1875503571409920:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 2704us (3120us total) trans 4294967643 rc 0/0 00000100:00100000:0.0:1788619913.893546:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 526 00002000:00100000:0.0:1788619913.893556:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5770e00 x1875503571409920/t4294967643(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:9/0 lens 488/448 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619913.893565:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5770e00 with x1875503571409920 00000100:00100000:3.0:1788619913.896475:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619913.896571:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571410048 00000100:00100000:0.0:1788619913.896582:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571410048 00000100:00100000:0.0:1788619913.896638:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 891 00000100:00100000:0.0:1788619913.896648:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5770a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+33:2354:x1875503571410048:12345-192.168.201.5@tcp:16:fsx.0 00000001:00080000:1.0:1788619913.929629:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967643 is committed 00000001:00080000:1.0:1788619913.929647:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967641 is committed 00000100:00100000:0.0:1788619913.930066:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5770a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571410048:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 33412us (33592us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619913.930078:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 891 00000100:00100000:3.0:1788619913.932757:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619913.932831:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571410176 00000100:00100000:2.0:1788619913.932848:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571410176 00000100:00100000:2.0:1788619913.932923:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 892 00000100:00100000:2.0:1788619913.932932:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5773480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:14908:x1875503571410176:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619913.932950:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619913.932970:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079800/0x1cd7fe53687346a6 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bb3c expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.933044:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe53687346ad lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.933059:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe53687346ad lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.933070:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe53687346ad lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788619913.933093:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 913332 00000004:00200000:2.0:1788619913.933096:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788619913.933123:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe53687346ad lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.933146:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079800/0x1cd7fe53687346a6 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb3c expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.933163:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe53687346ad lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb3c expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.933190:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619913.933193:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788619913.933197:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079000, rc 0) 00000100:00100000:2.0:1788619913.933254:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5773480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571410176:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 323us (505us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619913.933261:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 892 00000100:00100000:0.0:1788619913.935374:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.935500:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571410304 00000100:00100000:1.0:1788619913.935508:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571410304 00000100:00100000:1.0:1788619913.935566:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 675 00000100:00100000:1.0:1788619913.935576:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6343800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571410304:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619913.935591:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619913.935608:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002d38f023/0x1cd7fe53687346b4 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bb43 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619913.935675:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d58700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589094400:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619913.935767:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd963f00 going to sleep for 11 seconds 00000100:00100000:0.0:1788619913.938145:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d58700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589094400:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619913.938186:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002d38f023/0x1cd7fe53687346b4 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb43 expref: 17 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.938211:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d38f023/0x1cd7fe53687346b4 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb43 expref: 17 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.938242:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78ccca6e00, rc 0) 00000100:00100000:0.0:1788619913.938302:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6343800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571410304:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2727us (2936us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619913.938310:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 675 00000100:00100000:3.0:1788619913.940844:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619913.940926:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571410432 00000100:00100000:2.0:1788619913.940936:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571410432 00000100:00100000:2.0:1788619913.940993:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 676 00000100:00100000:2.0:1788619913.941004:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5772300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571410432:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619913.941023:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619913.941042:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dfdc9508/0x1cd7fe53687346bb lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bb4a expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619913.941103:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c77e2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589094528:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619913.941192:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc780 going to sleep for 11 seconds 00000100:00100000:2.0:1788619913.943497:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c77e2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589094528:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619913.943535:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000dfdc9508/0x1cd7fe53687346bb lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb4a expref: 17 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.943560:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dfdc9508/0x1cd7fe53687346bb lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb4a expref: 17 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619913.943573:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8078200, rc 0) 00000100:00100000:2.0:1788619913.943641:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5772300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571410432:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2638us (2813us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619913.943649:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 676 00000100:00100000:1.0:1788619913.946432:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619913.946514:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571410560 00000100:00100000:3.0:1788619913.946524:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571410560 00000100:00100000:3.0:1788619913.946581:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 893 00000100:00100000:3.0:1788619913.946594:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d58000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571410560:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619913.946616:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619913.946637:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687346c2 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bb51 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619913.946687:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe536873467c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb12 expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619913.946730:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5771c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589094656:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619913.946827:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164780 going to sleep for 11 seconds 00000004:00010000:1.0:1788619913.949880:0:5845:0:(mdt_lvb.c:201:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks: 646 -> 902 00000100:00100000:1.0:1788619913.949921:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5771c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589094656:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619913.949944:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 913332 00010000:00010000:1.0:1788619913.949964:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687346c2 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb51 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.949986:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619913.949989:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619913.950011:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687346c2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb51 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619913.950026:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683da00, rc 0) 00000100:00100000:1.0:1788619913.950091:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d58000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571410560:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3499us (3668us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619913.950099:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 893 00000100:00100000:1.0:1788619913.953573:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619913.953654:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571410688 00000100:00100000:0.0:1788619913.953674:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571410688 00000100:00100000:0.0:1788619913.953733:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 894 00000100:00100000:0.0:1788619913.953744:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d58700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571410688:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619913.953763:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619913.953783:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe53687346c9 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bb58 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.953813:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe536873467c lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb12 expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.953824:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe53687346c9 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb58 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.953866:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe536873467c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb12 expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.953879:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe536873467c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb12 expref: 12 pid: 11397 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619913.953891:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6340a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589094784:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619913.953999:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:0.0:1788619913.956601:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6340a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589094784:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619913.956638:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe53687346c9 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb58 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.956656:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619913.956658:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619913.956675:0:5846:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe53687346c9 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb58 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.956685:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6ee00, rc 0) 00000100:00100000:0.0:1788619913.956787:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d58700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:14908:x1875503571410688:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3043us (3223us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619913.956796:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 894 00000100:00100000:2.0:1788619913.958441:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619913.958519:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571410816 00000100:00100000:0.0:1788619913.958529:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571410816 00010000:00010000:0.0:1788619913.958567:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe536873467c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb12 expref: 13 pid: 11397 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619913.958607:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 302 00000100:00100000:0.0:1788619913.958616:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c77e2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571410816:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619913.958632:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619913.958657:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe536873467c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb12 expref: 13 pid: 11397 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.958732:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe536873467c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb12 expref: 13 pid: 11397 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.958766:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe536873467c lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb12 expref: 13 pid: 11397 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.958783:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619913.958789:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe53687346c9 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb58 expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.958800:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe53687346c9 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb58 expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.958808:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe53687346c9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb58 expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619913.958853:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe53687346c9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb58 expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619913.958865:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6340a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589094912:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619913.958936:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:0.0:1788619913.962249:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6340a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589094912:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619913.962275:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619913.962356:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c77e2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2355:x1875503571410816:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3740us (3920us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619913.962367:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 302 00000100:00100000:3.0:1788619913.965211:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.965603:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571410944 00000100:00100000:1.0:1788619913.965613:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571410944 00002000:00100000:1.0:1788619913.965718:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5771180 x1875503571410944/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619913.965730:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5771180 with x1875503571410944 00000100:00100000:1.0:1788619913.965805:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 527 00000100:00100000:1.0:1788619913.965815:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5771180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571410944:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619913.966090:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5771180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571410944:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 275us (886us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619913.966097:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 527 00002000:00100000:1.0:1788619913.966113:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5771180 x1875503571410944/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:9/0 lens 488/4536 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619913.966120:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5771180 with x1875503571410944 00000100:00100000:2.0:1788619913.968744:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619913.969321:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571411072 00000100:00100000:0.0:1788619913.969345:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571411072 00002000:00100000:0.0:1788619913.969408:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c77e2300 x1875503571411072/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619913.969419:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c77e2300 with x1875503571411072 00000100:00100000:0.0:1788619913.969442:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 528 00000100:00100000:0.0:1788619913.969452:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c77e2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571411072:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619913.969877:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c77e2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571411072:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 425us (1140us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619913.969885:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 528 00002000:00100000:0.0:1788619913.969892:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c77e2300 x1875503571411072/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:9/0 lens 488/4536 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619913.969898:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c77e2300 with x1875503571411072 00000100:00100000:1.0:1788619913.972930:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619913.973030:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571411200 00000100:00100000:0.0:1788619913.973040:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571411200 00002000:00100000:0.0:1788619913.973086:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5c33b80 x1875503571411200/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619913.973096:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5c33b80 with x1875503571411200 00000100:00100000:0.0:1788619913.973221:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 529 00000100:00100000:0.0:1788619913.973232:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c33b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571411200:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619913.973496:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c33b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571411200:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 266us (574us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619913.973503:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 529 00002000:00100000:0.0:1788619913.973510:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5c33b80 x1875503571411200/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:9/0 lens 488/4536 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619913.973517:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5c33b80 with x1875503571411200 00000100:00100000:2.0:1788619913.976434:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.976777:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571411328 00000100:00100000:1.0:1788619913.976787:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571411328 00002000:00100000:1.0:1788619913.976844:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c77e1f80 x1875503571411328/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619913.976872:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c77e1f80 with x1875503571411328 00000100:00100000:3.0:1788619913.976916:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 530 00000100:00100000:3.0:1788619913.976927:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c77e1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571411328:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619913.977195:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c77e1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571411328:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 269us (769us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619913.977202:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 530 00002000:00100000:3.0:1788619913.977209:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c77e1f80 x1875503571411328/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:9/0 lens 488/4536 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619913.977229:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c77e1f80 with x1875503571411328 00000100:00100000:2.0:1788619913.980418:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619913.980498:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571411456 00000100:00100000:0.0:1788619913.980509:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571411456 00002000:00100000:0.0:1788619913.980554:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c77e0380 x1875503571411456/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619913.980564:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c77e0380 with x1875503571411456 00000100:00100000:0.0:1788619913.980596:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 531 00000100:00100000:0.0:1788619913.980605:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c77e0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571411456:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619913.980872:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c77e0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571411456:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 267us (463us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619913.980880:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 531 00002000:00100000:0.0:1788619913.980887:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c77e0380 x1875503571411456/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:9/0 lens 488/4536 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619913.980894:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c77e0380 with x1875503571411456 00000100:00100000:2.0:1788619913.983938:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619913.984013:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571411584 00000100:00100000:3.0:1788619913.984023:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571411584 00002000:00100000:3.0:1788619913.984062:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c77e1880 x1875503571411584/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619913.984074:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c77e1880 with x1875503571411584 00000100:00100000:3.0:1788619913.984111:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 532 00000100:00100000:3.0:1788619913.984120:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c77e1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571411584:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619913.984398:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c77e1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571411584:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 278us (466us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619913.984406:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 532 00002000:00100000:3.0:1788619913.984413:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c77e1880 x1875503571411584/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:9/0 lens 488/4536 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619913.984420:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c77e1880 with x1875503571411584 00000100:00100000:2.0:1788619913.987600:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.987708:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571411712 00000100:00100000:1.0:1788619913.987718:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571411712 00002000:00100000:1.0:1788619913.987767:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c77e0a80 x1875503571411712/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619913.987778:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c77e0a80 with x1875503571411712 00000100:00100000:1.0:1788619913.987817:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 533 00000100:00100000:1.0:1788619913.987828:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c77e0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571411712:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619913.988201:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c77e0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571411712:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 275us (512us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619913.988210:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 533 00002000:00100000:1.0:1788619913.988218:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c77e0a80 x1875503571411712/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:9/0 lens 488/4536 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619913.988226:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c77e0a80 with x1875503571411712 00000100:00100000:2.0:1788619913.991179:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619913.991283:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571411840 00000100:00100000:0.0:1788619913.991293:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571411840 00002000:00100000:0.0:1788619913.991362:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c77e2a00 x1875503571411840/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619913.991373:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c77e2a00 with x1875503571411840 00000100:00100000:0.0:1788619913.991410:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 534 00000100:00100000:0.0:1788619913.991420:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c77e2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571411840:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619913.991672:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c77e2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571411840:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 254us (504us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619913.991680:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 534 00002000:00100000:0.0:1788619913.991687:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c77e2a00 x1875503571411840/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:9/0 lens 488/4536 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619913.991714:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c77e2a00 with x1875503571411840 00000100:00100000:2.0:1788619913.995059:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619913.995243:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571411968 00000100:00100000:1.0:1788619913.995253:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571411968 00002000:00100000:1.0:1788619913.995304:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79008afb80 x1875503571411968/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619913.995314:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79008afb80 with x1875503571411968 00000100:00100000:1.0:1788619913.995379:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 535 00000100:00100000:1.0:1788619913.995389:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79008afb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571411968:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619913.995654:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79008afb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571411968:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 265us (603us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619913.995670:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 535 00002000:00100000:1.0:1788619913.995691:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79008afb80 x1875503571411968/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:9/0 lens 488/4536 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619913.995699:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79008afb80 with x1875503571411968 00000100:00100000:2.0:1788619913.998764:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619913.998852:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571412096 00000100:00100000:3.0:1788619913.998862:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571412096 00002000:00100000:3.0:1788619913.998909:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79008ac380 x1875503571412096/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:9/0 lens 488/0 e 0 to 0 dl 1788619924 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619913.998920:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79008ac380 with x1875503571412096 00000100:00100000:3.0:1788619913.998982:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 536 00000100:00100000:3.0:1788619913.998992:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79008ac380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571412096:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619913.999261:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79008ac380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571412096:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 269us (506us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619913.999268:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 536 00002000:00100000:3.0:1788619913.999290:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79008ac380 x1875503571412096/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:9/0 lens 488/4536 e 0 to 0 dl 1788619924 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619913.999298:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79008ac380 with x1875503571412096 00000100:00100000:2.0:1788619914.002690:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619914.002785:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571412224 00000100:00100000:0.0:1788619914.002795:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571412224 00002000:00100000:0.0:1788619914.002866:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79008ac700 x1875503571412224/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:10/0 lens 488/0 e 0 to 0 dl 1788619925 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619914.002886:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79008ac700 with x1875503571412224 00000100:00100000:0.0:1788619914.002918:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 537 00000100:00100000:0.0:1788619914.002929:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79008ac700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571412224:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619914.003321:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79008ac700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571412224:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 393us (638us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619914.003353:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 537 00002000:00100000:0.0:1788619914.003361:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79008ac700 x1875503571412224/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:10/0 lens 488/4536 e 0 to 0 dl 1788619925 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619914.003368:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79008ac700 with x1875503571412224 00000100:00080000:1.0:1788619914.003842:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1788619914.006530:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619914.006630:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571412352 00000100:00100000:1.0:1788619914.006640:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571412352 00002000:00100000:1.0:1788619914.006693:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79008ad180 x1875503571412352/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:10/0 lens 488/0 e 0 to 0 dl 1788619925 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619914.006705:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79008ad180 with x1875503571412352 00000100:00100000:1.0:1788619914.006734:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 538 00000100:00100000:1.0:1788619914.006744:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79008ad180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571412352:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619914.007031:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79008ad180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571412352:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 287us (510us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619914.007038:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 538 00002000:00100000:1.0:1788619914.007046:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79008ad180 x1875503571412352/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:10/0 lens 488/4536 e 0 to 0 dl 1788619925 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619914.007054:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79008ad180 with x1875503571412352 00000100:00100000:3.0:1788619914.010638:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619914.010897:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571412480 00000100:00100000:0.0:1788619914.010907:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571412480 00002000:00100000:0.0:1788619914.010957:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5773100 x1875503571412480/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:10/0 lens 488/0 e 0 to 0 dl 1788619925 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619914.010968:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5773100 with x1875503571412480 00000100:00100000:0.0:1788619914.011006:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 539 00000100:00100000:0.0:1788619914.011016:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5773100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571412480:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619914.011306:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5773100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571412480:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 291us (677us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619914.011313:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 539 00002000:00100000:0.0:1788619914.011373:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5773100 x1875503571412480/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:10/0 lens 488/4536 e 0 to 0 dl 1788619925 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619914.011380:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5773100 with x1875503571412480 00000100:00100000:2.0:1788619914.014637:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619914.014740:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571412608 00000100:00100000:3.0:1788619914.014749:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571412608 00002000:00100000:3.0:1788619914.014797:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79008af100 x1875503571412608/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:10/0 lens 488/0 e 0 to 0 dl 1788619925 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619914.014809:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79008af100 with x1875503571412608 00000100:00100000:3.0:1788619914.014856:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 540 00000100:00100000:3.0:1788619914.014867:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79008af100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571412608:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619914.016014:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79008af100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571412608:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 1143us (1380us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619914.016025:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 540 00002000:00100000:2.0:1788619914.016034:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79008af100 x1875503571412608/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:10/0 lens 488/4536 e 0 to 0 dl 1788619925 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619914.016043:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79008af100 with x1875503571412608 00010000:00010000:1.0:1788619914.018646:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788619914.024822:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619914.024931:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571412736 00000100:00100000:1.0:1788619914.024941:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571412736 00002000:00100000:1.0:1788619914.024991:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79008ace00 x1875503571412736/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:10/0 lens 488/0 e 0 to 0 dl 1788619925 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619914.025003:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79008ace00 with x1875503571412736 00000100:00100000:1.0:1788619914.025075:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 541 00000100:00100000:1.0:1788619914.025085:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79008ace00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571412736:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:3.0:1788619914.028781:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 913332 -> 913332 00000004:00010000:3.0:1788619914.028791:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619913 -> 1788619913 00000004:00010000:3.0:1788619914.028794:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619913 -> 1788619913 00000004:00010000:3.0:1788619914.028796:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619913 -> 1788619913 00000004:00010000:3.0:1788619914.028799:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 902 -> 902 00000100:00100000:3.0:1788619914.028888:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79008ace00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2352:x1875503571412736:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 3799us (4071us total) trans 4294967644 rc 0/0 00000100:00100000:3.0:1788619914.028901:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 541 00002000:00100000:3.0:1788619914.028913:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79008ace00 x1875503571412736/t4294967644(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:10/0 lens 488/448 e 0 to 0 dl 1788619925 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619914.028923:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79008ace00 with x1875503571412736 00000100:00100000:1.0:1788619914.031322:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619914.031443:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571412864 00000100:00100000:2.0:1788619914.031452:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571412864 00000100:00100000:2.0:1788619914.031533:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 895 00000100:00100000:2.0:1788619914.031543:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c31c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2352:x1875503571412864:12345-192.168.201.5@tcp:16:fsx.0 00000100:00100000:3.0:1788619914.053140:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619914.053554:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571412992 00000100:00100000:1.0:1788619914.053565:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571412992 00000100:00100000:1.0:1788619914.053617:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 289 00000100:00100000:1.0:1788619914.053627:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5ce1050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:597450a2-e90c-408c-b401-a0ce48eed204+13:2354:x1875503571412992:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:1.0:1788619914.053796:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5ce1050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:597450a2-e90c-408c-b401-a0ce48eed204+13:2354:x1875503571412992:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 169us (664us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619914.053804:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 289 00000001:00080000:0.0:1788619914.062609:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967644 is committed 00000100:00100000:2.0:1788619914.062962:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c31c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571412864:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 31414us (31639us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619914.062993:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 895 00000100:00100000:2.0:1788619914.066154:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619914.066197:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619914.066269:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571413120 00000100:00100000:1.0:1788619914.066279:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571413120 00000100:00100000:1.0:1788619914.066365:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 896 00000100:00100000:1.0:1788619914.066374:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79008ad880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571413120:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619914.066393:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619914.066413:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7c00/0x1cd7fe53687346d0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bb5f expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.066447:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe53687346c9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb58 expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619914.066485:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c32680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589095040:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619914.066564:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd9636c0 going to sleep for 11 seconds 00000100:00080000:3.0:1788619914.067569:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1788619914.068158:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571413248 00000100:00100000:0.0:1788619914.068165:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571413248 00000100:00100000:0.0:1788619914.068225:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 677 00000100:00100000:0.0:1788619914.068235:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79008ac000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571413248:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619914.068248:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619914.068262:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000004390e69/0x1cd7fe53687346d7 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bb66 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619914.068309:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6340a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589095168:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619914.068376:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa240 going to sleep for 11 seconds 00000100:00080000:1.0:1788619914.068454:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000004:00010000:1.0:1788619914.070555:0:5845:0:(mdt_lvb.c:201:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks: 902 -> 1158 00000100:00100000:1.0:1788619914.070584:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c32680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589095040:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619914.070671:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 913332 00010000:00010000:1.0:1788619914.070689:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7c00/0x1cd7fe53687346d0 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb5f expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.070721:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619914.070724:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619914.070730:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7c00/0x1cd7fe53687346d0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb5f expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.070744:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78ccca7c00, rc 0) 00000100:00100000:1.0:1788619914.070795:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79008ad880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571413120:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4421us (4705us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619914.070803:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 896 00000100:00100000:2.0:1788619914.072589:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6340a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589095168:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619914.072733:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000004390e69/0x1cd7fe53687346d7 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb66 expref: 17 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.072756:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000004390e69/0x1cd7fe53687346d7 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb66 expref: 17 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.072771:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6fe00, rc 0) 00000100:00100000:2.0:1788619914.072875:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79008ac000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571413248:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4640us (6676us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619914.072883:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 677 00000100:00100000:0.0:1788619914.076423:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619914.076468:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619914.076552:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571413376 00000100:00100000:2.0:1788619914.076565:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571413376 00000100:00100000:2.0:1788619914.076632:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 897 00000100:00100000:2.0:1788619914.076645:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6342d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571413376:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619914.076676:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619914.076699:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe53687346de lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bb6d expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.076736:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe53687346c9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb58 expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619914.076800:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010ff480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589095296:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619914.076884:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc240 going to sleep for 11 seconds 00000100:00100000:3.0:1788619914.077051:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571413504 00000100:00100000:3.0:1788619914.077061:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571413504 00000100:00100000:3.0:1788619914.077097:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 678 00000100:00100000:3.0:1788619914.077116:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6342300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571413504:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619914.077125:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619914.077135:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe53687346e5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bb74 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619914.077169:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5770700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589095424:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619914.077201:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164300 going to sleep for 11 seconds 00000100:00100000:0.0:1788619914.080252:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010ff480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589095296:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619914.080282:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 913332 00010000:00010000:0.0:1788619914.080303:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe53687346de lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb6d expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.080322:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619914.080536:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619914.080544:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe53687346de lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb6d expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.080559:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079400, rc 0) 00000100:00100000:0.0:1788619914.080634:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6342d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571413376:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3991us (4220us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619914.080643:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 897 00000100:00100000:3.0:1788619914.081980:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5770700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589095424:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619914.082010:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe53687346e5 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb74 expref: 17 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.082029:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe53687346e5 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb74 expref: 17 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.082040:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683dc00, rc 0) 00000100:00100000:3.0:1788619914.082170:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6342300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571413504:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5065us (5700us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619914.082178:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 678 00000100:00100000:1.0:1788619914.084832:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619914.085039:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571413632 00000100:00100000:2.0:1788619914.085049:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571413632 00000100:00100000:2.0:1788619914.085155:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 542 00000100:00100000:2.0:1788619914.085165:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c32680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571413632:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:2.0:1788619914.085607:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 913332 -> 913332 00000004:00010000:2.0:1788619914.085612:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619913 -> 1788619913 00000004:00010000:2.0:1788619914.085615:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619913 -> 1788619913 00000004:00010000:2.0:1788619914.085617:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619913 -> 1788619913 00000004:00010000:2.0:1788619914.085619:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1158 -> 1158 00000100:00100000:2.0:1788619914.085694:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c32680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2352:x1875503571413632:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 530us (868us total) trans 4294967645 rc 0/0 00000100:00100000:2.0:1788619914.085701:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 542 00000100:00100000:1.0:1788619914.089503:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619914.089597:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571413760 00000100:00100000:0.0:1788619914.089607:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571413760 00000100:00100000:0.0:1788619914.089667:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 679 00000100:00100000:0.0:1788619914.089678:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5695c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:14908:x1875503571413760:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619914.089696:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619914.089713:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ff7ffc7/0x1cd7fe53687346ec lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bb7b expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.089746:0:7654:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000001e71691f/0x1cd7fe5368734652 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1baef expref: 7 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.089755:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(7 8) 00010000:00010000:0.0:1788619914.089760:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(8 8) 00010000:00010000:0.0:1788619914.089766:0:7654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ff7ffc7/0x1cd7fe53687346ec lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1527808->1581055] (req 1527808->1581055) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb7b expref: 17 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.089800:0:7654:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000001e71691f/0x1cd7fe5368734652 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1baef expref: 7 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.089813:0:7654:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000001e71691f/0x1cd7fe5368734652 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1baef expref: 7 pid: 6697 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619914.089827:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6341f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589095552:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619914.089929:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa840 going to sleep for 11 seconds 00000100:00100000:0.0:1788619914.091950:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6341f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589095552:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619914.091981:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003ff7ffc7/0x1cd7fe53687346ec lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1527808->1581055] (req 1527808->1581055) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb7b expref: 17 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.092002:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6e200, rc 0) 00000100:00100000:0.0:1788619914.092053:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5695c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+17:14908:x1875503571413760:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2377us (2560us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619914.092061:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 679 00000100:00100000:3.0:1788619914.093745:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619914.093878:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571413888 00000100:00100000:2.0:1788619914.093886:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571413888 00010000:00010000:2.0:1788619914.093949:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000001e71691f/0x1cd7fe5368734652 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1baef expref: 8 pid: 6697 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619914.093991:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 303 00000100:00100000:2.0:1788619914.094001:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5770700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+8:2355:x1875503571413888:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619914.094014:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619914.094038:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000001e71691f/0x1cd7fe5368734652 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1baef expref: 8 pid: 6697 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.094071:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000001e71691f/0x1cd7fe5368734652 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1baef expref: 8 pid: 6697 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.094093:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001e71691f/0x1cd7fe5368734652 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1baef expref: 8 pid: 6697 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.094121:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(8 8) 00010000:00010000:2.0:1788619914.094125:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(8 8) 00010000:00010000:2.0:1788619914.094134:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000003ff7ffc7/0x1cd7fe53687346ec lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1527808->1581055) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb7b expref: 16 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.094144:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ff7ffc7/0x1cd7fe53687346ec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1527808->1581055) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb7b expref: 16 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.094183:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000003ff7ffc7/0x1cd7fe53687346ec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1527808->1581055) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb7b expref: 16 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619914.094199:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fdc00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589095680:192.168.201.5@tcp:105: 00000100:00080000:1.0:1788619914.094318:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1788619914.094554:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc000 going to sleep for 11 seconds 00000100:00100000:2.0:1788619914.097307:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fdc00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589095680:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619914.097363:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619914.097414:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5770700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571413888:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3413us (3673us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619914.097425:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 303 00000100:00100000:0.0:1788619914.103889:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619914.103993:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571414016 00000100:00100000:2.0:1788619914.104003:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571414016 00000100:00100000:2.0:1788619914.104061:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 680 00000100:00100000:2.0:1788619914.104072:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6341f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571414016:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619914.104090:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619914.104119:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000040c3172a/0x1cd7fe53687346f3 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bb82 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619914.104178:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589095808:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619914.104262:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc000 going to sleep for 11 seconds 00002000:00010000:2.0:1788619914.106681:0:6698:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size: 0 -> 1577540 00000100:00100000:2.0:1788619914.106711:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589095808:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619914.106744:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000040c3172a/0x1cd7fe53687346f3 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb82 expref: 8 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.106767:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000040c3172a/0x1cd7fe53687346f3 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb82 expref: 8 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.106781:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079400, rc 0) 00000100:00100000:2.0:1788619914.106902:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6341f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571414016:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2831us (3022us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619914.106911:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 680 00000100:00100000:1.0:1788619914.110171:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619914.110258:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571414144 00000100:00100000:2.0:1788619914.110268:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571414144 00000100:00100000:2.0:1788619914.110381:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 681 00000100:00100000:2.0:1788619914.110392:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5696a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571414144:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619914.110427:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619914.110445:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000855993/0x1cd7fe53687346fa lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bb89 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619914.110502:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589095936:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619914.110583:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc000 going to sleep for 11 seconds 00000100:00100000:2.0:1788619914.112625:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589095936:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619914.112661:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000000855993/0x1cd7fe53687346fa lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb89 expref: 8 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.112696:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000855993/0x1cd7fe53687346fa lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb89 expref: 8 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.112708:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079c00, rc 0) 00000100:00100000:2.0:1788619914.112760:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5696a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571414144:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2371us (2661us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619914.112769:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 681 00000100:00100000:3.0:1788619914.115428:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619914.115523:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571414272 00000100:00100000:2.0:1788619914.115532:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571414272 00000100:00100000:2.0:1788619914.115593:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 898 00000100:00100000:2.0:1788619914.115604:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5771c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:14908:x1875503571414272:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619914.115624:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619914.115643:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368734701 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bb90 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.115702:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe53687346c9 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb58 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.115712:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368734701 lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb90 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.115744:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe53687346c9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb58 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.115757:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe53687346c9 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb58 expref: 10 pid: 5846 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619914.115771:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fdc00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589096064:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619914.115862:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc000 going to sleep for 11 seconds 00000100:00100000:2.0:1788619914.117783:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fdc00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589096064:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619914.117816:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368734701 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb90 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.117845:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619914.117849:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619914.117854:0:5844:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368734701 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb90 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.117865:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079400, rc 0) 00000100:00100000:2.0:1788619914.117929:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5771c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571414272:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2326us (2511us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619914.117951:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 898 00000100:00100000:0.0:1788619914.120025:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619914.120204:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571414400 00000100:00100000:1.0:1788619914.120213:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571414400 00010000:00010000:1.0:1788619914.120250:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe53687346c9 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb58 expref: 11 pid: 5846 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619914.120290:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 304 00000100:00100000:1.0:1788619914.120300:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6343100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571414400:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619914.120316:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619914.120370:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe53687346c9 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb58 expref: 11 pid: 5846 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.120383:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe53687346c9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb58 expref: 11 pid: 5846 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.120420:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ee00/0x1cd7fe53687346c9 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb58 expref: 11 pid: 5846 timeout: 506 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.120435:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619914.120441:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368734701 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb90 expref: 12 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.120451:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368734701 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb90 expref: 12 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.120459:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368734701 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb90 expref: 12 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.120496:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368734701 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb90 expref: 12 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619914.120509:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501d880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589096192:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619914.120590:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fd9636c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619914.126372:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501d880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589096192:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619914.126401:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619914.126451:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6343100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571414400:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 6151us (6432us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619914.126462:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 304 00000100:00100000:3.0:1788619914.131769:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619914.131902:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571414656 00000100:00100000:1.0:1788619914.131912:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571414656 00002000:00100000:1.0:1788619914.131975:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1527807) req@ffff8e77c6a89c00 x1875503571414656/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 488/0 e 0 to 0 dl 1788619925 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619914.131986:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a89c00 with x1875503571414656 ext(1048576->1527807) 00002000:00010000:1.0:1788619914.131992:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a89c00 00000100:00100000:1.0:1788619914.132029:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 361 00000100:00100000:1.0:1788619914.132038:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a89c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2355:x1875503571414656:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619914.134244:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619914.134320:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571414528 00000100:00100000:3.0:1788619914.134358:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571414528 00002000:00100000:3.0:1788619914.134401:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6340700 x1875503571414528/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 488/0 e 0 to 0 dl 1788619925 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619914.134423:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6340700 with x1875503571414528 00000100:00100000:3.0:1788619914.134449:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 543 00000100:00100000:3.0:1788619914.134459:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6340700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571414528:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619914.144515:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a89c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2355:x1875503571414656:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 12462us (12740us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619914.144533:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 361 00002000:00100000:2.0:1788619914.144548:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1527807) req@ffff8e77c6a89c00 x1875503571414656/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 488/440 e 0 to 0 dl 1788619925 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619914.144560:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a89c00 with x1875503571414656 ext(1048576->1527807) 00002000:00010000:2.0:1788619914.144567:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a89c00 00000100:00100000:2.0:1788619914.146665:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6340700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571414528:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 12189us (12411us total) trans 0 rc 913408/913408 00000100:00100000:2.0:1788619914.146678:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 543 00002000:00100000:2.0:1788619914.146690:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6340700 x1875503571414528/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 488/440 e 0 to 0 dl 1788619925 ref 1 fl Complete:/600/0 rc 913408/913408 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619914.146700:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6340700 with x1875503571414528 00000100:00100000:0.0:1788619914.153254:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619914.153360:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571414784 00000100:00100000:1.0:1788619914.153369:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571414784 00000100:00100000:1.0:1788619914.153420:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 899 00000100:00100000:1.0:1788619914.153429:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6340000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571414784:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619914.153449:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619914.153453:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:1.0:1788619914.153496:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368734701 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb90 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.153515:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619914.153520:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788619914.153532:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190c00/0x1cd7fe5368734708 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bb97 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.153556:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190c00/0x1cd7fe5368734708 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb97 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.153568:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190c00/0x1cd7fe5368734708 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb97 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.153594:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619914.153597:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619914.153600:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78df190c00, rc 0) 00000100:00100000:1.0:1788619914.153666:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6340000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571414784:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 230us (413us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619914.153673:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 899 00000100:00100000:2.0:1788619914.173846:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619914.173947:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571414912 00000100:00100000:0.0:1788619914.173956:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571414912 00002000:00100000:0.0:1788619914.174009:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79010fdc00 x1875503571414912/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 488/0 e 0 to 0 dl 1788619925 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619914.174019:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79010fdc00 with x1875503571414912 00000100:00100000:0.0:1788619914.174057:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 544 00000100:00100000:0.0:1788619914.174066:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010fdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571414912:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619914.174364:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010fdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571414912:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 259us (500us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619914.174374:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 544 00002000:00100000:0.0:1788619914.174382:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79010fdc00 x1875503571414912/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 488/4536 e 0 to 0 dl 1788619925 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619914.174408:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79010fdc00 with x1875503571414912 00000100:00100000:3.0:1788619914.186296:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619914.186451:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571415040 00000100:00100000:2.0:1788619914.186461:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571415040 00002000:00100000:2.0:1788619914.186510:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6a8b480 x1875503571415040/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 488/0 e 0 to 0 dl 1788619925 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619914.186520:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6a8b480 with x1875503571415040 00000100:00100000:2.0:1788619914.186546:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 545 00000100:00100000:2.0:1788619914.186555:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a8b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571415040:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619914.186872:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a8b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571415040:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 317us (585us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619914.186889:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 545 00002000:00100000:2.0:1788619914.186897:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6a8b480 x1875503571415040/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 488/4536 e 0 to 0 dl 1788619925 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619914.186905:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6a8b480 with x1875503571415040 00000100:00100000:1.0:1788619914.191212:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619914.191378:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571415168 00000100:00100000:2.0:1788619914.191388:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571415168 00002000:00100000:2.0:1788619914.191552:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c501d880 x1875503571415168/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 488/0 e 0 to 0 dl 1788619925 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619914.191563:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c501d880 with x1875503571415168 00000100:00100000:2.0:1788619914.191604:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 546 00000100:00100000:2.0:1788619914.191614:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571415168:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619914.191874:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571415168:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 261us (670us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619914.191881:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 546 00002000:00100000:2.0:1788619914.191888:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c501d880 x1875503571415168/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 488/4536 e 0 to 0 dl 1788619925 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619914.191896:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c501d880 with x1875503571415168 00000100:00100000:2.0:1788619914.195986:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619914.196084:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571415296 00000100:00100000:0.0:1788619914.196094:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571415296 00002000:00100000:0.0:1788619914.196153:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79010fc700 x1875503571415296/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 488/0 e 0 to 0 dl 1788619925 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619914.196164:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79010fc700 with x1875503571415296 00000100:00100000:0.0:1788619914.196239:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 547 00000100:00100000:0.0:1788619914.196249:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010fc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571415296:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619914.196544:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010fc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571415296:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 296us (565us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619914.196552:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 547 00002000:00100000:0.0:1788619914.196559:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79010fc700 x1875503571415296/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 488/4536 e 0 to 0 dl 1788619925 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619914.196567:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79010fc700 with x1875503571415296 00000100:00100000:3.0:1788619914.200081:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619914.200215:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571415424 00000100:00100000:2.0:1788619914.200225:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571415424 00002000:00100000:2.0:1788619914.200272:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6a8a300 x1875503571415424/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 488/0 e 0 to 0 dl 1788619925 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619914.200284:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6a8a300 with x1875503571415424 00000100:00100000:2.0:1788619914.200321:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 548 00000100:00100000:2.0:1788619914.200359:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a8a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571415424:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619914.200744:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a8a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571415424:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 386us (671us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619914.200751:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 548 00002000:00100000:2.0:1788619914.200758:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6a8a300 x1875503571415424/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 488/4536 e 0 to 0 dl 1788619925 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619914.200766:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6a8a300 with x1875503571415424 00000100:00100000:1.0:1788619914.215054:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619914.215279:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571415552 00000100:00100000:3.0:1788619914.215290:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571415552 00002000:00100000:3.0:1788619914.215376:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e647a680 x1875503571415552/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 488/0 e 0 to 0 dl 1788619925 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619914.215388:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e647a680 with x1875503571415552 00000100:00100000:3.0:1788619914.215425:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 549 00000100:00100000:3.0:1788619914.215435:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e647a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571415552:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619914.215681:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e647a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571415552:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 246us (637us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619914.215688:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 549 00002000:00100000:3.0:1788619914.215695:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e647a680 x1875503571415552/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 488/4536 e 0 to 0 dl 1788619925 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619914.215703:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e647a680 with x1875503571415552 00000100:00100000:3.0:1788619914.220172:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619914.220269:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571415680 00000100:00100000:0.0:1788619914.220281:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571415680 00002000:00100000:0.0:1788619914.220363:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6a88700 x1875503571415680/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 488/0 e 0 to 0 dl 1788619925 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619914.220374:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6a88700 with x1875503571415680 00000100:00100000:0.0:1788619914.220418:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 550 00000100:00100000:0.0:1788619914.220428:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a88700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571415680:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619914.220708:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a88700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571415680:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 280us (543us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619914.220715:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 550 00002000:00100000:0.0:1788619914.220731:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6a88700 x1875503571415680/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 488/4536 e 0 to 0 dl 1788619925 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619914.220738:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6a88700 with x1875503571415680 00000100:00100000:0.0:1788619914.224969:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619914.230394:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571415808 00000100:00100000:2.0:1788619914.230521:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571415808 00002000:00100000:2.0:1788619914.230586:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6340a80 x1875503571415808/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 488/0 e 0 to 0 dl 1788619925 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619914.230597:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6340a80 with x1875503571415808 00000100:00100000:2.0:1788619914.230686:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 551 00000100:00100000:2.0:1788619914.230695:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6340a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571415808:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619914.230990:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6340a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571415808:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 295us (6031us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619914.230999:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 551 00002000:00100000:2.0:1788619914.231007:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6340a80 x1875503571415808/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 488/4536 e 0 to 0 dl 1788619925 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619914.231014:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6340a80 with x1875503571415808 00000100:00100000:1.0:1788619914.236639:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619914.236703:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571415936 00000100:00100000:3.0:1788619914.236713:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571415936 00002000:00100000:3.0:1788619914.236765:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6f84380 x1875503571415936/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 488/0 e 0 to 0 dl 1788619925 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619914.236777:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6f84380 with x1875503571415936 00000100:00100000:3.0:1788619914.236822:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 552 00000100:00100000:3.0:1788619914.236841:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6f84380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571415936:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619914.237091:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6f84380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571415936:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 259us (462us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619914.237183:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 552 00002000:00100000:3.0:1788619914.237190:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6f84380 x1875503571415936/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 488/4536 e 0 to 0 dl 1788619925 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619914.237207:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6f84380 with x1875503571415936 00000100:00100000:3.0:1788619914.242065:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619914.242488:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571416064 00000100:00100000:2.0:1788619914.242498:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571416064 00002000:00100000:2.0:1788619914.242542:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6a88000 x1875503571416064/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 488/0 e 0 to 0 dl 1788619925 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619914.242550:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6a88000 with x1875503571416064 00000100:00100000:2.0:1788619914.242591:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 553 00000100:00100000:2.0:1788619914.242601:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a88000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571416064:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619914.242957:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a88000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571416064:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 356us (965us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619914.242964:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 553 00002000:00100000:2.0:1788619914.242972:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6a88000 x1875503571416064/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 488/4536 e 0 to 0 dl 1788619925 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619914.242980:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6a88000 with x1875503571416064 00000100:00100000:0.0:1788619914.248163:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619914.248298:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571416192 00000100:00100000:3.0:1788619914.248309:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571416192 00002000:00100000:3.0:1788619914.248380:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6340380 x1875503571416192/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 488/0 e 0 to 0 dl 1788619925 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619914.248390:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6340380 with x1875503571416192 00000100:00100000:3.0:1788619914.248536:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 554 00000100:00100000:3.0:1788619914.248547:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6340380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571416192:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619914.249092:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6340380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571416192:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 545us (993us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619914.249108:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 554 00002000:00100000:3.0:1788619914.249117:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6340380 x1875503571416192/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 488/4536 e 0 to 0 dl 1788619925 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619914.249124:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6340380 with x1875503571416192 00000100:00100000:1.0:1788619914.256043:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619914.256224:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571416320 00000100:00100000:2.0:1788619914.256239:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571416320 00002000:00100000:2.0:1788619914.256291:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6f86d80 x1875503571416320/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 488/0 e 0 to 0 dl 1788619925 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619914.256303:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6f86d80 with x1875503571416320 00000100:00100000:2.0:1788619914.256371:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 555 00000100:00100000:2.0:1788619914.256381:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6f86d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571416320:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619914.256649:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6f86d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571416320:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 269us (615us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619914.256665:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 555 00002000:00100000:2.0:1788619914.256672:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6f86d80 x1875503571416320/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 488/4536 e 0 to 0 dl 1788619925 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619914.256679:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6f86d80 with x1875503571416320 00000100:00100000:2.0:1788619914.260940:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619914.261026:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571416448 00000100:00100000:3.0:1788619914.261037:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571416448 00002000:00100000:3.0:1788619914.261087:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79010fea00 x1875503571416448/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 488/0 e 0 to 0 dl 1788619925 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619914.261167:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79010fea00 with x1875503571416448 00000100:00100000:3.0:1788619914.261203:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 556 00000100:00100000:3.0:1788619914.261214:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010fea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571416448:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619914.264139:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010fea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571416448:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 2917us (3197us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619914.264165:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 556 00002000:00100000:2.0:1788619914.264178:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79010fea00 x1875503571416448/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 488/4536 e 0 to 0 dl 1788619925 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619914.264188:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79010fea00 with x1875503571416448 00000100:00100000:3.0:1788619914.269711:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619914.269911:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571416576 00000100:00100000:0.0:1788619914.269921:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571416576 00002000:00100000:0.0:1788619914.269968:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6a88380 x1875503571416576/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 488/0 e 0 to 0 dl 1788619925 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619914.269976:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6a88380 with x1875503571416576 00000100:00100000:0.0:1788619914.270022:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 557 00000100:00100000:0.0:1788619914.270031:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a88380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571416576:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619914.270409:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a88380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571416576:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 261us (587us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619914.270418:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 557 00002000:00100000:0.0:1788619914.270426:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6a88380 x1875503571416576/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 488/440 e 0 to 0 dl 1788619925 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619914.270434:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6a88380 with x1875503571416576 00000100:00100000:1.0:1788619914.275554:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619914.275698:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571416704 00000100:00100000:2.0:1788619914.275707:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571416704 00002000:00100000:2.0:1788619914.275757:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6f87100 x1875503571416704/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 488/0 e 0 to 0 dl 1788619925 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619914.275768:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6f87100 with x1875503571416704 00000100:00100000:2.0:1788619914.275874:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 558 00000100:00100000:2.0:1788619914.275885:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6f87100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571416704:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619914.276143:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6f87100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571416704:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 259us (596us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619914.276150:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 558 00002000:00100000:2.0:1788619914.276157:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6f87100 x1875503571416704/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 488/440 e 0 to 0 dl 1788619925 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619914.276164:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6f87100 with x1875503571416704 00000100:00100000:2.0:1788619914.283979:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619914.284112:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571416832 00000100:00100000:0.0:1788619914.284123:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571416832 00002000:00100000:0.0:1788619914.284174:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79010fd180 x1875503571416832/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 488/0 e 0 to 0 dl 1788619925 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619914.284184:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79010fd180 with x1875503571416832 00000100:00100000:0.0:1788619914.284237:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 559 00000100:00100000:0.0:1788619914.284248:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010fd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571416832:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619914.284565:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010fd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571416832:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 318us (596us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619914.284573:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 559 00002000:00100000:0.0:1788619914.284580:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79010fd180 x1875503571416832/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 488/440 e 0 to 0 dl 1788619925 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619914.284587:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79010fd180 with x1875503571416832 00000100:00100000:1.0:1788619914.288565:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619914.289980:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571416960 00000100:00100000:2.0:1788619914.289992:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571416960 00002000:00100000:2.0:1788619914.290051:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6f85880 x1875503571416960/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 488/0 e 0 to 0 dl 1788619925 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619914.290063:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6f85880 with x1875503571416960 00000100:00100000:2.0:1788619914.290228:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 560 00000100:00100000:2.0:1788619914.290254:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6f85880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571416960:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619914.290546:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6f85880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571416960:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 294us (1991us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619914.290554:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 560 00002000:00100000:2.0:1788619914.290562:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6f85880 x1875503571416960/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 488/440 e 0 to 0 dl 1788619925 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619914.290570:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6f85880 with x1875503571416960 00000100:00100000:1.0:1788619914.295636:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619914.295920:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571417088 00000100:00100000:0.0:1788619914.295930:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571417088 00002000:00100000:0.0:1788619914.295983:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6f87800 x1875503571417088/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 488/0 e 0 to 0 dl 1788619925 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619914.295995:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6f87800 with x1875503571417088 00000100:00100000:0.0:1788619914.296091:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 561 00000100:00100000:0.0:1788619914.296102:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6f87800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571417088:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:0.0:1788619914.310323:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 913332 -> 929792 00000004:00010000:0.0:1788619914.310374:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619913 -> 1788619914 00000004:00010000:0.0:1788619914.310377:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619913 -> 1788619913 00000004:00010000:0.0:1788619914.310379:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619913 -> 1788619914 00000004:00010000:0.0:1788619914.310382:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1158 -> 1158 00000100:00100000:0.0:1788619914.310461:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6f87800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2352:x1875503571417088:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 14356us (14832us total) trans 4294967646 rc 0/0 00000100:00100000:0.0:1788619914.310473:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 561 00002000:00100000:0.0:1788619914.310487:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6f87800 x1875503571417088/t4294967646(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 488/448 e 0 to 0 dl 1788619925 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619914.310497:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6f87800 with x1875503571417088 00000100:00100000:1.0:1788619914.314441:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619914.314757:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571417216 00000100:00100000:0.0:1788619914.314767:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571417216 00000100:00100000:0.0:1788619914.314828:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 900 00000100:00100000:0.0:1788619914.314849:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6f84700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571417216:12345-192.168.201.5@tcp:16:fsx.0 00000100:00100000:0.0:1788619914.338462:0:3316:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1788619914.338485:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6341500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589096320:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1788619914.338552:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619914.339351:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589096320 00000100:00100000:1.0:1788619914.339361:0:6699:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589096320 00000100:00100000:1.0:1788619914.339413:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 86 00000100:00100000:1.0:1788619914.339423:0:6699:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6341180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503589096320:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1788619914.339458:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3856662528 avail 3854565376 00000100:00100000:1.0:1788619914.339510:0:6699:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6341180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503589096320:12345-0@lo:13:osp-pre-1-0.0 Request processed in 89us (958us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619914.339517:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 86 00000100:00100000:1.0:1788619914.339564:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6341500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589096320:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1788619914.343135:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6f84700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571417216:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 28279us (28694us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619914.343164:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 900 00000001:00080000:1.0:1788619914.343197:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967646 is committed 00000001:00080000:1.0:1788619914.343210:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967645 is committed 00000100:00100000:1.0:1788619914.353244:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619914.353351:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619914.353421:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571417344 00000100:00100000:2.0:1788619914.353431:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571417344 00000100:00100000:2.0:1788619914.353488:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 901 00000100:00100000:2.0:1788619914.353499:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571417344:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619914.353523:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619914.353545:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8078200/0x1cd7fe536873470f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bb9e expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.353583:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190c00/0x1cd7fe5368734708 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb97 expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619914.353622:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fd500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589096448:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619914.353711:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccb40 going to sleep for 11 seconds 00000100:00100000:3.0:1788619914.356710:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571417472 00000100:00100000:3.0:1788619914.356724:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571417472 00000100:00100000:3.0:1788619914.356786:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 682 00000100:00100000:3.0:1788619914.356795:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6f86a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571417472:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619914.356813:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619914.356840:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe5368734716 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bba5 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619914.356900:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a88a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589096576:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619914.357273:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164600 going to sleep for 11 seconds 00000004:00010000:3.0:1788619914.359062:0:5844:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619913 -> 1788619914 00000004:00010000:3.0:1788619914.359070:0:5844:0:(mdt_lvb.c:201:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks: 1158 -> 1414 00000100:00100000:3.0:1788619914.359102:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fd500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589096448:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619914.359124:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 929792 00010000:00010000:3.0:1788619914.359141:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8078200/0x1cd7fe536873470f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb9e expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.359159:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619914.359163:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619914.359168:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8078200/0x1cd7fe536873470f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb9e expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.359204:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8078200, rc 0) 00000100:00100000:3.0:1788619914.359254:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571417344:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5756us (5901us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619914.359262:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 901 00002000:00010000:3.0:1788619914.360731:0:6697:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime: 1788619913 -> 1788619914 00002000:00010000:3.0:1788619914.360737:0:6697:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619913 -> 1788619914 00002000:00010000:3.0:1788619914.360739:0:6697:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime: 1788619913 -> 1788619914 00000100:00100000:3.0:1788619914.360757:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a88a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589096576:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619914.360784:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe5368734716 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bba5 expref: 8 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.360803:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe5368734716 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bba5 expref: 8 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.360815:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683dc00, rc 0) 00000100:00100000:3.0:1788619914.360887:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6f86a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571417472:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4092us (7651us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619914.360894:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 682 00000100:00100000:0.0:1788619914.364139:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619914.364188:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619914.364321:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571417728 00000100:00100000:1.0:1788619914.364364:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571417728 00000100:00100000:1.0:1788619914.364420:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 683 00000100:00100000:1.0:1788619914.364431:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6340e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571417728:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619914.364452:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619914.364470:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000010f5ea39/0x1cd7fe536873471d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bbb3 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619914.364537:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cda300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589096704:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619914.364621:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48ce40 going to sleep for 11 seconds 00000100:00100000:3.0:1788619914.365562:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571417600 00000100:00100000:3.0:1788619914.365569:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571417600 00000100:00100000:3.0:1788619914.365622:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 902 00000100:00100000:3.0:1788619914.365630:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6341c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571417600:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619914.365644:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619914.365670:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368734724 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bbac expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.365724:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190c00/0x1cd7fe5368734708 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb97 expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619914.365757:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a88a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589096832:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619914.365818:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164600 going to sleep for 11 seconds 00000100:00100000:1.0:1788619914.367704:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cda300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589096704:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619914.367754:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000010f5ea39/0x1cd7fe536873471d lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bbb3 expref: 8 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.367780:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000010f5ea39/0x1cd7fe536873471d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bbb3 expref: 8 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.367794:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78df190800, rc 0) 00000100:00100000:1.0:1788619914.367860:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6340e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571417728:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3429us (3729us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619914.367886:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 683 00000100:00100000:3.0:1788619914.371030:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a88a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589096832:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619914.371054:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 929792 00010000:00010000:3.0:1788619914.371073:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368734724 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bbac expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.371092:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619914.371095:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619914.371115:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368734724 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bbac expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.371131:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d200, rc 0) 00000100:00100000:3.0:1788619914.371190:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6341c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571417600:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5560us (6999us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619914.371198:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 902 00000100:00100000:2.0:1788619914.384460:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619914.384524:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619914.384606:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571417984 00000100:00100000:3.0:1788619914.384617:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571417984 00002000:00100000:3.0:1788619914.384678:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1581056->1978367) req@ffff8e79010fd880 x1875503571417984/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 488/0 e 0 to 0 dl 1788619925 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619914.384690:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e79010fd880 with x1875503571417984 ext(1581056->1978367) 00002000:00010000:3.0:1788619914.384695:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e79010fd880 00000100:00100000:3.0:1788619914.384736:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 362 00000100:00100000:3.0:1788619914.384745:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010fd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2355:x1875503571417984:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619914.385577:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571417856 00000100:00100000:0.0:1788619914.385584:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571417856 00002000:00100000:0.0:1788619914.385616:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79010fc380 x1875503571417856/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 504/0 e 0 to 0 dl 1788619925 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619914.385626:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79010fc380 with x1875503571417856 00000100:00100000:0.0:1788619914.385665:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 562 00000100:00100000:0.0:1788619914.385673:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010fc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571417856:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619914.390455:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010fd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2355:x1875503571417984:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 5699us (5919us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619914.390469:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 362 00002000:00100000:3.0:1788619914.390481:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1581056->1978367) req@ffff8e79010fd880 x1875503571417984/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 488/440 e 0 to 0 dl 1788619925 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619914.390491:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e79010fd880 with x1875503571417984 ext(1581056->1978367) 00002000:00010000:3.0:1788619914.390498:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e79010fd880 00000100:00100000:2.0:1788619914.400117:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010fc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571417856:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 14434us (15667us total) trans 0 rc 860160/860160 00000100:00100000:2.0:1788619914.400132:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 562 00002000:00100000:2.0:1788619914.400146:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79010fc380 x1875503571417856/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 504/440 e 0 to 0 dl 1788619925 ref 1 fl Complete:/600/0 rc 860160/860160 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619914.400173:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79010fc380 with x1875503571417856 00000100:00100000:0.0:1788619914.406517:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619914.406695:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571418112 00000100:00100000:1.0:1788619914.406704:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571418112 00000100:00100000:1.0:1788619914.406760:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 903 00000100:00100000:1.0:1788619914.406769:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6343b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571418112:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619914.406789:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619914.406808:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df191600/0x1cd7fe536873472b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bbba expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.406846:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190c00/0x1cd7fe5368734708 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb97 expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.406856:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df191600/0x1cd7fe536873472b lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bbba expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.406885:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190c00/0x1cd7fe5368734708 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb97 expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.406898:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190c00/0x1cd7fe5368734708 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb97 expref: 12 pid: 5845 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619914.406911:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cda300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589096960:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619914.406978:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48ce40 going to sleep for 11 seconds 00000100:00100000:1.0:1788619914.411162:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619914.411291:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cda300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589096960:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619914.411363:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df191600/0x1cd7fe536873472b lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bbba expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.411381:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619914.411385:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619914.411389:0:5846:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df191600/0x1cd7fe536873472b lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bbba expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.411420:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78df191600, rc 0) 00000100:00100000:0.0:1788619914.411475:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6343b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:14908:x1875503571418112:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4706us (4969us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619914.411483:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 903 00000100:00100000:3.0:1788619914.411610:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571418240 00000100:00100000:3.0:1788619914.411617:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571418240 00010000:00010000:3.0:1788619914.411647:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190c00/0x1cd7fe5368734708 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb97 expref: 13 pid: 5845 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619914.411681:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 305 00000100:00100000:3.0:1788619914.411690:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd8a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571418240:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619914.411701:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619914.411726:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190c00/0x1cd7fe5368734708 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb97 expref: 13 pid: 5845 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.411738:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190c00/0x1cd7fe5368734708 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb97 expref: 13 pid: 5845 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.411758:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190c00/0x1cd7fe5368734708 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb97 expref: 13 pid: 5845 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.411783:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619914.411788:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df191600/0x1cd7fe536873472b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bbba expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.411797:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df191600/0x1cd7fe536873472b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bbba expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.411806:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df191600/0x1cd7fe536873472b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bbba expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.411852:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df191600/0x1cd7fe536873472b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bbba expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619914.411866:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a88a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589097088:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619914.411915:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164600 going to sleep for 11 seconds 00000100:00100000:3.0:1788619914.415981:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a88a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589097088:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619914.416007:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619914.416069:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd8a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2355:x1875503571418240:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 4378us (4916us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619914.416079:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 305 00000100:00100000:0.0:1788619914.416238:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619914.416798:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571418368 00000100:00100000:2.0:1788619914.416806:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571418368 00000100:00100000:2.0:1788619914.416862:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 684 00000100:00100000:2.0:1788619914.416872:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6341500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571418368:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619914.416890:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619914.416919:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dfdc9508/0x1cd7fe5368734732 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bbc1 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619914.416974:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010ffb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589097216:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619914.417042:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccb40 going to sleep for 11 seconds 00002000:00010000:2.0:1788619914.418998:0:6698:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size: 1577540 -> 2025263 00000100:00100000:2.0:1788619914.419029:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010ffb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589097216:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619914.419061:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000dfdc9508/0x1cd7fe5368734732 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bbc1 expref: 8 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.419084:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dfdc9508/0x1cd7fe5368734732 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bbc1 expref: 8 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.419108:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8078200, rc 0) 00000100:00100000:2.0:1788619914.419163:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6341500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571418368:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2291us (2925us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619914.419170:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 684 00000100:00100000:0.0:1788619914.425048:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619914.425169:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571418496 00000100:00100000:2.0:1788619914.425194:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571418496 00002000:00100000:2.0:1788619914.425245:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5d56a00 x1875503571418496/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:10/0 lens 504/0 e 0 to 0 dl 1788619925 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619914.425256:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5d56a00 with x1875503571418496 00000100:00100000:2.0:1788619914.425290:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 563 00000100:00100000:2.0:1788619914.425300:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d56a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571418496:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619914.439663:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d56a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571418496:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 14343us (14603us total) trans 0 rc 929792/929792 00000100:00100000:2.0:1788619914.439677:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 563 00002000:00100000:2.0:1788619914.439690:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5d56a00 x1875503571418496/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:10/0 lens 504/440 e 0 to 0 dl 1788619925 ref 1 fl Complete:/600/0 rc 929792/929792 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619914.439700:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5d56a00 with x1875503571418496 00000100:00100000:1.0:1788619914.443019:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619914.443207:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571418624 00000100:00100000:2.0:1788619914.443218:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571418624 00002000:00100000:2.0:1788619914.443269:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5cdb480 x1875503571418624/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:10/0 lens 488/0 e 0 to 0 dl 1788619925 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619914.443280:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5cdb480 with x1875503571418624 00000100:00100000:2.0:1788619914.443424:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 564 00000100:00100000:2.0:1788619914.443435:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cdb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571418624:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619914.443911:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cdb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571418624:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 476us (899us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619914.443918:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 564 00002000:00100000:2.0:1788619914.443927:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5cdb480 x1875503571418624/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:10/0 lens 488/440 e 0 to 0 dl 1788619925 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619914.443935:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5cdb480 with x1875503571418624 00000100:00100000:2.0:1788619914.446296:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619914.446402:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571418752 00000100:00100000:3.0:1788619914.446412:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571418752 00002000:00100000:3.0:1788619914.446455:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79010ffb80 x1875503571418752/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:10/0 lens 488/0 e 0 to 0 dl 1788619925 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619914.446467:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79010ffb80 with x1875503571418752 00000100:00100000:3.0:1788619914.446514:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 565 00000100:00100000:3.0:1788619914.446524:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010ffb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571418752:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619914.446944:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010ffb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571418752:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 421us (656us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619914.446952:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 565 00002000:00100000:3.0:1788619914.446960:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79010ffb80 x1875503571418752/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:10/0 lens 488/440 e 0 to 0 dl 1788619925 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619914.446968:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79010ffb80 with x1875503571418752 00000100:00100000:1.0:1788619914.449643:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619914.449861:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571418880 00000100:00100000:2.0:1788619914.449871:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571418880 00002000:00100000:2.0:1788619914.449921:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5cdbb80 x1875503571418880/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:10/0 lens 488/0 e 0 to 0 dl 1788619925 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619914.449930:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5cdbb80 with x1875503571418880 00000100:00100000:2.0:1788619914.449966:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 566 00000100:00100000:2.0:1788619914.449975:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cdbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571418880:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619914.450412:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cdbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571418880:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 436us (778us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619914.450420:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 566 00002000:00100000:2.0:1788619914.450428:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5cdbb80 x1875503571418880/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:10/0 lens 488/440 e 0 to 0 dl 1788619925 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619914.450436:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5cdbb80 with x1875503571418880 00000100:00100000:1.0:1788619914.453248:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619914.453350:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571419008 00000100:00100000:0.0:1788619914.453360:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571419008 00002000:00100000:0.0:1788619914.453406:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5cda680 x1875503571419008/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:10/0 lens 488/0 e 0 to 0 dl 1788619925 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619914.453416:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5cda680 with x1875503571419008 00000100:00100000:0.0:1788619914.453507:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 567 00000100:00100000:0.0:1788619914.453516:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cda680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571419008:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619914.454012:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cda680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571419008:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 495us (773us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619914.454019:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 567 00002000:00100000:0.0:1788619914.454027:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5cda680 x1875503571419008/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:10/0 lens 488/440 e 0 to 0 dl 1788619925 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619914.454034:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5cda680 with x1875503571419008 00000100:00100000:2.0:1788619914.456835:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619914.456929:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571419136 00000100:00100000:1.0:1788619914.456938:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571419136 00000100:00100000:1.0:1788619914.456997:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 685 00000100:00100000:1.0:1788619914.457008:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010ff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571419136:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619914.457024:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619914.457042:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bad85f32/0x1cd7fe5368734739 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bbc8 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619914.457111:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cd9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589097344:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619914.457197:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c600 going to sleep for 11 seconds 00000100:00100000:3.0:1788619914.459372:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cd9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589097344:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619914.459412:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bad85f32/0x1cd7fe5368734739 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bbc8 expref: 8 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.459436:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bad85f32/0x1cd7fe5368734739 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bbc8 expref: 8 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.459451:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78df191800, rc 0) 00000100:00100000:3.0:1788619914.459510:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010ff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571419136:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2503us (2692us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619914.459518:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 685 00000100:00100000:3.0:1788619914.462590:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619914.462779:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571419264 00000100:00100000:2.0:1788619914.462789:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571419264 00000100:00100000:2.0:1788619914.462859:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 686 00000100:00100000:2.0:1788619914.462870:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571419264:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619914.462888:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619914.462914:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d6587d69/0x1cd7fe5368734740 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bbcf expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619914.462973:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589097472:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619914.463053:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccb40 going to sleep for 11 seconds 00000100:00100000:3.0:1788619914.466723:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589097472:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619914.466765:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d6587d69/0x1cd7fe5368734740 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bbcf expref: 8 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.466813:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d6587d69/0x1cd7fe5368734740 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bbcf expref: 8 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.466837:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079800, rc 0) 00000100:00100000:3.0:1788619914.466913:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571419264:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4036us (4324us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619914.466922:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 686 00000100:00100000:1.0:1788619914.470537:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619914.470633:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571419392 00000100:00100000:2.0:1788619914.470667:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571419392 00000100:00100000:2.0:1788619914.470761:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 687 00000100:00100000:2.0:1788619914.470771:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571419392:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619914.470789:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619914.470806:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5a362fc/0x1cd7fe5368734747 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bbd6 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619914.470875:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589097600:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619914.470956:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccc00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619914.476597:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589097600:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619914.476639:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a5a362fc/0x1cd7fe5368734747 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bbd6 expref: 8 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.476784:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5a362fc/0x1cd7fe5368734747 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bbd6 expref: 8 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.476799:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079600, rc 0) 00000100:00100000:2.0:1788619914.476916:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571419392:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6145us (6388us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619914.476925:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 687 00000100:00100000:0.0:1788619914.480998:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619914.481145:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571419520 00000100:00100000:2.0:1788619914.481156:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571419520 00000100:00100000:2.0:1788619914.481246:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 688 00000100:00100000:2.0:1788619914.481257:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d57b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571419520:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619914.481275:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619914.481304:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000855993/0x1cd7fe536873474e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bbdd expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619914.481396:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589097728:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619914.481476:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccc00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619914.483630:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589097728:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619914.483765:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000000855993/0x1cd7fe536873474e lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bbdd expref: 8 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.483791:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000855993/0x1cd7fe536873474e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bbdd expref: 8 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.483803:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079c00, rc 0) 00000100:00100000:2.0:1788619914.483919:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d57b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571419520:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2664us (2930us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619914.483926:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 688 00000100:00100000:1.0:1788619914.488786:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619914.488886:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571419648 00000100:00100000:3.0:1788619914.488897:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571419648 00000100:00100000:3.0:1788619914.489226:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 904 00000100:00100000:3.0:1788619914.489237:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cdb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:14908:x1875503571419648:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619914.489257:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619914.489260:0:5844:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1788619914.489302:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df191600/0x1cd7fe536873472b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bbba expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.489321:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619914.489364:0:5844:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788619914.489379:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368734755 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bbe4 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.489405:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368734755 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bbe4 expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.489417:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368734755 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bbe4 expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.489443:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619914.489446:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619914.489448:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c000, rc 0) 00000100:00100000:3.0:1788619914.489502:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cdb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:14908:x1875503571419648:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 266us (724us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619914.489509:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 904 00000100:00100000:2.0:1788619914.492052:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619914.492194:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571419776 00000100:00100000:0.0:1788619914.492204:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571419776 00000100:00100000:0.0:1788619914.492298:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 568 00000100:00100000:0.0:1788619914.492310:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010fce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571419776:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:0.0:1788619914.492670:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 929792 -> 929792 00000004:00010000:0.0:1788619914.492675:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619914 -> 1788619914 00000004:00010000:0.0:1788619914.492678:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619914 -> 1788619914 00000004:00010000:0.0:1788619914.492680:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619914 -> 1788619914 00000004:00010000:0.0:1788619914.492682:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1414 -> 1414 00000100:00100000:0.0:1788619914.492796:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010fce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571419776:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 489us (752us total) trans 4294967647 rc 0/0 00000100:00100000:0.0:1788619914.492804:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 568 00000100:00100000:0.0:1788619914.496424:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619914.496494:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571419904 00000100:00100000:3.0:1788619914.496503:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571419904 00000100:00100000:3.0:1788619914.496580:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 905 00000100:00100000:3.0:1788619914.496590:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d5c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571419904:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619914.496610:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619914.496630:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe536873475c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bbeb expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.496776:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368734755 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bbe4 expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619914.496817:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a8b100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589097856:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619914.496907:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164600 going to sleep for 11 seconds 00000100:00100000:0.0:1788619914.498747:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a8b100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589097856:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619914.498770:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 929792 00010000:00010000:0.0:1788619914.498787:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe536873475c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bbeb expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.498806:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619914.498809:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619914.498814:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe536873475c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bbeb expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.498837:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683dc00, rc 0) 00000100:00100000:0.0:1788619914.498926:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d5c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571419904:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2336us (2605us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619914.498933:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 905 00000100:00100000:2.0:1788619914.501512:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619914.501882:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571420032 00000100:00100000:1.0:1788619914.501892:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571420032 00000100:00100000:2.0:1788619914.502031:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 906 00000100:00100000:2.0:1788619914.502041:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010fd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571420032:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619914.502062:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619914.502086:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368734763 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bbf2 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.502133:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368734755 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bbe4 expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619914.502182:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010ff480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589097984:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619914.502257:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccc00 going to sleep for 11 seconds 00000100:00100000:3.0:1788619914.504774:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010ff480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589097984:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619914.504803:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 929792 00010000:00010000:3.0:1788619914.504822:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368734763 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bbf2 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.504852:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619914.504871:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619914.504877:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368734763 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bbf2 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.504892:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079400, rc 0) 00000100:00100000:3.0:1788619914.505136:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010fd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571420032:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3095us (3633us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619914.505146:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 906 00000100:00100000:2.0:1788619914.507964:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619914.508040:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571420160 00000100:00100000:3.0:1788619914.508050:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571420160 00000100:00100000:3.0:1788619914.508184:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 689 00000100:00100000:3.0:1788619914.508196:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010ff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571420160:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619914.508213:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619914.508230:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f980cda4/0x1cd7fe536873476a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bbf9 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619914.508288:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a89500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589098112:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619914.508470:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164e40 going to sleep for 11 seconds 00000100:00100000:3.0:1788619914.510718:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a89500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589098112:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619914.510753:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f980cda4/0x1cd7fe536873476a lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bbf9 expref: 8 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.510776:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f980cda4/0x1cd7fe536873476a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bbf9 expref: 8 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.510789:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d200, rc 0) 00000100:00100000:3.0:1788619914.510916:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010ff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571420160:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2709us (3003us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619914.510925:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 689 00000100:00100000:2.0:1788619914.513894:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619914.514015:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571420288 00000100:00100000:3.0:1788619914.514025:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571420288 00000100:00100000:3.0:1788619914.514157:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 690 00000100:00100000:3.0:1788619914.514168:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571420288:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619914.514187:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619914.514204:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c4fc38e/0x1cd7fe5368734771 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bc00 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619914.514260:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a89500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589098240:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619914.514444:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164e40 going to sleep for 11 seconds 00000100:00100000:3.0:1788619914.516788:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a89500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589098240:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619914.516823:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004c4fc38e/0x1cd7fe5368734771 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc00 expref: 8 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.516857:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c4fc38e/0x1cd7fe5368734771 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc00 expref: 8 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.516870:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683ce00, rc 0) 00000100:00100000:3.0:1788619914.517035:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571420288:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2867us (3148us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619914.517043:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 690 00000100:00100000:0.0:1788619914.519939:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619914.520058:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571420416 00000100:00100000:2.0:1788619914.520067:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571420416 00000100:00100000:2.0:1788619914.520294:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 691 00000100:00100000:2.0:1788619914.520306:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d5ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571420416:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619914.520343:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619914.520360:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000855993/0x1cd7fe5368734778 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bc07 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619914.520620:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5ff7800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589098368:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619914.520764:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccd80 going to sleep for 11 seconds 00000100:00100000:0.0:1788619914.522574:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5ff7800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589098368:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619914.522624:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000000855993/0x1cd7fe5368734778 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc07 expref: 8 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.522649:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000855993/0x1cd7fe5368734778 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc07 expref: 8 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.522676:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079c00, rc 0) 00000100:00100000:0.0:1788619914.522734:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d5ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571420416:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2431us (2804us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619914.522742:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 691 00000100:00100000:3.0:1788619914.526015:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619914.526132:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571420544 00000100:00100000:2.0:1788619914.526141:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571420544 00000100:00100000:2.0:1788619914.526197:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 692 00000100:00100000:2.0:1788619914.526208:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a89500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571420544:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619914.526225:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619914.526243:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dfdc9508/0x1cd7fe536873477f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bc0e expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619914.526304:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5ff7100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589098496:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619914.526418:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619914.528506:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5ff7100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589098496:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619914.528543:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000dfdc9508/0x1cd7fe536873477f lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc0e expref: 8 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.528566:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dfdc9508/0x1cd7fe536873477f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc0e expref: 8 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.528579:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8078200, rc 0) 00000100:00100000:0.0:1788619914.528724:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a89500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571420544:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2516us (2715us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619914.528734:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 692 00000100:00100000:0.0:1788619914.531697:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619914.531807:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571420672 00000100:00100000:3.0:1788619914.531816:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571420672 00000100:00100000:3.0:1788619914.531880:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 693 00000100:00100000:3.0:1788619914.531890:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c08e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+7:2352:x1875503571420672:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619914.531907:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619914.531926:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c4fc38e/0x1cd7fe5368734786 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bc15 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619914.531981:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a8bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589098624:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619914.532063:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164e40 going to sleep for 11 seconds 00000100:00100000:1.0:1788619914.533880:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a8bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589098624:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619914.533919:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004c4fc38e/0x1cd7fe5368734786 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc15 expref: 8 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.533940:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c4fc38e/0x1cd7fe5368734786 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc15 expref: 8 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.533953:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683ce00, rc 0) 00000100:00100000:1.0:1788619914.533997:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c08e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+7:2352:x1875503571420672:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2108us (2310us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619914.534005:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 693 00000100:00100000:2.0:1788619914.537113:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619914.537232:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571420800 00000100:00100000:3.0:1788619914.537240:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571420800 00000100:00100000:3.0:1788619914.537296:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 694 00000100:00100000:3.0:1788619914.537307:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571420800:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619914.537356:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619914.537374:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe536873478d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bc1c expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619914.537431:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a89880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589098752:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619914.537512:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164780 going to sleep for 11 seconds 00000100:00100000:3.0:1788619914.539843:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a89880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589098752:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619914.539875:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe536873478d lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc1c expref: 8 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.539898:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe536873478d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc1c expref: 8 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.539930:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c200, rc 0) 00000100:00100000:3.0:1788619914.539977:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571420800:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2672us (2883us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619914.539986:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 694 00000100:00100000:2.0:1788619914.543039:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619914.543114:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571421056 00000100:00100000:3.0:1788619914.543122:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571421056 00000100:00100000:3.0:1788619914.543173:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 907 00000100:00100000:3.0:1788619914.543182:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571421056:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:3.0:1788619914.543234:0:5844:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619914.543258:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06c00/0x1cd7fe5368734660 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1baf6 expref: 11 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.543275:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619914.543279:0:5844:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788619914.543490:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368734794 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.543503:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe53687346ad lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb3c expref: 11 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.543512:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368734794 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.543539:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe53687346ad lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb3c expref: 11 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.543568:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe53687346ad lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb3c expref: 11 pid: 5844 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619914.543582:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a89880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589098880:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619914.543644:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164780 going to sleep for 11 seconds 00000100:00100000:3.0:1788619914.545878:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a89880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589098880:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619914.545916:0:5844:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368734794 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619914.547181:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619914.547266:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571421184 00000100:00100000:0.0:1788619914.547273:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571421184 00010000:00010000:0.0:1788619914.547311:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe53687346ad lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb3c expref: 12 pid: 5844 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619914.547393:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 306 00000100:00100000:0.0:1788619914.547403:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8023050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571421184:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619914.547417:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619914.547424:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe53687346ad lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb3c expref: 12 pid: 5844 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.547435:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe53687346ad lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb3c expref: 12 pid: 5844 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.547456:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe53687346ad lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb3c expref: 12 pid: 5844 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.547471:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619914.547476:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368734794 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.547487:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368734794 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.547495:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368734794 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.547518:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619914.547569:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8023050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+11:2354:x1875503571421184:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 167us (391us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619914.547576:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 306 00010000:00010000:3.0:1788619914.547717:0:5844:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368734794 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.547725:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368734794 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788619914.547837:0:5844:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967648 00000004:00010000:3.0:1788619914.547881:0:5844:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 929792 -> 929792 00000004:00010000:3.0:1788619914.547884:0:5844:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619914 -> 1788619914 00000004:00010000:3.0:1788619914.547886:0:5844:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619914 -> 1788619914 00000004:00010000:3.0:1788619914.547888:0:5844:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619914 -> 1788619914 00000004:00010000:3.0:1788619914.547890:0:5844:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1414 -> 1414 00000004:00010000:3.0:1788619914.547896:0:5844:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000059384ec9 reply state 00000000e173fcf0 transno 4294967649 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368734794 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619914.547913:0:5844:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788619914.547972:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+13:14908:x1875503571421056:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 4791us (4941us total) trans 4294967649 rc 0/0 00000100:00100000:3.0:1788619914.547988:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 907 00000020:00080000:3.0:1788619914.547993:0:5844:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788619914.549939:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368734794 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.549964:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368734794 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.549971:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368734794 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.549977:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c683d200 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788619914.549981:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368734794 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.549989:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368734794 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.550004:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619914.550006:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788619914.550011:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368734794 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.550017:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368734794 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619914.551586:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619914.551666:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571421312 00000100:00100000:0.0:1788619914.551676:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571421312 00000100:00100000:0.0:1788619914.551732:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 695 00000100:00100000:0.0:1788619914.551742:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:14908:x1875503571421312:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619914.551759:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619914.551776:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d412ceb8/0x1cd7fe536873479b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bc23 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.551807:0:6696:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000003ff7ffc7/0x1cd7fe53687346ec lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1527808->1581055) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb7b expref: 16 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.551835:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=1s v=(8 8) 00010000:00010000:0.0:1788619914.551840:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(9 8) 00010000:00010000:0.0:1788619914.551847:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d412ceb8/0x1cd7fe536873479b lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc23 expref: 8 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.551879:0:6696:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000003ff7ffc7/0x1cd7fe53687346ec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1527808->1581055) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb7b expref: 16 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.551893:0:6696:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000003ff7ffc7/0x1cd7fe53687346ec lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1527808->1581055) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb7b expref: 16 pid: 7654 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619914.551918:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c0bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589099008:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619914.551997:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa240 going to sleep for 11 seconds 00000100:00100000:2.0:1788619914.554996:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619914.555080:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571421440 00000100:00100000:1.0:1788619914.555088:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571421440 00002000:00100000:1.0:1788619914.555143:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1527808->2025262) req@ffff8e78df25e850 x1875503571421440/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 504/0 e 0 to 0 dl 1788619925 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619914.555155:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df25e850 with x1875503571421440 ext(1527808->2025262) 00010000:00010000:1.0:1788619914.555164:0:6701:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000003ff7ffc7/0x1cd7fe53687346ec lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1527808->1581055) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb7b expref: 17 pid: 7654 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.555180:0:6701:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000003ff7ffc7/0x1cd7fe53687346ec lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1527808->1581055) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb7b expref: 17 pid: 7654 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788619914.555188:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff8e78df25e850 00000100:00100000:1.0:1788619914.555217:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 14 00000100:00100000:1.0:1788619914.555226:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df25e850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2354:x1875503571421440:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:0.0:1788619914.555855:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c0bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589099008:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619914.555885:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d412ceb8/0x1cd7fe536873479b lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc23 expref: 8 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.555911:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767ba00, rc 0) 00000100:00100000:0.0:1788619914.555955:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+8:14908:x1875503571421312:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4214us (4375us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619914.555962:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 695 00000100:00100000:1.0:1788619914.561458:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df25e850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2354:x1875503571421440:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 6222us (6463us total) trans 4294967405 rc 0/0 00000100:00100000:1.0:1788619914.561472:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 14 00002000:00100000:1.0:1788619914.561484:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1527808->2025262) req@ffff8e78df25e850 x1875503571421440/t4294967405(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 504/448 e 0 to 0 dl 1788619925 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619914.561494:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df25e850 with x1875503571421440 ext(1527808->2025262) 00010000:00010000:1.0:1788619914.561504:0:6701:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000003ff7ffc7/0x1cd7fe53687346ec lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1527808->1581055) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb7b expref: 19 pid: 7654 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.561522:0:6701:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000003ff7ffc7/0x1cd7fe53687346ec lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1527808->1581055) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb7b expref: 19 pid: 7654 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788619914.561542:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff8e78df25e850 00000100:00100000:3.0:1788619914.567643:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619914.567795:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571421568 00000100:00100000:2.0:1788619914.567805:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571421568 00010000:00010000:2.0:1788619914.567853:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000003ff7ffc7/0x1cd7fe53687346ec lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1527808->1581055) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb7b expref: 19 pid: 7654 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619914.567899:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 307 00000100:00100000:2.0:1788619914.567920:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a89880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2352:x1875503571421568:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619914.567936:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619914.567959:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000003ff7ffc7/0x1cd7fe53687346ec lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1527808->1581055) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb7b expref: 19 pid: 7654 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.567973:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000003ff7ffc7/0x1cd7fe53687346ec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1527808->1581055) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb7b expref: 19 pid: 7654 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.568000:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ff7ffc7/0x1cd7fe53687346ec lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1527808->1581055) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bb7b expref: 19 pid: 7654 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.568018:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(9 8) 00010000:00010000:2.0:1788619914.568022:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(9 8) 00010000:00010000:2.0:1788619914.568031:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000d412ceb8/0x1cd7fe536873479b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc23 expref: 7 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.568060:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d412ceb8/0x1cd7fe536873479b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc23 expref: 7 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.568111:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000d412ceb8/0x1cd7fe536873479b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc23 expref: 7 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619914.568126:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5ff4000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589099136:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619914.568216:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc840 going to sleep for 11 seconds 00000100:00100000:2.0:1788619914.571440:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619914.571468:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619914.571570:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5ff4000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589099136:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619914.571593:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619914.571661:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a89880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2352:x1875503571421568:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3742us (4027us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619914.571673:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 307 00000100:00100000:0.0:1788619914.572062:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571421696 00000100:00100000:0.0:1788619914.572070:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571421696 00002000:00010000:0.0:1788619914.572126:0:5857:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:0.0:1788619914.572131:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5ff5880 with x1875503571421696 00000100:00100000:0.0:1788619914.572166:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 569 00000100:00100000:0.0:1788619914.572176:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571421696:12345-192.168.201.5@tcp:10:fsx.0 00000004:00010000:0.0:1788619914.572506:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 929792 -> 172969 00000004:00010000:0.0:1788619914.572510:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619914 -> 1788619914 00000004:00010000:0.0:1788619914.572513:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619914 -> 1788619914 00000004:00010000:0.0:1788619914.572515:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619914 -> 1788619914 00000004:00010000:0.0:1788619914.572517:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1414 -> 1414 00000100:00100000:0.0:1788619914.572560:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571421696:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 386us (1130us total) trans 4294967650 rc 0/0 00000100:00100000:0.0:1788619914.572566:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 569 00002000:00010000:0.0:1788619914.572569:0:5857:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:0.0:1788619914.572571:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5ff5880 with x1875503571421696 00000100:00100000:3.0:1788619914.573028:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571421824 00000100:00100000:3.0:1788619914.573033:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571421824 00002000:00010000:3.0:1788619914.573056:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:3.0:1788619914.573060:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5ff4a80 with x1875503571421824 ext(0->18446744073709551615) 00002000:00010000:3.0:1788619914.573068:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5ff4a80. 00000100:00100000:3.0:1788619914.573096:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 363 00000100:00100000:3.0:1788619914.573104:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+8:2354:x1875503571421824:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:3.0:1788619914.573311:0:6703:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 2025263 -> 0 00002000:00010000:3.0:1788619914.573314:0:6703:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619914 -> 1788619914 00002000:00010000:3.0:1788619914.573317:0:6703:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619914 -> 1788619913 00002000:00010000:3.0:1788619914.573319:0:6703:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619914 -> 1788619914 00002000:00010000:3.0:1788619914.573322:0:6703:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 135 -> 135 00000100:00100000:3.0:1788619914.573391:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571421824:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 287us (1921us total) trans 4294967406 rc 0/0 00000100:00100000:3.0:1788619914.573398:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 363 00002000:00010000:3.0:1788619914.573401:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:3.0:1788619914.573403:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5ff4a80 with x1875503571421824 ext(0->18446744073709551615) 00002000:00010000:3.0:1788619914.573408:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5ff4a80. 00000100:00100000:2.0:1788619914.576895:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619914.576984:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571421952 00000100:00100000:0.0:1788619914.576994:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571421952 00000100:00100000:0.0:1788619914.577050:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 908 00000100:00100000:0.0:1788619914.577061:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+13:14908:x1875503571421952:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619914.577083:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619914.577114:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767b000/0x1cd7fe53687347a2 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bc2a expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.577199:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767ac00/0x1cd7fe53687347a9 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.577214:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767ac00/0x1cd7fe53687347a9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.577243:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767ac00/0x1cd7fe53687347a9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619914.577271:0:9417:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 172969 00000004:00200000:0.0:1788619914.577275:0:9417:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619914.577293:0:9417:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767ac00/0x1cd7fe53687347a9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.577318:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767b000/0x1cd7fe53687347a2 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc2a expref: 15 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.577377:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767ac00/0x1cd7fe53687347a9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc2a expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.577386:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619914.577388:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619914.577392:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767ac00, rc 0) 00000100:00100000:0.0:1788619914.577440:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+14:14908:x1875503571421952:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 381us (555us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619914.577446:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 908 00000100:00100000:0.0:1788619914.580971:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619914.581047:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571422080 00000100:00100000:3.0:1788619914.581057:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571422080 00000100:00100000:3.0:1788619914.581119:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 909 00000100:00100000:3.0:1788619914.581129:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c0bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+11:14908:x1875503571422080:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619914.581167:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619914.581187:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe53687347b0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bc31 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.581269:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe53687347b7 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.581283:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe53687347b7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.581295:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe53687347b7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788619914.581322:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 172969 00000004:00200000:3.0:1788619914.581364:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788619914.581384:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe53687347b7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.581409:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe53687347b0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc31 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.581427:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe53687347b7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc31 expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.581436:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619914.581439:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788619914.581442:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c200, rc 0) 00000100:00100000:3.0:1788619914.581488:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c0bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+12:14908:x1875503571422080:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 361us (527us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619914.581495:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 909 00000100:00100000:1.0:1788619914.584423:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619914.584583:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571422208 00000100:00100000:3.0:1788619914.584592:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571422208 00000100:00100000:3.0:1788619914.584657:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 910 00000100:00100000:3.0:1788619914.584667:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571422208:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619914.584686:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619914.584706:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe53687347be lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bc38 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.584741:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368734755 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bbe4 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619914.584782:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a8a680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589099264:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619914.584871:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164780 going to sleep for 11 seconds 00000100:00100000:2.0:1788619914.586396:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619914.586476:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571422336 00000100:00100000:0.0:1788619914.586482:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571422336 00000100:00100000:0.0:1788619914.586522:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 696 00000100:00100000:0.0:1788619914.586531:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2354:x1875503571422336:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619914.586543:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619914.586557:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000072940acc/0x1cd7fe53687347c5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bc3f expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619914.586620:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c08700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589099392:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619914.586690:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa840 going to sleep for 11 seconds 00000100:00100000:3.0:1788619914.588794:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a8a680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589099264:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619914.588820:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 172969 00010000:00010000:3.0:1788619914.588849:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe53687347be lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc38 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.588869:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619914.588872:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619914.588877:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe53687347be lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc38 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.588891:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c800, rc 0) 00000100:00100000:3.0:1788619914.588947:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571422208:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4281us (4533us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619914.588955:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 910 00002000:00010000:0.0:1788619914.589056:0:6698:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619913 -> 1788619914 00000100:00100000:0.0:1788619914.589072:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c08700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589099392:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619914.589089:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000072940acc/0x1cd7fe53687347c5 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc3f expref: 19 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.589117:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000072940acc/0x1cd7fe53687347c5 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc3f expref: 19 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.589126:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767b000, rc 0) 00000100:00100000:0.0:1788619914.589149:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2354:x1875503571422336:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2619us (2757us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619914.589155:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 696 00000100:00100000:1.0:1788619914.592321:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619914.592494:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619914.592563:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571422592 00000100:00100000:3.0:1788619914.592573:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571422592 00000100:00100000:3.0:1788619914.592626:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 697 00000100:00100000:3.0:1788619914.592636:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2355:x1875503571422592:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619914.592662:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619914.592682:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe53687347cc lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bc4d expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619914.592741:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a8a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589099520:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619914.592845:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164780 going to sleep for 11 seconds 00000100:00100000:2.0:1788619914.593634:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571422464 00000100:00100000:2.0:1788619914.593641:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571422464 00000100:00100000:2.0:1788619914.593770:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 911 00000100:00100000:2.0:1788619914.593779:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cdad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571422464:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619914.593792:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619914.593806:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe53687347d3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bc46 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.593845:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368734755 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bbe4 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619914.593873:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5ff4380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589099648:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619914.593941:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc600 going to sleep for 11 seconds 00000100:00100000:2.0:1788619914.596860:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a8a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589099520:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619914.596897:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe53687347cc lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc4d expref: 19 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.596920:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe53687347cc lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc4d expref: 19 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.596932:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683ca00, rc 0) 00000100:00100000:2.0:1788619914.596989:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2355:x1875503571422592:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4353us (4493us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619914.596998:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 697 00000100:00100000:0.0:1788619914.597951:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5ff4380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589099648:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619914.597970:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 172969 00010000:00010000:0.0:1788619914.597984:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe53687347d3 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc46 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.598000:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619914.598003:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619914.598008:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe53687347d3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc46 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.598039:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079000, rc 0) 00000100:00100000:0.0:1788619914.598082:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cdad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571422464:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4304us (5770us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619914.598089:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 911 00000100:00100000:1.0:1788619914.601808:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619914.601912:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571422720 00000100:00100000:3.0:1788619914.601922:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571422720 00000100:00100000:3.0:1788619914.601988:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 912 00000100:00100000:3.0:1788619914.601998:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cdb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571422720:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619914.602020:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619914.602043:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe53687347da lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bc54 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.602081:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368734755 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bbe4 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619914.602133:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a8ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589099776:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619914.602212:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164900 going to sleep for 11 seconds 00000100:00100000:2.0:1788619914.604119:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619914.604298:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a8ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589099776:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619914.604322:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 172969 00010000:00010000:1.0:1788619914.604370:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe53687347da lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc54 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.604389:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619914.604392:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619914.604398:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe53687347da lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc54 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.604412:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c800, rc 0) 00000100:00100000:1.0:1788619914.604457:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cdb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571422720:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2460us (2660us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619914.604466:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 912 00000100:00100000:3.0:1788619914.604851:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571422848 00000100:00100000:3.0:1788619914.604858:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571422848 00000100:00100000:3.0:1788619914.604912:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 698 00000100:00100000:3.0:1788619914.604919:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2354:x1875503571422848:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619914.604945:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619914.604961:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe53687347e1 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bc5b expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619914.605008:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a8ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589099904:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619914.605070:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164e40 going to sleep for 11 seconds 00000100:00100000:3.0:1788619914.608410:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a8ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589099904:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619914.608448:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe53687347e1 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc5b expref: 19 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.608471:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe53687347e1 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc5b expref: 19 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.608483:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683ca00, rc 0) 00000100:00100000:3.0:1788619914.608578:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2354:x1875503571422848:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3658us (4465us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619914.608587:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 698 00000100:00100000:1.0:1788619914.615752:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619914.615845:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619914.615989:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571422976 00000100:00100000:3.0:1788619914.616000:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571422976 00000100:00100000:3.0:1788619914.616054:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 913 00000100:00100000:3.0:1788619914.616064:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571422976:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619914.616084:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619914.616114:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687347e8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bc62 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.616164:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368734755 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bbe4 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619914.616201:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a8ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589100032:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619914.616286:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164e40 going to sleep for 11 seconds 00000100:00100000:1.0:1788619914.618609:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a8ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589100032:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619914.618635:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 172969 00010000:00010000:1.0:1788619914.618664:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687347e8 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc62 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.618684:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619914.618687:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619914.618692:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687347e8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc62 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.618706:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d200, rc 0) 00000100:00100000:1.0:1788619914.618762:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571422976:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2698us (3019us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619914.618771:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 913 00000100:00100000:0.0:1788619914.620285:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571423104 00000100:00100000:0.0:1788619914.620302:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571423104 00000100:00100000:1.0:1788619914.620568:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 699 00000100:00100000:1.0:1788619914.620580:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2354:x1875503571423104:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619914.620595:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619914.620613:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000059ec8ce2/0x1cd7fe53687347ef lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bc69 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619914.620679:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cd9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589100160:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619914.620767:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c000 going to sleep for 11 seconds 00000100:00100000:1.0:1788619914.622690:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cd9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589100160:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619914.622727:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000059ec8ce2/0x1cd7fe53687347ef lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc69 expref: 19 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.622771:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000059ec8ce2/0x1cd7fe53687347ef lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc69 expref: 19 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.622782:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78df190400, rc 0) 00000100:00100000:1.0:1788619914.622838:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2354:x1875503571423104:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2261us (6992us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619914.622847:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 699 00000100:00100000:1.0:1788619914.629731:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619914.629775:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619914.629853:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571423232 00000100:00100000:2.0:1788619914.629863:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571423232 00000100:00100000:2.0:1788619914.629930:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 914 00000100:00100000:2.0:1788619914.629941:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571423232:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619914.629961:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619914.629981:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe53687347f6 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bc70 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.630016:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368734755 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bbe4 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619914.630054:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5ff7480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589100288:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619914.630156:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619914.630572:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571423360 00000100:00100000:3.0:1788619914.630577:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571423360 00000100:00100000:3.0:1788619914.630608:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 700 00000100:00100000:3.0:1788619914.630616:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cda300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2354:x1875503571423360:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619914.630624:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619914.630635:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe53687347fd lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bc77 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619914.630671:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a8a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589100416:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619914.630711:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164600 going to sleep for 11 seconds 00000100:00100000:1.0:1788619914.633013:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a8a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589100416:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619914.633050:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe53687347fd lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc77 expref: 19 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.633074:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe53687347fd lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc77 expref: 19 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.633086:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683ca00, rc 0) 00000100:00100000:1.0:1788619914.633270:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cda300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2354:x1875503571423360:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2653us (3492us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619914.633279:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 700 00000100:00100000:3.0:1788619914.633392:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5ff7480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589100288:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619914.633407:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 172969 00010000:00010000:3.0:1788619914.633420:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe53687347f6 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc70 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.633433:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619914.633436:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619914.633440:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe53687347f6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc70 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.633468:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079000, rc 0) 00000100:00100000:3.0:1788619914.633489:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571423232:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3550us (3769us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619914.633495:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 914 00000100:00100000:3.0:1788619914.635979:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619914.636018:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619914.636085:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571423488 00000100:00100000:1.0:1788619914.636104:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571423488 00000100:00100000:1.0:1788619914.636163:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 915 00000100:00100000:1.0:1788619914.636186:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a89180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571423488:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619914.636205:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619914.636227:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190e00/0x1cd7fe5368734804 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bc7e expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.636262:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368734755 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bbe4 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619914.636304:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5697800 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589100544:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619914.636406:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c000 going to sleep for 11 seconds 00000100:00100000:2.0:1788619914.636841:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571423616 00000100:00100000:2.0:1788619914.636847:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571423616 00000100:00100000:2.0:1788619914.636888:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 701 00000100:00100000:2.0:1788619914.636896:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a89f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2354:x1875503571423616:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619914.636906:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619914.636917:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000855993/0x1cd7fe536873480b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bc85 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619914.636955:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5ff7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589100672:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619914.636994:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccf00 going to sleep for 11 seconds 00000100:00100000:1.0:1788619914.639264:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5697800 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589100544:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619914.639292:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 172969 00010000:00010000:1.0:1788619914.639313:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190e00/0x1cd7fe5368734804 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc7e expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.639364:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619914.639368:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619914.639375:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190e00/0x1cd7fe5368734804 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc7e expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.639389:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78df190e00, rc 0) 00000100:00100000:1.0:1788619914.639444:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a89180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571423488:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3259us (3471us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619914.639453:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 915 00000100:00100000:2.0:1788619914.639558:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5ff7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589100672:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619914.639578:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000000855993/0x1cd7fe536873480b lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc85 expref: 19 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.639594:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000855993/0x1cd7fe536873480b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc85 expref: 19 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.639604:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079c00, rc 0) 00000100:00100000:2.0:1788619914.639626:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a89f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2354:x1875503571423616:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2732us (3608us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619914.639632:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 701 00000100:00100000:3.0:1788619914.643532:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619914.643634:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571423744 00000100:00100000:1.0:1788619914.643643:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571423744 00000100:00100000:1.0:1788619914.643701:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 702 00000100:00100000:1.0:1788619914.643724:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a8b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+18:14908:x1875503571423744:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619914.643745:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619914.643761:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000585a0430/0x1cd7fe5368734812 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bc8c expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.643793:0:6698:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000d412ceb8/0x1cd7fe536873479b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc23 expref: 8 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.643802:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(9 8) 00010000:00010000:1.0:1788619914.643807:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(10 8) 00010000:00010000:1.0:1788619914.643814:0:6698:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000585a0430/0x1cd7fe5368734812 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1073152->1138687] (req 1073152->1138687) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc8c expref: 19 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.643861:0:6698:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000d412ceb8/0x1cd7fe536873479b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc23 expref: 8 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.643875:0:6698:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000d412ceb8/0x1cd7fe536873479b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc23 expref: 8 pid: 6696 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619914.643890:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5697800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589100800:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619914.643977:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c000 going to sleep for 11 seconds 00000100:00100000:3.0:1788619914.646949:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619914.647038:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5697800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589100800:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619914.647070:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000585a0430/0x1cd7fe5368734812 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1073152->1138687] (req 1073152->1138687) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc8c expref: 19 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.647121:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78df191200, rc 0) 00000100:00100000:1.0:1788619914.647177:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a8b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+19:14908:x1875503571423744:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3455us (3653us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619914.647186:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 702 00000100:00100000:0.0:1788619914.647312:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571423872 00000100:00100000:0.0:1788619914.647319:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571423872 00010000:00010000:0.0:1788619914.647385:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000d412ceb8/0x1cd7fe536873479b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc23 expref: 9 pid: 6696 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619914.647416:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 308 00000100:00100000:0.0:1788619914.647429:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a88e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571423872:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619914.647442:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619914.647462:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d412ceb8/0x1cd7fe536873479b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc23 expref: 9 pid: 6696 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.647475:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d412ceb8/0x1cd7fe536873479b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc23 expref: 9 pid: 6696 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.647497:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d412ceb8/0x1cd7fe536873479b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc23 expref: 9 pid: 6696 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.647510:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(10 8) 00010000:00010000:0.0:1788619914.647514:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(10 8) 00010000:00010000:0.0:1788619914.647529:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000585a0430/0x1cd7fe5368734812 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1073152->1138687) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc8c expref: 18 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.647539:0:5831:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000585a0430/0x1cd7fe5368734812 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1073152->1138687) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc8c expref: 18 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.647572:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000585a0430/0x1cd7fe5368734812 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1073152->1138687) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc8c expref: 18 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619914.647586:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c08700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5831:1875503589100928:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619914.647635:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa840 going to sleep for 11 seconds 00000100:00100000:3.0:1788619914.652150:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619914.652257:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c08700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5831:1875503589100928:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619914.652282:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619914.652377:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a88e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+8:2354:x1875503571423872:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 4951us (5435us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619914.652389:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 308 00000100:00100000:1.0:1788619914.652531:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571424000 00000100:00100000:1.0:1788619914.652538:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571424000 00000100:00100000:1.0:1788619914.652591:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 916 00000100:00100000:1.0:1788619914.652600:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a8aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571424000:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619914.652618:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619914.652636:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190e00/0x1cd7fe5368734819 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bc93 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.652683:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368734755 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bbe4 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619914.652720:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5697800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589101056:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619914.652775:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c000 going to sleep for 11 seconds 00000100:00100000:1.0:1788619914.655247:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5697800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589101056:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619914.655272:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 172969 00010000:00010000:1.0:1788619914.655290:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190e00/0x1cd7fe5368734819 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc93 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.655308:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619914.655311:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619914.655371:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190e00/0x1cd7fe5368734819 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc93 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.655385:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78df190e00, rc 0) 00000100:00100000:1.0:1788619914.655438:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a8aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571424000:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2839us (3296us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619914.655446:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 916 00000100:00100000:3.0:1788619914.658772:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619914.658878:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571424128 00000100:00100000:2.0:1788619914.658887:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571424128 00000100:00100000:2.0:1788619914.658948:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 917 00000100:00100000:2.0:1788619914.658958:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a89c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571424128:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619914.658993:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619914.659014:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe5368734820 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bc9a expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.659053:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368734755 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bbe4 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619914.659105:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5ff7b80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589101184:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619914.659192:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccf00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619914.661800:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5ff7b80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589101184:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619914.661839:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 172969 00010000:00010000:2.0:1788619914.661859:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe5368734820 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc9a expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.661881:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619914.661885:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619914.661890:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe5368734820 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc9a expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.661912:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079c00, rc 0) 00000100:00100000:2.0:1788619914.661972:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a89c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571424128:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3015us (3211us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619914.661982:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 917 00000100:00100000:0.0:1788619914.669982:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619914.670455:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571424384 00000100:00100000:1.0:1788619914.670467:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571424384 00000100:00100000:1.0:1788619914.670561:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 918 00000100:00100000:1.0:1788619914.670574:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c08700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+12:14908:x1875503571424384:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:1.0:1788619914.670625:0:11397:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619914.670664:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe53687347b7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc31 expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.670683:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619914.670688:0:11397:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788619914.670900:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190e00/0x1cd7fe5368734827 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.670929:0:11397:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767ac00/0x1cd7fe53687347a9 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc2a expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.670938:0:11397:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190e00/0x1cd7fe5368734827 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.670970:0:11397:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767ac00/0x1cd7fe53687347a9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc2a expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.671003:0:11397:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767ac00/0x1cd7fe53687347a9 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc2a expref: 13 pid: 9417 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619914.671018:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5697800 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589101312:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619914.671112:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c000 going to sleep for 11 seconds 00000100:00100000:3.0:1788619914.674754:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619914.674859:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571424512 00000100:00100000:2.0:1788619914.674869:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571424512 00010000:00010000:2.0:1788619914.674918:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767ac00/0x1cd7fe53687347a9 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc2a expref: 14 pid: 9417 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619914.674981:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 309 00000100:00100000:2.0:1788619914.674992:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a8bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+14:2354:x1875503571424512:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619914.675007:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619914.675014:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767ac00/0x1cd7fe53687347a9 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc2a expref: 14 pid: 9417 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.675027:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767ac00/0x1cd7fe53687347a9 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc2a expref: 14 pid: 9417 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.675049:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619914.675053:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190e00/0x1cd7fe5368734827 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.675062:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190e00/0x1cd7fe5368734827 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.675070:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190e00/0x1cd7fe5368734827 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.675087:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619914.675137:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a8bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+14:2354:x1875503571424512:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 147us (391us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619914.675144:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 309 00010000:00010000:1.0:1788619914.675699:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767ac00/0x1cd7fe53687347a9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc2a expref: 13 pid: 9417 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619914.675717:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5697800 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589101312:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619914.675735:0:11397:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190e00/0x1cd7fe5368734827 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.675743:0:11397:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190e00/0x1cd7fe5368734827 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.675748:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190e00/0x1cd7fe5368734827 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788619914.675867:0:11397:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967651 00000004:00010000:1.0:1788619914.676160:0:11397:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 172969 -> 172969 00000004:00010000:1.0:1788619914.676164:0:11397:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619914 -> 1788619914 00000004:00010000:1.0:1788619914.676166:0:11397:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619914 -> 1788619914 00000004:00010000:1.0:1788619914.676168:0:11397:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619914 -> 1788619914 00000004:00010000:1.0:1788619914.676169:0:11397:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1414 -> 1414 00000004:00010000:1.0:1788619914.676175:0:11397:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000004272db9 reply state 0000000020ba5be8 transno 4294967652 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190e00/0x1cd7fe5368734827 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619914.676195:0:11397:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788619914.676350:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c08700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571424384:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 5747us (6345us total) trans 4294967652 rc 0/0 00000100:00100000:1.0:1788619914.676357:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 918 00000020:00080000:1.0:1788619914.676361:0:11397:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1788619914.679567:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619914.679689:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571424640 00000100:00100000:1.0:1788619914.679700:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571424640 00000100:00100000:1.0:1788619914.679756:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 919 00000100:00100000:1.0:1788619914.679768:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a8b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571424640:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619914.679793:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619914.679814:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190400/0x1cd7fe536873482e lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bca1 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.679855:0:11397:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368734755 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bbe4 expref: 12 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.679867:0:11397:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190400/0x1cd7fe536873482e lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bca1 expref: 15 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.679918:0:11397:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368734755 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bbe4 expref: 12 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.679930:0:11397:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368734755 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bbe4 expref: 12 pid: 5844 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619914.679942:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5697800 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589101440:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619914.680022:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c000 going to sleep for 11 seconds 00000100:00100000:0.0:1788619914.682962:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5697800 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589101440:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619914.682993:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190400/0x1cd7fe536873482e lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bca1 expref: 15 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.683012:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619914.683015:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619914.683020:0:11397:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190400/0x1cd7fe536873482e lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bca1 expref: 15 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.683047:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78df190400, rc 0) 00000100:00100000:0.0:1788619914.683109:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a8b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+15:14908:x1875503571424640:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3344us (3551us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619914.683118:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 919 00000100:00100000:3.0:1788619914.683440:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619914.684313:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571424768 00000100:00100000:1.0:1788619914.684322:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571424768 00010000:00010000:1.0:1788619914.684371:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368734755 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bbe4 expref: 13 pid: 5844 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619914.684400:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 310 00000100:00100000:1.0:1788619914.684408:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a8a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571424768:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619914.684421:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619914.684444:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368734755 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bbe4 expref: 13 pid: 5844 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.684456:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368734755 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bbe4 expref: 13 pid: 5844 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.684477:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368734755 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bbe4 expref: 13 pid: 5844 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.684492:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619914.684497:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190400/0x1cd7fe536873482e lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bca1 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.684507:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190400/0x1cd7fe536873482e lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bca1 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.684515:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190400/0x1cd7fe536873482e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bca1 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.684563:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190400/0x1cd7fe536873482e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bca1 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619914.684575:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5696680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589101568:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619914.684634:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c900 going to sleep for 11 seconds 00000100:00100000:3.0:1788619914.688314:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5696680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589101568:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619914.688568:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619914.688623:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a8a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571424768:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 4214us (5182us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619914.688634:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 310 00010000:00010000:2.0:1788619914.688764:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190e00/0x1cd7fe5368734827 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.688777:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190e00/0x1cd7fe5368734827 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.688797:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190e00/0x1cd7fe5368734827 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.688804:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78df190e00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:2.0:1788619914.688810:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190e00/0x1cd7fe5368734827 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.688818:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190e00/0x1cd7fe5368734827 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.688846:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619914.688852:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1788619914.688852:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00010000:00010000:2.0:1788619914.688858:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190e00/0x1cd7fe5368734827 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.688866:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190e00/0x1cd7fe5368734827 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619914.689126:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571424896 00000100:00100000:3.0:1788619914.689133:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571424896 00002000:00010000:3.0:1788619914.689165:0:6701:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1875493->18446744073709551615) 00002000:00100000:3.0:1788619914.689169:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5696300 with x1875503571424896 ext(1875493->18446744073709551615) 00002000:00010000:3.0:1788619914.689179:0:6701:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5696300. 00000100:00100000:3.0:1788619914.689205:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 364 00000100:00100000:3.0:1788619914.689215:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5696300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2354:x1875503571424896:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:3.0:1788619914.689467:0:6701:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 0 -> 1875493 00002000:00010000:3.0:1788619914.689471:0:6701:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619914 -> 1788619914 00002000:00010000:3.0:1788619914.689473:0:6701:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619914 -> 1788619913 00002000:00010000:3.0:1788619914.689476:0:6701:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619914 -> 1788619914 00002000:00010000:3.0:1788619914.689478:0:6701:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 135 -> 135 00000100:00100000:3.0:1788619914.689510:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5696300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2354:x1875503571424896:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 296us (662us total) trans 4294967407 rc 0/0 00000100:00100000:3.0:1788619914.689517:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 364 00002000:00010000:3.0:1788619914.689520:0:6701:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1875493->18446744073709551615) 00002000:00100000:3.0:1788619914.689523:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5696300 with x1875503571424896 ext(1875493->18446744073709551615) 00002000:00010000:3.0:1788619914.689527:0:6701:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5696300. 00000100:00100000:1.0:1788619914.705798:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619914.705874:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619914.705888:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571425024 00000100:00100000:2.0:1788619914.705899:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571425024 00000100:00100000:0.0:1788619914.705941:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571425152 00000100:00100000:0.0:1788619914.705948:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571425152 00002000:00100000:2.0:1788619914.705961:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1875967) req@ffff8e78c5694700 x1875503571425024/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 504/0 e 0 to 0 dl 1788619925 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619914.705973:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5694700 with x1875503571425024 ext(1048576->1875967) 00002000:00010000:2.0:1788619914.705980:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5694700 00002000:00100000:0.0:1788619914.705981:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5697480 x1875503571425152/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 488/0 e 0 to 0 dl 1788619925 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619914.705992:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5697480 with x1875503571425152 00000100:00100000:2.0:1788619914.706016:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 365 00000100:00100000:2.0:1788619914.706027:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5694700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2352:x1875503571425024:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619914.706029:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 570 00000100:00100000:0.0:1788619914.706044:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5697480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2352:x1875503571425152:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619914.716906:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5697480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2352:x1875503571425152:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 10856us (11022us total) trans 0 rc 176128/176128 00000100:00100000:2.0:1788619914.716924:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5694700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2352:x1875503571425024:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 10895us (11132us total) trans 0 rc 761856/761856 00000100:00100000:0.0:1788619914.716932:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 570 00002000:00100000:0.0:1788619914.716945:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5697480 x1875503571425152/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 488/440 e 0 to 0 dl 1788619925 ref 1 fl Complete:/600/0 rc 176128/176128 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619914.716956:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5697480 with x1875503571425152 00000100:00100000:2.0:1788619914.717019:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 365 00002000:00100000:2.0:1788619914.717028:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1875967) req@ffff8e78c5694700 x1875503571425024/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 504/440 e 0 to 0 dl 1788619925 ref 1 fl Complete:/600/0 rc 761856/761856 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619914.717037:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5694700 with x1875503571425024 ext(1048576->1875967) 00002000:00010000:2.0:1788619914.717041:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5694700 00000100:00100000:3.0:1788619914.727685:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619914.727795:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571425280 00000100:00100000:1.0:1788619914.727804:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571425280 00002000:00100000:1.0:1788619914.727862:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1073152->1875492) req@ffff8e77c6a8ad80 x1875503571425280/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 504/0 e 0 to 0 dl 1788619925 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619914.727872:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a8ad80 with x1875503571425280 ext(1073152->1875492) 00002000:00010000:1.0:1788619914.727878:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a8ad80 00000100:00100000:1.0:1788619914.727909:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 366 00000100:00100000:1.0:1788619914.727918:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a8ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2354:x1875503571425280:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:2.0:1788619914.733893:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a8ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2354:x1875503571425280:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 5963us (6205us total) trans 4294967408 rc 0/0 00000100:00100000:2.0:1788619914.733911:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 366 00002000:00100000:2.0:1788619914.733927:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1073152->1875492) req@ffff8e77c6a8ad80 x1875503571425280/t4294967408(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 504/448 e 0 to 0 dl 1788619925 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619914.733938:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a8ad80 with x1875503571425280 ext(1073152->1875492) 00002000:00010000:2.0:1788619914.733951:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a8ad80 00000100:00100000:1.0:1788619914.739666:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619914.739774:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571425408 00000100:00100000:2.0:1788619914.739784:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571425408 00000100:00100000:2.0:1788619914.739847:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 703 00000100:00100000:2.0:1788619914.739857:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5695500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2354:x1875503571425408:12345-192.168.201.5@tcp:16:fsx.0 00000100:00100000:2.0:1788619914.769981:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5695500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2354:x1875503571425408:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 30115us (30313us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619914.769995:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 703 00000001:00080000:1.0:1788619914.770184:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967399 is committed 00000001:00080000:1.0:1788619914.770196:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967397 is committed 00000001:00080000:1.0:1788619914.770203:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967405 is committed 00000001:00080000:1.0:1788619914.770210:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967404 is committed 00000001:00080000:1.0:1788619914.770222:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967408 is committed 00000001:00080000:1.0:1788619914.770242:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967406 is committed 00000100:00100000:0.0:1788619914.772833:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619914.772941:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571425536 00000100:00100000:2.0:1788619914.772951:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571425536 00000100:00100000:2.0:1788619914.773040:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 920 00000100:00100000:2.0:1788619914.773051:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c0b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571425536:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619914.773071:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619914.773114:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368734835 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bca8 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.773207:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe536873483c lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.773221:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe536873483c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.773234:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe536873483c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788619914.773263:0:5846:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1875493 00000004:00200000:2.0:1788619914.773267:0:5846:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788619914.773284:0:5846:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe536873483c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.773310:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368734835 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bca8 expref: 16 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.773415:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe536873483c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bca8 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.773424:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619914.773427:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788619914.773431:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079c00, rc 0) 00000100:00100000:2.0:1788619914.773487:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c0b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:14908:x1875503571425536:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 439us (674us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619914.773494:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 920 00000100:00100000:3.0:1788619914.777312:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619914.777712:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571425664 00000100:00100000:0.0:1788619914.777723:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571425664 00000100:00100000:0.0:1788619914.777781:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 921 00000100:00100000:0.0:1788619914.777791:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+12:14908:x1875503571425664:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619914.777812:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619914.777859:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a200/0x1cd7fe5368734843 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bcaf expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.777942:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767be00/0x1cd7fe536873484a lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.777956:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767be00/0x1cd7fe536873484a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.777966:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767be00/0x1cd7fe536873484a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619914.777991:0:9417:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1875493 00000004:00200000:0.0:1788619914.777997:0:9417:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619914.778014:0:9417:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767be00/0x1cd7fe536873484a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.778038:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a200/0x1cd7fe5368734843 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bcaf expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.778054:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767be00/0x1cd7fe536873484a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bcaf expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.778061:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619914.778064:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619914.778068:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767be00, rc 0) 00000100:00100000:0.0:1788619914.778244:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+13:14908:x1875503571425664:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 453us (942us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619914.778251:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 921 00000100:00100000:3.0:1788619914.780697:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619914.780740:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619914.780910:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571425792 00000100:00100000:0.0:1788619914.780932:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571425792 00000100:00100000:0.0:1788619914.780983:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 922 00000100:00100000:0.0:1788619914.780992:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571425792:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619914.781014:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619914.781031:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a800/0x1cd7fe5368734851 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bcb6 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.781064:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190400/0x1cd7fe536873482e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bca1 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619914.781114:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c0aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589101696:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619914.781193:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa840 going to sleep for 11 seconds 00000100:00100000:0.0:1788619914.783057:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c0aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589101696:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619914.783079:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 172969 00010000:00010000:0.0:1788619914.783106:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a800/0x1cd7fe5368734851 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bcb6 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.783124:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619914.783128:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619914.783133:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a800/0x1cd7fe5368734851 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bcb6 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.783145:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767a800, rc 0) 00000100:00100000:0.0:1788619914.783193:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571425792:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2202us (2508us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619914.783201:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 922 00000100:00100000:2.0:1788619914.783803:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571425920 00000100:00100000:2.0:1788619914.783835:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571425920 00000100:00100000:2.0:1788619914.783875:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 704 00000100:00100000:2.0:1788619914.783885:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571425920:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619914.783898:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619914.783912:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008447b954/0x1cd7fe5368734858 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bcbd expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619914.783961:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5ff7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589101824:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619914.784023:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc9c0 going to sleep for 11 seconds 00002000:00010000:2.0:1788619914.786576:0:6697:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619913 -> 1788619914 00002000:00010000:2.0:1788619914.786584:0:6697:0:(ofd_lvb.c:284:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks: 135 -> 519 00000100:00100000:2.0:1788619914.786605:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5ff7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589101824:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619914.786637:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008447b954/0x1cd7fe5368734858 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bcbd expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.786671:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008447b954/0x1cd7fe5368734858 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bcbd expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.786684:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079000, rc 0) 00000100:00100000:2.0:1788619914.786738:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571425920:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2855us (5996us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619914.786746:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 704 00000100:00100000:0.0:1788619914.789621:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619914.789785:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571426048 00000100:00100000:3.0:1788619914.789795:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571426048 00000100:00100000:3.0:1788619914.789865:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 923 00000100:00100000:3.0:1788619914.789875:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c0aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571426048:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619914.789894:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619914.789913:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe536873485f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bcc4 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.789949:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190400/0x1cd7fe536873482e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bca1 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619914.790002:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900d5e680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589101952:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619914.790085:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164780 going to sleep for 11 seconds 00000100:00100000:3.0:1788619914.792791:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619914.792965:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900d5e680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589101952:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619914.792988:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 172969 00010000:00010000:0.0:1788619914.793007:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe536873485f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bcc4 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.793027:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619914.793030:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619914.793036:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe536873485f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bcc4 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.793049:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d000, rc 0) 00000100:00100000:0.0:1788619914.793116:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c0aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571426048:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3242us (3504us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619914.793124:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 923 00000100:00100000:2.0:1788619914.793286:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571426176 00000100:00100000:2.0:1788619914.793295:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571426176 00000100:00100000:2.0:1788619914.793376:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 705 00000100:00100000:2.0:1788619914.793385:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571426176:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619914.793410:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619914.793424:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008447b954/0x1cd7fe5368734866 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bccb expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619914.793472:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5ff7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589102080:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619914.793522:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619914.795505:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5ff7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589102080:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619914.795539:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008447b954/0x1cd7fe5368734866 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bccb expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.795560:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008447b954/0x1cd7fe5368734866 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bccb expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.795572:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079000, rc 0) 00000100:00100000:2.0:1788619914.795623:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571426176:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2239us (2839us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619914.795632:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 705 00000100:00100000:3.0:1788619914.800838:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619914.801045:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571426432 00000100:00100000:0.0:1788619914.801056:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571426432 00000100:00100000:0.0:1788619914.801159:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 924 00000100:00100000:0.0:1788619914.801169:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+15:14908:x1875503571426432:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:0.0:1788619914.801223:0:11397:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619914.801250:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe536873483c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bca8 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.801270:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619914.801275:0:11397:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788619914.801511:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a800/0x1cd7fe536873486d lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.801528:0:11397:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767be00/0x1cd7fe536873484a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bcaf expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.801549:0:11397:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a800/0x1cd7fe536873486d lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.801578:0:11397:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767be00/0x1cd7fe536873484a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bcaf expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.801592:0:11397:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767be00/0x1cd7fe536873484a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bcaf expref: 12 pid: 9417 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619914.801607:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c09c00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589102208:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619914.801692:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa840 going to sleep for 11 seconds 00000100:00100000:0.0:1788619914.804368:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c09c00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589102208:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619914.804398:0:11397:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a800/0x1cd7fe536873486d lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619914.805497:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619914.806319:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571426560 00000100:00100000:2.0:1788619914.806483:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571426560 00010000:00010000:2.0:1788619914.806529:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767be00/0x1cd7fe536873484a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bcaf expref: 13 pid: 9417 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619914.806568:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 311 00000100:00100000:2.0:1788619914.806578:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+13:2355:x1875503571426560:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619914.806592:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619914.806611:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767be00/0x1cd7fe536873484a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bcaf expref: 13 pid: 9417 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.806624:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767be00/0x1cd7fe536873484a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bcaf expref: 13 pid: 9417 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.806658:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767be00/0x1cd7fe536873484a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bcaf expref: 13 pid: 9417 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.806673:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619914.806679:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a800/0x1cd7fe536873486d lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.806690:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a800/0x1cd7fe536873486d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.806698:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a800/0x1cd7fe536873486d lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.806733:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619914.806785:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571426560:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 208us (1294us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619914.806792:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 311 00010000:00010000:1.0:1788619914.807001:0:11397:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a800/0x1cd7fe536873486d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.807019:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a800/0x1cd7fe536873486d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788619914.807145:0:11397:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967653 00000004:00010000:1.0:1788619914.807197:0:11397:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 172969 -> 172969 00000004:00010000:1.0:1788619914.807201:0:11397:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619914 -> 1788619914 00000004:00010000:1.0:1788619914.807203:0:11397:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619914 -> 1788619914 00000004:00010000:1.0:1788619914.807206:0:11397:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619914 -> 1788619914 00000004:00010000:1.0:1788619914.807208:0:11397:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1414 -> 1414 00000004:00010000:1.0:1788619914.807215:0:11397:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000a77d081b reply state 000000003788bd93 transno 4294967654 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a800/0x1cd7fe536873486d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619914.807223:0:11397:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788619914.807303:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+17:14908:x1875503571426432:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 6135us (6487us total) trans 4294967654 rc 0/0 00000100:00100000:1.0:1788619914.807310:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 924 00000020:00080000:1.0:1788619914.807314:0:11397:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788619914.809306:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a800/0x1cd7fe536873486d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.809358:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a800/0x1cd7fe536873486d lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.809367:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a800/0x1cd7fe536873486d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.809373:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c767a800 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788619914.809378:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a800/0x1cd7fe536873486d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.809401:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a800/0x1cd7fe536873486d lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.809416:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619914.809419:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788619914.809424:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a800/0x1cd7fe536873486d lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.809431:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a800/0x1cd7fe536873486d lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619914.811630:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619914.811849:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571426688 00000100:00100000:2.0:1788619914.811861:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571426688 00002000:00010000:2.0:1788619914.811919:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (2074055->18446744073709551615) 00002000:00100000:2.0:1788619914.811926:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5695880 with x1875503571426688 ext(2074055->18446744073709551615) 00002000:00010000:2.0:1788619914.811938:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5695880. 00000100:00100000:2.0:1788619914.811968:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 367 00000100:00100000:2.0:1788619914.811978:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5695880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571426688:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:2.0:1788619914.812276:0:6702:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1875493 -> 2074055 00002000:00010000:2.0:1788619914.812280:0:6702:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619914 -> 1788619914 00002000:00010000:2.0:1788619914.812283:0:6702:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619914 -> 1788619914 00002000:00010000:2.0:1788619914.812285:0:6702:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619914 -> 1788619914 00002000:00010000:2.0:1788619914.812287:0:6702:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 519 -> 519 00000100:00100000:2.0:1788619914.812369:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5695880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2352:x1875503571426688:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 391us (745us total) trans 4294967409 rc 0/0 00000100:00100000:2.0:1788619914.812376:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 367 00002000:00010000:2.0:1788619914.812381:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (2074055->18446744073709551615) 00002000:00100000:2.0:1788619914.812384:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5695880 with x1875503571426688 ext(2074055->18446744073709551615) 00002000:00010000:2.0:1788619914.812389:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5695880. 00000100:00100000:3.0:1788619914.816028:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619914.816115:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571426816 00000100:00100000:0.0:1788619914.816124:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571426816 00000100:00100000:0.0:1788619914.816183:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 925 00000100:00100000:0.0:1788619914.816193:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:14908:x1875503571426816:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619914.816215:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619914.816235:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a200/0x1cd7fe5368734874 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bcd2 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.816318:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a000/0x1cd7fe536873487b lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.816359:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a000/0x1cd7fe536873487b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.816370:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a000/0x1cd7fe536873487b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619914.816396:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 2074055 00000004:00200000:0.0:1788619914.816400:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619914.816435:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a000/0x1cd7fe536873487b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.816461:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a200/0x1cd7fe5368734874 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bcd2 expref: 18 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.816480:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a000/0x1cd7fe536873487b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bcd2 expref: 17 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.816489:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619914.816491:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619914.816495:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767a000, rc 0) 00000100:00100000:0.0:1788619914.816554:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+17:14908:x1875503571426816:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 362us (535us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619914.816562:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 925 00000100:00100000:1.0:1788619914.821739:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619914.821839:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571426944 00000100:00100000:3.0:1788619914.821850:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571426944 00000100:00100000:3.0:1788619914.821913:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 706 00000100:00100000:3.0:1788619914.821924:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe138850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:14908:x1875503571426944:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619914.821950:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619914.821974:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe5368734882 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bcd9 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.822016:0:6698:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000585a0430/0x1cd7fe5368734812 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1073152->1138687) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc8c expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.822026:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(10 8) 00010000:00010000:3.0:1788619914.822031:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(11 8) 00010000:00010000:3.0:1788619914.822037:0:6698:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe5368734882 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1875968->1933311] (req 1875968->1933311) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bcd9 expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.822078:0:6698:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000585a0430/0x1cd7fe5368734812 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1073152->1138687) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc8c expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.822125:0:6698:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000585a0430/0x1cd7fe5368734812 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1073152->1138687) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc8c expref: 6 pid: 6698 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619914.822141:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900d5d500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589102336:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619914.822420:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164180 going to sleep for 11 seconds 00000100:00100000:1.0:1788619914.826550:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619914.826664:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900d5d500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589102336:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619914.826696:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe5368734882 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1875968->1933311] (req 1875968->1933311) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bcd9 expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.826717:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683dc00, rc 0) 00000100:00100000:3.0:1788619914.826774:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe138850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:14908:x1875503571426944:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4851us (5045us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619914.826783:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 706 00000100:00100000:0.0:1788619914.827547:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571427072 00000100:00100000:0.0:1788619914.827557:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571427072 00010000:00010000:0.0:1788619914.827593:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000585a0430/0x1cd7fe5368734812 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1073152->1138687) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc8c expref: 7 pid: 6698 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619914.827631:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 312 00000100:00100000:0.0:1788619914.827650:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5696680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571427072:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619914.827663:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619914.827684:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000585a0430/0x1cd7fe5368734812 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1073152->1138687) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc8c expref: 7 pid: 6698 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.827697:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000585a0430/0x1cd7fe5368734812 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1073152->1138687) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc8c expref: 7 pid: 6698 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.827743:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000585a0430/0x1cd7fe5368734812 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1073152->1138687) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bc8c expref: 7 pid: 6698 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.827759:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(11 8) 00010000:00010000:0.0:1788619914.827763:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(11 8) 00010000:00010000:0.0:1788619914.827770:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe5368734882 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1875968->1933311) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bcd9 expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.827780:0:5831:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe5368734882 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1875968->1933311) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bcd9 expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.827817:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe5368734882 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1875968->1933311) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bcd9 expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619914.827841:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c09c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5831:1875503589102464:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619914.828108:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa840 going to sleep for 11 seconds 00000100:00100000:1.0:1788619914.831744:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619914.831899:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c09c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5831:1875503589102464:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619914.831932:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619914.831986:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5696680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571427072:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 4346us (5445us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619914.831999:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 312 00000100:00100000:2.0:1788619914.832957:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571427200 00000100:00100000:2.0:1788619914.832967:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571427200 00000100:00100000:2.0:1788619914.833025:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 368 00000100:00100000:2.0:1788619914.833035:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5697800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571427200:12345-192.168.201.5@tcp:22:fsx.0 00002000:00010000:2.0:1788619914.833500:0:6701:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 2074055 -> 2074055 00002000:00010000:2.0:1788619914.833505:0:6701:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619914 -> 1788619914 00002000:00010000:2.0:1788619914.833507:0:6701:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619914 -> 1788619914 00002000:00010000:2.0:1788619914.833509:0:6701:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619914 -> 1788619914 00002000:00010000:2.0:1788619914.833511:0:6701:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 519 -> 519 00000100:00100000:2.0:1788619914.833561:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5697800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571427200:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 526us (1826us total) trans 4294967410 rc 0/0 00000100:00100000:2.0:1788619914.833569:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 368 00000100:00100000:1.0:1788619914.836879:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619914.836957:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571427328 00000100:00100000:0.0:1788619914.836968:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571427328 00000100:00100000:0.0:1788619914.837030:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 926 00000100:00100000:0.0:1788619914.837065:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5695180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:14908:x1875503571427328:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619914.837086:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619914.837113:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a800/0x1cd7fe5368734889 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bce0 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.837194:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767b600/0x1cd7fe5368734890 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.837209:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767b600/0x1cd7fe5368734890 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.837221:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767b600/0x1cd7fe5368734890 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619914.837248:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 2074055 00000004:00200000:0.0:1788619914.837252:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619914.837269:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767b600/0x1cd7fe5368734890 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.837294:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a800/0x1cd7fe5368734889 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bce0 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.837313:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767b600/0x1cd7fe5368734890 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bce0 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.837321:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619914.837366:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619914.837371:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767b600, rc 0) 00000100:00100000:0.0:1788619914.837427:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5695180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+13:14908:x1875503571427328:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 365us (557us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619914.837443:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 926 00000100:00100000:1.0:1788619914.840313:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619914.840444:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571427456 00000100:00100000:0.0:1788619914.840454:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571427456 00000100:00100000:0.0:1788619914.840503:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 927 00000100:00100000:0.0:1788619914.840513:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5694e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+13:2355:x1875503571427456:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619914.840534:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619914.840556:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767b800/0x1cd7fe5368734897 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bce7 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.840597:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190400/0x1cd7fe536873482e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bca1 expref: 16 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619914.840655:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c09c00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589102592:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619914.840729:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa840 going to sleep for 11 seconds 00000100:00100000:0.0:1788619914.843271:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c09c00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589102592:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619914.843291:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 172969 00010000:00010000:0.0:1788619914.843308:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767b800/0x1cd7fe5368734897 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bce7 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.843385:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619914.843388:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619914.843393:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767b800/0x1cd7fe5368734897 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bce7 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.843403:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767b800, rc 0) 00000100:00100000:0.0:1788619914.843472:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5694e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+13:2355:x1875503571427456:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2961us (3169us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619914.843490:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 927 00000100:00100000:2.0:1788619914.845304:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619914.845438:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571427584 00000100:00100000:3.0:1788619914.845445:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571427584 00000100:00100000:3.0:1788619914.845507:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 928 00000100:00100000:3.0:1788619914.845514:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571427584:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619914.845529:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619914.845544:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe536873489e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bcee expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.845567:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190400/0x1cd7fe536873482e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bca1 expref: 16 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619914.845612:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900d5ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589102720:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619914.845685:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164180 going to sleep for 11 seconds 00000100:00100000:3.0:1788619914.847668:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900d5ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589102720:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619914.847696:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 172969 00010000:00010000:3.0:1788619914.847713:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe536873489e lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bcee expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.847733:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619914.847736:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619914.847742:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe536873489e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bcee expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.847761:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683ce00, rc 0) 00000100:00100000:3.0:1788619914.847818:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571427584:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2304us (2522us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619914.847835:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 928 00000100:00100000:1.0:1788619914.849963:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619914.850045:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571427712 00000100:00100000:0.0:1788619914.850054:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571427712 00000100:00100000:0.0:1788619914.850107:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 707 00000100:00100000:0.0:1788619914.850114:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5695f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571427712:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619914.850129:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619914.850142:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001f02fef/0x1cd7fe53687348a5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bcf5 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619914.850191:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c09c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589102848:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619914.850272:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa840 going to sleep for 11 seconds 00000100:00100000:0.0:1788619914.852060:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c09c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589102848:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619914.852096:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000001f02fef/0x1cd7fe53687348a5 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bcf5 expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.852116:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001f02fef/0x1cd7fe53687348a5 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bcf5 expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619914.852125:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767a800, rc 0) 00000100:00100000:0.0:1788619914.852170:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5695f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571427712:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2057us (2217us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619914.852177:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 707 00000100:00100000:1.0:1788619914.854431:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619914.854740:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571427840 00000100:00100000:2.0:1788619914.854761:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571427840 00000100:00100000:2.0:1788619914.854864:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 708 00000100:00100000:2.0:1788619914.854875:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5697b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:14908:x1875503571427840:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619914.854893:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619914.854921:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d6587d69/0x1cd7fe53687348ac lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bcfc expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.854952:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe5368734882 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1875968->1933311) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bcd9 expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.854961:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(11 8) 00010000:00010000:2.0:1788619914.854965:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(12 8) 00010000:00010000:2.0:1788619914.854971:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d6587d69/0x1cd7fe53687348ac lrc: 4/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1712128->1716223] (req 1712128->1716223) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bcfc expref: 7 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.855004:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe5368734882 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1875968->1933311) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bcd9 expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.855018:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe5368734882 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1875968->1933311) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bcd9 expref: 6 pid: 6698 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619914.855032:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5ff7100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589102976:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619914.855118:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619914.861866:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619914.862144:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5ff7100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589102976:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619914.862179:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d6587d69/0x1cd7fe53687348ac lrc: 3/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1712128->1716223] (req 1712128->1716223) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bcfc expref: 7 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.862221:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079800, rc 0) 00000100:00100000:2.0:1788619914.862289:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5697b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:14908:x1875503571427840:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7416us (7866us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619914.862298:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 708 00000100:00100000:3.0:1788619914.862623:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571427968 00000100:00100000:3.0:1788619914.862630:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571427968 00010000:00010000:3.0:1788619914.862672:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe5368734882 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1875968->1933311) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bcd9 expref: 7 pid: 6698 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619914.862712:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 313 00000100:00100000:3.0:1788619914.862721:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8022050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571427968:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619914.862736:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619914.862759:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe5368734882 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1875968->1933311) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bcd9 expref: 7 pid: 6698 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.862781:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe5368734882 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1875968->1933311) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bcd9 expref: 7 pid: 6698 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.862805:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe5368734882 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1875968->1933311) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bcd9 expref: 7 pid: 6698 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.862834:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(12 8) 00010000:00010000:3.0:1788619914.862839:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(12 8) 00010000:00010000:3.0:1788619914.862856:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000d6587d69/0x1cd7fe53687348ac lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1712128->1716223) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bcfc expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.862868:0:13330:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d6587d69/0x1cd7fe53687348ac lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1712128->1716223) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bcfc expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.863203:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000d6587d69/0x1cd7fe53687348ac lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1712128->1716223) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bcfc expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619914.863221:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900d5ea00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589103104:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619914.863280:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164180 going to sleep for 11 seconds 00000100:00100000:3.0:1788619914.865732:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900d5ea00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589103104:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619914.865760:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619914.865814:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8022050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571427968:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3094us (3956us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619914.865836:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 313 00000100:00100000:1.0:1788619914.874424:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619914.874521:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571428096 00000100:00100000:3.0:1788619914.874531:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571428096 00002000:00100000:3.0:1788619914.874577:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1048576->2076671) req@ffff8e78c5694380 x1875503571428096/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 488/0 e 0 to 0 dl 1788619925 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619914.874588:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5694380 with x1875503571428096 ext(1048576->2076671) 00002000:00010000:3.0:1788619914.874594:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5694380 00000100:00100000:3.0:1788619914.874616:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 369 00000100:00100000:3.0:1788619914.874625:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5694380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571428096:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619914.888317:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5694380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571428096:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 13679us (13889us total) trans 0 rc 1028096/1028096 00000100:00100000:2.0:1788619914.888372:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 369 00002000:00100000:2.0:1788619914.888386:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1048576->2076671) req@ffff8e78c5694380 x1875503571428096/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 488/440 e 0 to 0 dl 1788619925 ref 1 fl Complete:/600/0 rc 1028096/1028096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619914.888397:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5694380 with x1875503571428096 ext(1048576->2076671) 00002000:00010000:2.0:1788619914.888403:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5694380 00000100:00100000:1.0:1788619914.896408:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619914.896451:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619914.896528:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571428352 00000100:00100000:3.0:1788619914.896538:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571428352 00000100:00100000:3.0:1788619914.896600:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 709 00000100:00100000:3.0:1788619914.896611:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5696a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571428352:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619914.896630:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619914.896649:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe53687348b3 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bd0a expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.896675:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(12 8) 00010000:00010000:3.0:1788619914.896679:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(12 8) 00010000:00010000:3.0:1788619914.896690:0:7654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe53687348b3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd0a expref: 7 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.896709:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe53687348b3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd0a expref: 7 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.896721:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683dc00, rc 0) 00000100:00100000:3.0:1788619914.896786:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5696a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571428352:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 177us (334us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619914.896809:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 709 00000100:00100000:2.0:1788619914.897260:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571428224 00000100:00100000:2.0:1788619914.897266:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571428224 00000100:00100000:2.0:1788619914.897307:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 929 00000100:00100000:2.0:1788619914.897316:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5695c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571428224:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619914.897361:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619914.897376:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8078200/0x1cd7fe53687348ba lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bd03 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.897405:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190400/0x1cd7fe536873482e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bca1 expref: 16 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619914.897448:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5ff7100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589103232:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619914.897506:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619914.903715:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5ff7100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589103232:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619914.903746:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 172969 00010000:00010000:2.0:1788619914.903766:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8078200/0x1cd7fe53687348ba lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd03 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.903789:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619914.903793:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619914.903799:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8078200/0x1cd7fe53687348ba lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd03 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.903814:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8078200, rc 0) 00000100:00100000:2.0:1788619914.903891:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5695c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571428224:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6575us (7493us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619914.903900:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 929 00000100:00100000:0.0:1788619914.908988:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619914.909146:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571428480 00000100:00100000:1.0:1788619914.909157:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571428480 00000100:00100000:1.0:1788619914.909219:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 930 00000100:00100000:1.0:1788619914.909231:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c09c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+13:2355:x1875503571428480:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619914.909251:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619914.909272:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752600/0x1cd7fe53687348c1 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bd11 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.909308:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190400/0x1cd7fe536873482e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bca1 expref: 16 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619914.909383:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5695500 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589103360:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619914.909462:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c600 going to sleep for 11 seconds 00000100:00100000:2.0:1788619914.911850:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5695500 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589103360:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619914.911880:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 172969 00010000:00010000:2.0:1788619914.911899:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752600/0x1cd7fe53687348c1 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd11 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.911920:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619914.911924:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619914.911930:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752600/0x1cd7fe53687348c1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd11 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619914.911946:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752600, rc 0) 00000100:00100000:2.0:1788619914.912017:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c09c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+13:2355:x1875503571428480:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2787us (3040us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619914.912043:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 930 00000100:00100000:3.0:1788619914.920438:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619914.920623:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571428608 00000100:00100000:1.0:1788619914.920632:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571428608 00000100:00100000:1.0:1788619914.920703:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 710 00000100:00100000:1.0:1788619914.920713:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:14908:x1875503571428608:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619914.920734:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619914.920754:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dc75705a/0x1cd7fe53687348c8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bd18 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.920791:0:6698:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000d6587d69/0x1cd7fe53687348ac lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1712128->1716223) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bcfc expref: 8 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.920801:0:6698:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe53687348b3 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd0a expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.920833:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(12 8) 00010000:00010000:1.0:1788619914.920838:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(12 8) 00010000:00010000:1.0:1788619914.920844:0:6698:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc75705a/0x1cd7fe53687348c8 lrc: 5/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [1884160->1888255] (req 1884160->1888255) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd18 expref: 8 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.920881:0:6698:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe53687348b3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd0a expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.920896:0:6698:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe53687348b3 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd0a expref: 6 pid: 7654 timeout: 507 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619914.920911:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5695500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589103488:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619914.921064:0:6698:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000d6587d69/0x1cd7fe53687348ac lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1712128->1716223) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bcfc expref: 8 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619914.921073:0:6698:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000d6587d69/0x1cd7fe53687348ac lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1712128->1716223) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bcfc expref: 8 pid: 6697 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619914.921083:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5696680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589103616:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619914.921117:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619914.923963:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5695500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589103488:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619914.923980:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5696680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589103616:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619914.924011:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000dc75705a/0x1cd7fe53687348c8 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [1884160->1888255] (req 1884160->1888255) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd18 expref: 8 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.924031:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5753600, rc 0) 00000100:00100000:3.0:1788619914.924097:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:14908:x1875503571428608:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3386us (3670us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619914.924105:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 710 00000100:00100000:0.0:1788619914.928673:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619914.928731:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619914.928868:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571428736 00000100:00100000:3.0:1788619914.928880:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571428736 00010000:00010000:3.0:1788619914.928923:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe53687348b3 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd0a expref: 7 pid: 7654 timeout: 507 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619914.928968:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571428864 00000100:00100000:3.0:1788619914.928972:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571428864 00010000:00010000:3.0:1788619914.928985:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000d6587d69/0x1cd7fe53687348ac lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1712128->1716223) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bcfc expref: 8 pid: 6697 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619914.929014:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 314 00000100:00100000:3.0:1788619914.929023:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c0a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571428736:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619914.929037:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619914.929061:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe53687348b3 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd0a expref: 7 pid: 7654 timeout: 507 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.929082:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe53687348b3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd0a expref: 7 pid: 7654 timeout: 507 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.929114:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe53687348b3 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd0a expref: 7 pid: 7654 timeout: 507 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.929132:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619914.929180:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c0a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571428736:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 158us (517us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619914.929187:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 314 00000100:00100000:3.0:1788619914.929209:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 315 00000100:00100000:3.0:1788619914.929215:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c09880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571428864:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619914.929222:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619914.929230:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d6587d69/0x1cd7fe53687348ac lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1712128->1716223) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bcfc expref: 8 pid: 6697 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.929240:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d6587d69/0x1cd7fe53687348ac lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1712128->1716223) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bcfc expref: 8 pid: 6697 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.929252:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d6587d69/0x1cd7fe53687348ac lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1712128->1716223) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bcfc expref: 8 pid: 6697 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.929262:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(12 8) 00010000:00010000:3.0:1788619914.929266:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(12 8) 00010000:00010000:3.0:1788619914.929279:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000dc75705a/0x1cd7fe53687348c8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1884160->1888255) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd18 expref: 7 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.929289:0:5829:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dc75705a/0x1cd7fe53687348c8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1884160->1888255) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd18 expref: 7 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619914.929362:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000dc75705a/0x1cd7fe53687348c8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1884160->1888255) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd18 expref: 7 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619914.929376:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900d5ea00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1875503589103744:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619914.929428:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81646c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619914.932315:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900d5ea00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1875503589103744:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619914.932369:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619914.932415:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c09880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571428864:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3199us (3683us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619914.932425:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 315 00000100:00100000:1.0:1788619914.946886:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619914.947004:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571428992 00000100:00100000:2.0:1788619914.947017:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571428992 00002000:00100000:2.0:1788619914.947098:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1884160->1888255) req@ffff8e78c5696680 x1875503571428992/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 488/0 e 0 to 0 dl 1788619925 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619914.947111:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5696680 with x1875503571428992 ext(1884160->1888255) 00002000:00010000:2.0:1788619914.947117:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5696680 00000100:00100000:2.0:1788619914.947160:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 370 00000100:00100000:2.0:1788619914.947174:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5696680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571428992:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619914.947594:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5696680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571428992:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 423us (715us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619914.947603:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 370 00002000:00100000:2.0:1788619914.947609:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1884160->1888255) req@ffff8e78c5696680 x1875503571428992/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 488/4536 e 0 to 0 dl 1788619925 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619914.947618:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5696680 with x1875503571428992 ext(1884160->1888255) 00002000:00010000:2.0:1788619914.947621:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5696680 00000100:00100000:0.0:1788619914.955894:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619914.956381:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571429120 00000100:00100000:2.0:1788619914.956392:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571429120 00002000:00100000:2.0:1788619914.956446:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1888256->1892351) req@ffff8e78c5c0b100 x1875503571429120/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 488/0 e 0 to 0 dl 1788619925 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619914.956456:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5c0b100 with x1875503571429120 ext(1888256->1892351) 00002000:00010000:2.0:1788619914.956462:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5c0b100 00000100:00100000:2.0:1788619914.956486:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 371 00000100:00100000:2.0:1788619914.956495:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c0b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2353:x1875503571429120:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619914.956725:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c0b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2353:x1875503571429120:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 230us (840us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619914.956733:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 371 00002000:00100000:2.0:1788619914.956740:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1888256->1892351) req@ffff8e78c5c0b100 x1875503571429120/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 488/4536 e 0 to 0 dl 1788619925 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619914.956749:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5c0b100 with x1875503571429120 ext(1888256->1892351) 00002000:00010000:2.0:1788619914.956752:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5c0b100 00000100:00100000:1.0:1788619914.973315:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619914.973522:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571429248 00000100:00100000:2.0:1788619914.973533:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571429248 00002000:00100000:2.0:1788619914.973586:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1892352->1896447) req@ffff8e78c5695500 x1875503571429248/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 488/0 e 0 to 0 dl 1788619925 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619914.973597:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5695500 with x1875503571429248 ext(1892352->1896447) 00002000:00010000:2.0:1788619914.973602:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5695500 00000100:00100000:2.0:1788619914.973639:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 372 00000100:00100000:2.0:1788619914.973650:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5695500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2352:x1875503571429248:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619914.973874:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5695500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2352:x1875503571429248:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 225us (567us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619914.973881:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 372 00002000:00100000:2.0:1788619914.973888:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1892352->1896447) req@ffff8e78c5695500 x1875503571429248/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 488/4536 e 0 to 0 dl 1788619925 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619914.973896:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5695500 with x1875503571429248 ext(1892352->1896447) 00002000:00010000:2.0:1788619914.973900:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5695500 00000100:00100000:1.0:1788619914.976605:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619914.976746:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571429376 00000100:00100000:0.0:1788619914.976757:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571429376 00002000:00100000:0.0:1788619914.976801:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1884160->1896447) req@ffff8e77c6b12680 x1875503571429376/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 12776/0 e 0 to 0 dl 1788619925 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619914.976811:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6b12680 with x1875503571429376 ext(1884160->1896447) 00002000:00010000:0.0:1788619914.976816:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6b12680 00000100:00100000:0.0:1788619914.976866:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 373 00000100:00100000:0.0:1788619914.976875:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b12680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2352:x1875503571429376:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:0.0:1788619914.981344:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b12680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2352:x1875503571429376:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 4441us (4720us total) trans 4294967411 rc 0/0 00000100:00100000:0.0:1788619914.981356:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 373 00002000:00100000:0.0:1788619914.981367:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1884160->1896447) req@ffff8e77c6b12680 x1875503571429376/t4294967411(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:10/0 lens 12776/448 e 0 to 0 dl 1788619925 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619914.981376:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6b12680 with x1875503571429376 ext(1884160->1896447) 00002000:00010000:0.0:1788619914.981382:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6b12680 00000100:00100000:1.0:1788619914.986831:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619914.987048:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571429504 00000100:00100000:3.0:1788619914.987058:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571429504 00000100:00100000:3.0:1788619914.987127:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 711 00000100:00100000:3.0:1788619914.987136:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b13480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2353:x1875503571429504:12345-192.168.201.5@tcp:16:fsx.0 00000100:00100000:0.0:1788619915.042058:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b13480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2353:x1875503571429504:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 54911us (55234us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619915.042073:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 711 00000001:00080000:1.0:1788619915.042314:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967411 is committed 00000001:00080000:1.0:1788619915.042380:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967410 is committed 00010000:00010000:3.0:1788619915.043560:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788619915.048770:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619915.049199:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571429632 00000100:00100000:0.0:1788619915.049209:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571429632 00002000:00100000:0.0:1788619915.049258:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1757184->1761279) req@ffff8e77c6b13100 x1875503571429632/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/0 e 0 to 0 dl 1788619926 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619915.049269:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6b13100 with x1875503571429632 ext(1757184->1761279) 00002000:00010000:0.0:1788619915.049285:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6b13100 00000100:00100000:0.0:1788619915.049321:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 374 00000100:00100000:0.0:1788619915.049367:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b13100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571429632:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619915.049613:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b13100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571429632:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 249us (853us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619915.049621:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 374 00002000:00100000:0.0:1788619915.049627:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1757184->1761279) req@ffff8e77c6b13100 x1875503571429632/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/4536 e 0 to 0 dl 1788619926 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619915.049636:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6b13100 with x1875503571429632 ext(1757184->1761279) 00002000:00010000:0.0:1788619915.049650:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6b13100 00000100:00100000:1.0:1788619915.056415:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619915.057020:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571429760 00000100:00100000:0.0:1788619915.057031:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571429760 00002000:00100000:0.0:1788619915.057096:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1761280->1765375) req@ffff8e78c7582680 x1875503571429760/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/0 e 0 to 0 dl 1788619926 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619915.057107:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7582680 with x1875503571429760 ext(1761280->1765375) 00002000:00010000:0.0:1788619915.057113:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c7582680 00000100:00100000:0.0:1788619915.057151:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 375 00000100:00100000:0.0:1788619915.057160:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7582680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571429760:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619915.057447:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7582680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571429760:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 287us (1041us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619915.057454:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 375 00002000:00100000:0.0:1788619915.057460:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1761280->1765375) req@ffff8e78c7582680 x1875503571429760/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/4536 e 0 to 0 dl 1788619926 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619915.057468:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7582680 with x1875503571429760 ext(1761280->1765375) 00002000:00010000:0.0:1788619915.057485:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c7582680 00000100:00100000:1.0:1788619915.065153:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619915.065261:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571429888 00000100:00100000:2.0:1788619915.065285:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571429888 00002000:00100000:2.0:1788619915.065371:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1765376->1769471) req@ffff8e78c7580380 x1875503571429888/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/0 e 0 to 0 dl 1788619926 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619915.065381:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7580380 with x1875503571429888 ext(1765376->1769471) 00002000:00010000:2.0:1788619915.065386:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c7580380 00000100:00100000:2.0:1788619915.065411:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 376 00000100:00100000:2.0:1788619915.065423:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7580380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571429888:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619915.065695:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7580380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571429888:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 274us (551us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619915.065702:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 376 00002000:00100000:2.0:1788619915.065709:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1765376->1769471) req@ffff8e78c7580380 x1875503571429888/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/4536 e 0 to 0 dl 1788619926 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619915.065716:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7580380 with x1875503571429888 ext(1765376->1769471) 00002000:00010000:2.0:1788619915.065720:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c7580380 00000100:00100000:1.0:1788619915.070846:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619915.070960:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571430016 00000100:00100000:2.0:1788619915.070970:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571430016 00002000:00100000:2.0:1788619915.071018:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1769472->1773567) req@ffff8e78c7581880 x1875503571430016/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/0 e 0 to 0 dl 1788619926 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619915.071029:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7581880 with x1875503571430016 ext(1769472->1773567) 00002000:00010000:2.0:1788619915.071034:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c7581880 00000100:00100000:2.0:1788619915.071071:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 377 00000100:00100000:2.0:1788619915.071080:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7581880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571430016:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619915.071296:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7581880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571430016:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 216us (458us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619915.071319:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 377 00002000:00100000:2.0:1788619915.071365:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1769472->1773567) req@ffff8e78c7581880 x1875503571430016/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/4536 e 0 to 0 dl 1788619926 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619915.071374:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7581880 with x1875503571430016 ext(1769472->1773567) 00002000:00010000:2.0:1788619915.071378:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c7581880 00000100:00100000:1.0:1788619915.075268:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619915.075365:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571430144 00000100:00100000:0.0:1788619915.075383:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571430144 00002000:00100000:0.0:1788619915.075425:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1773568->1777663) req@ffff8e78c7582a00 x1875503571430144/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/0 e 0 to 0 dl 1788619926 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619915.075436:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7582a00 with x1875503571430144 ext(1773568->1777663) 00002000:00010000:0.0:1788619915.075441:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c7582a00 00000100:00100000:0.0:1788619915.075472:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 378 00000100:00100000:0.0:1788619915.075480:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7582a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571430144:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619915.075670:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7582a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571430144:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 190us (409us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619915.075677:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 378 00002000:00100000:0.0:1788619915.075683:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1773568->1777663) req@ffff8e78c7582a00 x1875503571430144/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/4536 e 0 to 0 dl 1788619926 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619915.075692:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7582a00 with x1875503571430144 ext(1773568->1777663) 00002000:00010000:0.0:1788619915.075695:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c7582a00 00000100:00100000:0.0:1788619915.078696:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619915.079105:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571430272 00000100:00100000:1.0:1788619915.079118:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571430272 00002000:00100000:1.0:1788619915.079178:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1777664->1781759) req@ffff8e78c5c0b480 x1875503571430272/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/0 e 0 to 0 dl 1788619926 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619915.079189:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5c0b480 with x1875503571430272 ext(1777664->1781759) 00002000:00010000:1.0:1788619915.079195:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5c0b480 00000100:00100000:1.0:1788619915.079225:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 379 00000100:00100000:1.0:1788619915.079235:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c0b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571430272:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619915.079491:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c0b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571430272:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 257us (803us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619915.079498:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 379 00002000:00100000:1.0:1788619915.079505:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1777664->1781759) req@ffff8e78c5c0b480 x1875503571430272/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/4536 e 0 to 0 dl 1788619926 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619915.079513:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5c0b480 with x1875503571430272 ext(1777664->1781759) 00002000:00010000:1.0:1788619915.079517:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5c0b480 00000100:00100000:1.0:1788619915.082316:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619915.084076:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571430400 00000100:00100000:2.0:1788619915.084110:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571430400 00002000:00100000:2.0:1788619915.084177:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1781760->1785855) req@ffff8e78c7580700 x1875503571430400/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/0 e 0 to 0 dl 1788619926 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619915.084189:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7580700 with x1875503571430400 ext(1781760->1785855) 00002000:00010000:2.0:1788619915.084196:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c7580700 00000100:00100000:2.0:1788619915.084244:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 380 00000100:00100000:2.0:1788619915.084255:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7580700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571430400:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619915.084757:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7580700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571430400:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 501us (2449us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619915.084764:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 380 00002000:00100000:2.0:1788619915.084773:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1781760->1785855) req@ffff8e78c7580700 x1875503571430400/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/4536 e 0 to 0 dl 1788619926 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619915.084782:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7580700 with x1875503571430400 ext(1781760->1785855) 00002000:00010000:2.0:1788619915.084786:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c7580700 00000100:00100000:2.0:1788619915.089387:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619915.089430:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619915.089532:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571430656 00000100:00100000:0.0:1788619915.089542:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571430656 00000100:00100000:0.0:1788619915.089605:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 712 00000100:00100000:0.0:1788619915.089616:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571430656:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619915.089638:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619915.089665:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001f02fef/0x1cd7fe53687348cf lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bd26 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619915.089749:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c08e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589103872:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619915.089852:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:3.0:1788619915.096140:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571430528 00000100:00100000:3.0:1788619915.096157:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571430528 00000100:00100000:3.0:1788619915.096229:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 931 00000100:00100000:3.0:1788619915.096240:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571430528:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619915.096262:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619915.096303:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe53687348d6 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bd1f expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619915.096375:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190400/0x1cd7fe536873482e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bca1 expref: 16 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619915.096417:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900d5ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589104000:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619915.096509:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c2c5ef00 going to sleep for 11 seconds 00000100:00100000:1.0:1788619915.097527:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c08e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589103872:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619915.097574:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000001f02fef/0x1cd7fe53687348cf lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd26 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.097597:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001f02fef/0x1cd7fe53687348cf lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd26 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.097609:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767a800, rc 0) 00000100:00100000:1.0:1788619915.097666:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571430656:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8051us (8292us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619915.097674:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 712 00000100:00100000:3.0:1788619915.101780:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900d5ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589104000:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619915.101805:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 172969 00010000:00010000:3.0:1788619915.101836:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe53687348d6 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd1f expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619915.101856:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619915.101858:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619915.101864:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe53687348d6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd1f expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619915.101877:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683dc00, rc 0) 00000100:00100000:3.0:1788619915.101936:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571430528:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5698us (12505us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619915.101945:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 931 00000100:00100000:3.0:1788619915.106071:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619915.106468:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571430784 00000100:00100000:2.0:1788619915.106491:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571430784 00000100:00100000:2.0:1788619915.106540:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 932 00000100:00100000:2.0:1788619915.106550:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+13:14908:x1875503571430784:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619915.106569:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619915.106589:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079600/0x1cd7fe53687348dd lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bd2d expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.106633:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190400/0x1cd7fe536873482e lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bca1 expref: 16 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.106653:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079600/0x1cd7fe53687348dd lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd2d expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.106693:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190400/0x1cd7fe536873482e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bca1 expref: 16 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.106706:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190400/0x1cd7fe536873482e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bca1 expref: 16 pid: 11397 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619915.106718:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5ff6300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589104128:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619915.106796:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619915.118312:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5ff6300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589104128:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619915.118396:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079600/0x1cd7fe53687348dd lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd2d expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.118416:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619915.118419:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619915.118443:0:5846:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079600/0x1cd7fe53687348dd lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd2d expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.118455:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079600, rc 0) 00000100:00100000:2.0:1788619915.118534:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+14:14908:x1875503571430784:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 11984us (12472us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619915.118543:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 932 00000100:00100000:3.0:1788619915.119601:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619915.119976:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571430912 00000100:00100000:0.0:1788619915.119985:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571430912 00010000:00010000:0.0:1788619915.120022:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190400/0x1cd7fe536873482e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bca1 expref: 17 pid: 11397 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619915.120061:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 316 00000100:00100000:0.0:1788619915.120070:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2355:x1875503571430912:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619915.120092:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619915.120118:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190400/0x1cd7fe536873482e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bca1 expref: 17 pid: 11397 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619915.120131:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190400/0x1cd7fe536873482e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bca1 expref: 17 pid: 11397 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619915.120152:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df190400/0x1cd7fe536873482e lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bca1 expref: 17 pid: 11397 timeout: 507 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619915.120167:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619915.120173:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079600/0x1cd7fe53687348dd lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd2d expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619915.120184:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079600/0x1cd7fe53687348dd lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd2d expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619915.120207:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079600/0x1cd7fe53687348dd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd2d expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619915.120242:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079600/0x1cd7fe53687348dd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd2d expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619915.120256:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c08a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589104256:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619915.120364:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaab40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619915.123165:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c08a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589104256:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619915.123191:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619915.123309:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571430912:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3239us (3711us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619915.123321:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 316 00000100:00100000:2.0:1788619915.130235:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619915.131031:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571431040 00000100:00100000:0.0:1788619915.131043:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571431040 00000100:00100000:0.0:1788619915.131116:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 713 00000100:00100000:0.0:1788619915.131127:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571431040:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619915.131146:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619915.131165:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006989f8b0/0x1cd7fe53687348e4 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bd34 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619915.131224:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c08a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589104384:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619915.131297:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaab40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619915.134777:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c08a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589104384:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619915.134838:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000006989f8b0/0x1cd7fe53687348e4 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd34 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619915.134863:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006989f8b0/0x1cd7fe53687348e4 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd34 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619915.134876:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767bc00, rc 0) 00000100:00100000:0.0:1788619915.134931:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571431040:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3805us (4705us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619915.134940:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 713 00000100:00100000:3.0:1788619915.137690:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619915.137773:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571431168 00000100:00100000:0.0:1788619915.137783:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571431168 00002000:00100000:0.0:1788619915.137843:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900d5d500 x1875503571431168/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:11/0 lens 488/0 e 0 to 0 dl 1788619926 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619915.137854:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900d5d500 with x1875503571431168 00000100:00100000:0.0:1788619915.137893:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 571 00000100:00100000:0.0:1788619915.137913:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+14:2354:x1875503571431168:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619915.138214:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+14:2354:x1875503571431168:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 312us (533us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619915.138222:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 571 00002000:00100000:0.0:1788619915.138229:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900d5d500 x1875503571431168/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:11/0 lens 488/440 e 0 to 0 dl 1788619926 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619915.138236:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900d5d500 with x1875503571431168 00000100:00100000:0.0:1788619915.141880:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619915.142050:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571431296 00000100:00100000:3.0:1788619915.142059:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571431296 00000100:00100000:3.0:1788619915.142808:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 714 00000100:00100000:3.0:1788619915.142846:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c08a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571431296:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619915.142866:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619915.142884:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c4fc38e/0x1cd7fe53687348eb lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bd3b expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619915.142942:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900d5fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589104512:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619915.143547:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c2c5ef00 going to sleep for 11 seconds 00000100:00100000:3.0:1788619915.146380:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900d5fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589104512:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619915.146428:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004c4fc38e/0x1cd7fe53687348eb lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd3b expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619915.146457:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c4fc38e/0x1cd7fe53687348eb lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd3b expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619915.146469:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683ce00, rc 0) 00000100:00100000:3.0:1788619915.146519:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c08a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571431296:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3675us (4648us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619915.146527:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 714 00000100:00100000:0.0:1788619915.150066:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619915.150777:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571431424 00000100:00100000:1.0:1788619915.150787:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571431424 00000100:00100000:1.0:1788619915.150845:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 715 00000100:00100000:1.0:1788619915.150855:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c09180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571431424:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619915.150874:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619915.150894:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f4c11a7a/0x1cd7fe53687348f2 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bd42 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619915.150976:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c7581f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589104640:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619915.151062:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c900 going to sleep for 11 seconds 00000100:00100000:1.0:1788619915.154408:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c7581f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589104640:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619915.154447:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f4c11a7a/0x1cd7fe53687348f2 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd42 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.154471:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f4c11a7a/0x1cd7fe53687348f2 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd42 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.154499:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752000, rc 0) 00000100:00100000:1.0:1788619915.154560:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c09180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571431424:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3705us (4608us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619915.154569:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 715 00000100:00100000:2.0:1788619915.158501:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619915.158722:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571431552 00000100:00100000:0.0:1788619915.158731:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571431552 00002000:00100000:0.0:1788619915.158777:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5ff5180 x1875503571431552/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:11/0 lens 488/0 e 0 to 0 dl 1788619926 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619915.158788:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5ff5180 with x1875503571431552 00000100:00100000:0.0:1788619915.158836:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 572 00000100:00100000:0.0:1788619915.158847:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+14:2355:x1875503571431552:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619915.159282:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+14:2355:x1875503571431552:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 435us (788us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619915.159290:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 572 00002000:00100000:0.0:1788619915.159297:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5ff5180 x1875503571431552/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:11/0 lens 488/440 e 0 to 0 dl 1788619926 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619915.159304:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5ff5180 with x1875503571431552 00000100:00100000:0.0:1788619915.163876:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619915.163958:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571431680 00000100:00100000:1.0:1788619915.163968:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571431680 00000100:00100000:1.0:1788619915.164024:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 716 00000100:00100000:1.0:1788619915.164033:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c09500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571431680:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619915.164051:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619915.164069:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e4c897/0x1cd7fe53687348f9 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bd49 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619915.164138:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c7581f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589104768:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619915.164221:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c900 going to sleep for 11 seconds 00000100:00100000:1.0:1788619915.166587:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c7581f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589104768:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619915.166624:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000029e4c897/0x1cd7fe53687348f9 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd49 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.166658:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029e4c897/0x1cd7fe53687348f9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd49 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.166671:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5753800, rc 0) 00000100:00100000:1.0:1788619915.166726:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c09500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571431680:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2694us (2862us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619915.166749:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 716 00000100:00100000:2.0:1788619915.170530:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619915.170631:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571431808 00000100:00100000:0.0:1788619915.170651:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571431808 00000100:00100000:0.0:1788619915.170707:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 717 00000100:00100000:0.0:1788619915.170718:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571431808:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619915.170737:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619915.170756:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d412ceb8/0x1cd7fe5368734900 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bd50 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619915.170830:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c08e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589104896:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619915.170909:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaab40 going to sleep for 11 seconds 00000100:00100000:2.0:1788619915.173181:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c08e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589104896:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619915.173220:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d412ceb8/0x1cd7fe5368734900 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd50 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.173243:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d412ceb8/0x1cd7fe5368734900 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd50 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.173257:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767ba00, rc 0) 00000100:00100000:2.0:1788619915.173317:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571431808:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2601us (2796us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619915.173368:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 717 00000100:00100000:3.0:1788619915.176663:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619915.176743:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571431936 00000100:00100000:1.0:1788619915.176753:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571431936 00002000:00100000:1.0:1788619915.176803:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900d5fb80 x1875503571431936/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:11/0 lens 488/0 e 0 to 0 dl 1788619926 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619915.176826:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900d5fb80 with x1875503571431936 00000100:00100000:1.0:1788619915.176859:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 573 00000100:00100000:1.0:1788619915.176867:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+14:2355:x1875503571431936:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619915.177129:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+14:2355:x1875503571431936:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 262us (478us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619915.177137:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 573 00002000:00100000:1.0:1788619915.177143:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900d5fb80 x1875503571431936/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:11/0 lens 488/440 e 0 to 0 dl 1788619926 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619915.177151:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900d5fb80 with x1875503571431936 00000100:00100000:3.0:1788619915.180536:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619915.180654:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571432064 00000100:00100000:1.0:1788619915.180663:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571432064 00000100:00100000:1.0:1788619915.180720:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 718 00000100:00100000:1.0:1788619915.180730:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571432064:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619915.180747:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619915.180764:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e4c897/0x1cd7fe5368734907 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bd57 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619915.180829:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c7581f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589105024:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619915.180901:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c900 going to sleep for 11 seconds 00000100:00100000:3.0:1788619915.185991:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c7581f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589105024:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619915.186033:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000029e4c897/0x1cd7fe5368734907 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd57 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619915.186080:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029e4c897/0x1cd7fe5368734907 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd57 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619915.186093:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5753800, rc 0) 00000100:00100000:3.0:1788619915.186160:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571432064:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5430us (5632us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619915.186169:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 718 00000100:00100000:3.0:1788619915.189163:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619915.189439:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571432192 00000100:00100000:1.0:1788619915.189448:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571432192 00000100:00100000:1.0:1788619915.189503:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 719 00000100:00100000:1.0:1788619915.189512:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571432192:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619915.189529:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619915.189544:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f4c11a7a/0x1cd7fe536873490e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bd5e expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619915.189601:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c7583800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589105152:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619915.189696:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c780 going to sleep for 11 seconds 00000100:00100000:1.0:1788619915.192124:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c7583800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589105152:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619915.192161:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f4c11a7a/0x1cd7fe536873490e lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd5e expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.192185:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f4c11a7a/0x1cd7fe536873490e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd5e expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.192198:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752000, rc 0) 00000100:00100000:1.0:1788619915.192290:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571432192:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2778us (3136us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619915.192299:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 719 00000100:00100000:0.0:1788619915.196543:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619915.196626:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571432320 00000100:00100000:2.0:1788619915.196636:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571432320 00002000:00100000:2.0:1788619915.196799:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5c08e00 x1875503571432320/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:11/0 lens 488/0 e 0 to 0 dl 1788619926 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619915.196811:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5c08e00 with x1875503571432320 00000100:00100000:2.0:1788619915.196921:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 574 00000100:00100000:2.0:1788619915.196931:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c08e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+14:2354:x1875503571432320:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619915.197269:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c08e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+14:2354:x1875503571432320:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 337us (734us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619915.197276:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 574 00002000:00100000:2.0:1788619915.197283:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5c08e00 x1875503571432320/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:11/0 lens 488/440 e 0 to 0 dl 1788619926 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619915.197291:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5c08e00 with x1875503571432320 00000100:00100000:3.0:1788619915.203501:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619915.205836:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571432448 00000100:00100000:1.0:1788619915.205851:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571432448 00000100:00100000:1.0:1788619915.205977:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 720 00000100:00100000:1.0:1788619915.205988:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571432448:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619915.206021:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619915.206037:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f4c11a7a/0x1cd7fe5368734915 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bd65 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619915.206107:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c7583800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589105280:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619915.206193:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c780 going to sleep for 11 seconds 00000100:00100000:1.0:1788619915.211801:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c7583800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589105280:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619915.211849:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f4c11a7a/0x1cd7fe5368734915 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd65 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.211872:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f4c11a7a/0x1cd7fe5368734915 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd65 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.211884:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752000, rc 0) 00000100:00100000:1.0:1788619915.211943:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571432448:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5958us (8451us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619915.211958:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 720 00000100:00100000:0.0:1788619915.218528:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619915.218653:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571432576 00000100:00100000:2.0:1788619915.218662:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571432576 00000100:00100000:2.0:1788619915.218724:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 721 00000100:00100000:2.0:1788619915.218735:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571432576:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619915.218752:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619915.218770:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000040c3172a/0x1cd7fe536873491c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bd6c expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619915.218839:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5ff4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589105408:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619915.218922:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619915.224478:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5ff4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589105408:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619915.224588:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000040c3172a/0x1cd7fe536873491c lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd6c expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.224616:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000040c3172a/0x1cd7fe536873491c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd6c expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.224651:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079400, rc 0) 00000100:00100000:2.0:1788619915.224714:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571432576:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5981us (6195us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619915.224723:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 721 00000100:00100000:3.0:1788619915.235019:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619915.235114:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571432704 00000100:00100000:2.0:1788619915.235125:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571432704 00002000:00100000:2.0:1788619915.235178:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900d5ce00 x1875503571432704/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:11/0 lens 488/0 e 0 to 0 dl 1788619926 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619915.235190:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900d5ce00 with x1875503571432704 00000100:00100000:2.0:1788619915.235240:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 575 00000100:00100000:2.0:1788619915.235250:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+14:2354:x1875503571432704:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619915.235546:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+14:2354:x1875503571432704:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 297us (539us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619915.235553:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 575 00002000:00100000:2.0:1788619915.235560:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900d5ce00 x1875503571432704/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:11/0 lens 488/440 e 0 to 0 dl 1788619926 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619915.235567:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900d5ce00 with x1875503571432704 00000100:00100000:0.0:1788619915.242887:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619915.243141:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571432832 00000100:00100000:3.0:1788619915.243152:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571432832 00000100:00100000:1.0:1788619915.243825:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 722 00000100:00100000:1.0:1788619915.243840:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571432832:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619915.243861:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619915.243878:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f4c11a7a/0x1cd7fe5368734923 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bd73 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619915.244108:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c7583800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589105536:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619915.244252:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c780 going to sleep for 11 seconds 00000100:00100000:1.0:1788619915.246840:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c7583800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589105536:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619915.246883:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f4c11a7a/0x1cd7fe5368734923 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd73 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.246910:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f4c11a7a/0x1cd7fe5368734923 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd73 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.246925:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752000, rc 0) 00000100:00100000:1.0:1788619915.247974:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571432832:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4132us (5089us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619915.247985:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 722 00000100:00100000:2.0:1788619915.251899:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619915.252459:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571432960 00000100:00100000:1.0:1788619915.252468:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571432960 00000100:00100000:1.0:1788619915.252564:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 723 00000100:00100000:1.0:1788619915.252575:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571432960:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619915.252601:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619915.252619:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd2e9c29/0x1cd7fe536873492a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bd7a expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619915.252789:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c7583800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589105664:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619915.252884:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c780 going to sleep for 11 seconds 00000100:00100000:1.0:1788619915.256184:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c7583800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589105664:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619915.256226:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bd2e9c29/0x1cd7fe536873492a lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd7a expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.256275:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd2e9c29/0x1cd7fe536873492a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd7a expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.256289:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752600, rc 0) 00000100:00100000:1.0:1788619915.256388:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571432960:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3815us (4498us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619915.256398:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 723 00000100:00100000:3.0:1788619915.259224:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619915.259370:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571433088 00000100:00100000:2.0:1788619915.259382:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571433088 00002000:00100000:2.0:1788619915.259433:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900d5c000 x1875503571433088/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:11/0 lens 488/0 e 0 to 0 dl 1788619926 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619915.259445:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900d5c000 with x1875503571433088 00000100:00100000:2.0:1788619915.259477:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 576 00000100:00100000:2.0:1788619915.259500:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+14:2353:x1875503571433088:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619915.259766:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+14:2353:x1875503571433088:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 267us (552us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619915.259774:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 576 00002000:00100000:2.0:1788619915.259782:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900d5c000 x1875503571433088/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:11/0 lens 488/440 e 0 to 0 dl 1788619926 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619915.259789:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900d5c000 with x1875503571433088 00000100:00100000:2.0:1788619915.263477:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619915.263569:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571433216 00000100:00100000:0.0:1788619915.263579:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571433216 00000100:00100000:0.0:1788619915.263634:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 724 00000100:00100000:0.0:1788619915.263653:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571433216:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619915.263672:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619915.263687:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ce178caa/0x1cd7fe5368734931 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bd81 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619915.263740:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cd5f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589105792:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619915.263822:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619915.266969:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cd5f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589105792:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619915.267013:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ce178caa/0x1cd7fe5368734931 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd81 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.267038:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ce178caa/0x1cd7fe5368734931 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd81 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.267052:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767ac00, rc 0) 00000100:00100000:1.0:1788619915.267123:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571433216:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3471us (3654us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619915.267130:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 724 00000100:00100000:3.0:1788619915.271290:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619915.271841:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571433344 00000100:00100000:2.0:1788619915.271851:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571433344 00000100:00100000:2.0:1788619915.271914:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 725 00000100:00100000:2.0:1788619915.271924:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571433344:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619915.271941:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619915.271961:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dfdc9508/0x1cd7fe5368734938 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bd88 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619915.272022:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5ff7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589105920:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619915.272112:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619915.276080:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5ff7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589105920:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619915.276180:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000dfdc9508/0x1cd7fe5368734938 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd88 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619915.276286:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dfdc9508/0x1cd7fe5368734938 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd88 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619915.276302:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8078200, rc 0) 00000100:00100000:0.0:1788619915.276508:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571433344:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4583us (5224us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619915.276517:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 725 00000100:00100000:0.0:1788619915.280039:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619915.280369:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571433472 00000100:00100000:2.0:1788619915.280379:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571433472 00002000:00100000:2.0:1788619915.280430:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5cd7100 x1875503571433472/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:11/0 lens 488/0 e 0 to 0 dl 1788619926 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619915.280441:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5cd7100 with x1875503571433472 00000100:00100000:2.0:1788619915.280476:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 577 00000100:00100000:2.0:1788619915.280487:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+14:2352:x1875503571433472:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619915.280779:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+14:2352:x1875503571433472:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 291us (748us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619915.280787:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 577 00002000:00100000:2.0:1788619915.280795:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5cd7100 x1875503571433472/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:11/0 lens 488/440 e 0 to 0 dl 1788619926 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619915.280802:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5cd7100 with x1875503571433472 00000100:00100000:3.0:1788619915.287253:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619915.287319:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571433600 00000100:00100000:1.0:1788619915.287424:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571433600 00000100:00100000:1.0:1788619915.287520:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 726 00000100:00100000:1.0:1788619915.287530:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571433600:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619915.287549:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619915.287579:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd2e9c29/0x1cd7fe536873493f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bd8f expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619915.287634:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c7583800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589106048:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619915.287714:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c780 going to sleep for 11 seconds 00000100:00100000:1.0:1788619915.291512:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c7583800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589106048:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619915.291553:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bd2e9c29/0x1cd7fe536873493f lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd8f expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.291586:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd2e9c29/0x1cd7fe536873493f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd8f expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.291600:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752600, rc 0) 00000100:00100000:1.0:1788619915.291659:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571433600:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4129us (4415us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619915.291667:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 726 00000100:00100000:3.0:1788619915.295389:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619915.295437:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571433728 00000100:00100000:1.0:1788619915.295446:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571433728 00000100:00100000:1.0:1788619915.295506:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 727 00000100:00100000:1.0:1788619915.295517:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571433728:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619915.295536:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619915.295554:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f4c11a7a/0x1cd7fe5368734946 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bd96 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619915.295615:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c7583800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589106176:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619915.295711:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c780 going to sleep for 11 seconds 00000100:00100000:1.0:1788619915.298542:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c7583800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589106176:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619915.298586:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f4c11a7a/0x1cd7fe5368734946 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd96 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.298613:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f4c11a7a/0x1cd7fe5368734946 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd96 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.298626:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752000, rc 0) 00000100:00100000:1.0:1788619915.298688:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571433728:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3173us (3308us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619915.298697:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 727 00000100:00100000:3.0:1788619915.302539:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619915.302591:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571433856 00000100:00100000:1.0:1788619915.302602:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571433856 00002000:00100000:1.0:1788619915.302649:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5d54e00 x1875503571433856/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:11/0 lens 488/0 e 0 to 0 dl 1788619926 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619915.302661:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5d54e00 with x1875503571433856 00000100:00100000:1.0:1788619915.302699:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 578 00000100:00100000:1.0:1788619915.302708:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d54e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+14:2353:x1875503571433856:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619915.302998:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d54e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+14:2353:x1875503571433856:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 283us (461us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619915.303006:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 578 00002000:00100000:1.0:1788619915.303014:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5d54e00 x1875503571433856/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:11/0 lens 488/440 e 0 to 0 dl 1788619926 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619915.303041:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5d54e00 with x1875503571433856 00000100:00100000:2.0:1788619915.306835:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619915.307395:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571433984 00000100:00100000:0.0:1788619915.307404:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571433984 00000100:00100000:0.0:1788619915.307515:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 728 00000100:00100000:0.0:1788619915.307540:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571433984:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619915.307558:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619915.307582:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000085e95fb9/0x1cd7fe536873494d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bd9d expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619915.307645:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cd4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589106304:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619915.307733:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619915.313933:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cd4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589106304:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619915.313981:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000085e95fb9/0x1cd7fe536873494d lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd9d expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619915.314016:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000085e95fb9/0x1cd7fe536873494d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd9d expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619915.314031:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767a400, rc 0) 00000100:00100000:0.0:1788619915.314094:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571433984:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6556us (7278us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619915.314102:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 728 00000100:00100000:3.0:1788619915.317689:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619915.317778:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571434112 00000100:00100000:0.0:1788619915.317788:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571434112 00000100:00100000:0.0:1788619915.317862:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 729 00000100:00100000:0.0:1788619915.317873:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d57b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571434112:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619915.317890:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619915.317908:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000033d84a7d/0x1cd7fe5368734954 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bda4 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619915.317983:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cd4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589106432:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619915.318064:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619915.320261:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cd4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589106432:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619915.320300:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000033d84a7d/0x1cd7fe5368734954 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bda4 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619915.320345:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000033d84a7d/0x1cd7fe5368734954 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bda4 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619915.320359:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767b200, rc 0) 00000100:00100000:0.0:1788619915.320415:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d57b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571434112:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2543us (2733us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619915.320423:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 729 00000100:00100000:3.0:1788619915.323054:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619915.323181:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571434240 00000100:00100000:0.0:1788619915.323191:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571434240 00002000:00100000:0.0:1788619915.323241:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5cd9500 x1875503571434240/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:11/0 lens 488/0 e 0 to 0 dl 1788619926 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619915.323251:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5cd9500 with x1875503571434240 00000100:00100000:0.0:1788619915.323289:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 579 00000100:00100000:0.0:1788619915.323298:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+14:2352:x1875503571434240:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619915.323577:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+14:2352:x1875503571434240:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 279us (530us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619915.323585:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 579 00002000:00100000:0.0:1788619915.323605:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5cd9500 x1875503571434240/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:11/0 lens 488/440 e 0 to 0 dl 1788619926 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619915.323613:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5cd9500 with x1875503571434240 00000100:00100000:3.0:1788619915.326783:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619915.326943:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571434368 00000100:00100000:1.0:1788619915.326953:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571434368 00000100:00100000:1.0:1788619915.327006:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 730 00000100:00100000:1.0:1788619915.327016:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571434368:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619915.327034:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619915.327051:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f4c11a7a/0x1cd7fe536873495b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bdab expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619915.327121:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c7583800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589106560:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619915.327207:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c780 going to sleep for 11 seconds 00000100:00100000:1.0:1788619915.329439:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c7583800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589106560:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619915.329476:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f4c11a7a/0x1cd7fe536873495b lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bdab expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.329501:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f4c11a7a/0x1cd7fe536873495b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bdab expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.329515:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752000, rc 0) 00000100:00100000:1.0:1788619915.329572:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571434368:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2557us (2796us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619915.329580:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 730 00000100:00100000:3.0:1788619915.332937:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619915.333011:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571434496 00000100:00100000:1.0:1788619915.333021:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571434496 00000100:00100000:1.0:1788619915.333102:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 731 00000100:00100000:1.0:1788619915.333113:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571434496:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619915.333130:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619915.333148:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd2e9c29/0x1cd7fe5368734962 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bdb2 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619915.333208:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c7583800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589106688:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619915.333293:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c780 going to sleep for 11 seconds 00000100:00100000:1.0:1788619915.335476:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c7583800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589106688:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619915.335517:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bd2e9c29/0x1cd7fe5368734962 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bdb2 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.335541:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd2e9c29/0x1cd7fe5368734962 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bdb2 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.335554:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752600, rc 0) 00000100:00100000:1.0:1788619915.335606:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571434496:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2494us (2675us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619915.335615:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 731 00000100:00100000:3.0:1788619915.338147:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619915.338578:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571434624 00000100:00100000:2.0:1788619915.338587:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571434624 00002000:00100000:2.0:1788619915.338650:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5cda300 x1875503571434624/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:11/0 lens 488/0 e 0 to 0 dl 1788619926 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619915.338660:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5cda300 with x1875503571434624 00000100:00100000:2.0:1788619915.338699:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 580 00000100:00100000:2.0:1788619915.338709:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cda300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+14:2353:x1875503571434624:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619915.338970:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cda300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+14:2353:x1875503571434624:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 261us (831us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619915.338978:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 580 00002000:00100000:2.0:1788619915.338994:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5cda300 x1875503571434624/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:11/0 lens 488/440 e 0 to 0 dl 1788619926 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619915.339003:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5cda300 with x1875503571434624 00000100:00100000:3.0:1788619915.342425:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619915.343166:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571434752 00000100:00100000:0.0:1788619915.343237:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571434752 00000100:00100000:0.0:1788619915.343356:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 732 00000100:00100000:0.0:1788619915.343368:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a89880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571434752:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619915.343385:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619915.343403:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000033d84a7d/0x1cd7fe5368734969 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bdb9 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619915.343464:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cd4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589106816:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619915.343591:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619915.346542:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cd4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589106816:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619915.346585:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000033d84a7d/0x1cd7fe5368734969 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bdb9 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.346646:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000033d84a7d/0x1cd7fe5368734969 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bdb9 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.346659:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767b200, rc 0) 00000100:00100000:2.0:1788619915.346712:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a89880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571434752:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3347us (4296us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619915.346722:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 732 00000100:00100000:2.0:1788619915.350285:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619915.350538:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571434880 00000100:00100000:3.0:1788619915.350547:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571434880 00000100:00100000:3.0:1788619915.350603:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 733 00000100:00100000:3.0:1788619915.350628:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571434880:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619915.350654:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619915.350673:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c4fc38e/0x1cd7fe5368734970 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bdc0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619915.350732:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a88e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589106944:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619915.350825:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c2c5ef00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619915.353739:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a88e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589106944:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619915.353779:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004c4fc38e/0x1cd7fe5368734970 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bdc0 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.353805:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c4fc38e/0x1cd7fe5368734970 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bdc0 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.353831:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683ce00, rc 0) 00000100:00100000:2.0:1788619915.353887:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571434880:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3262us (3610us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619915.353896:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 733 00000100:00100000:2.0:1788619915.357417:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619915.357492:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571435008 00000100:00100000:1.0:1788619915.357502:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571435008 00002000:00100000:1.0:1788619915.357551:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5ff7480 x1875503571435008/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:11/0 lens 488/0 e 0 to 0 dl 1788619926 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619915.357563:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5ff7480 with x1875503571435008 00000100:00100000:1.0:1788619915.357600:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 581 00000100:00100000:1.0:1788619915.357615:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+14:2352:x1875503571435008:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619915.357889:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+14:2352:x1875503571435008:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 273us (481us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619915.357897:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 581 00002000:00100000:1.0:1788619915.357905:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5ff7480 x1875503571435008/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:11/0 lens 488/440 e 0 to 0 dl 1788619926 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619915.357912:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5ff7480 with x1875503571435008 00000100:00100000:0.0:1788619915.361173:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619915.361249:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571435136 00000100:00100000:1.0:1788619915.361258:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571435136 00000100:00100000:1.0:1788619915.361313:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 734 00000100:00100000:1.0:1788619915.361323:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571435136:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619915.361370:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619915.361389:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd2e9c29/0x1cd7fe5368734977 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bdc7 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619915.361446:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c7583800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589107072:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619915.361533:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c780 going to sleep for 11 seconds 00000100:00100000:2.0:1788619915.363480:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c7581180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589107200:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1788619915.363543:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619915.363632:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589107200 00000100:00100000:3.0:1788619915.363640:0:6699:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589107200 00000100:00100000:3.0:1788619915.363708:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 87 00000100:00100000:3.0:1788619915.363717:0:6699:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d00700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589107200:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1788619915.363739:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 107282432 pending 0 free 3859808256 avail 3852468224 00000100:00080000:0.0:1788619915.363807:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1788619915.364700:0:6699:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d00700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589107200:12345-0@lo:13:osp-pre-0-0.0 Request processed in 980us (1152us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619915.364712:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 87 00000100:00100000:1.0:1788619915.364959:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c7583800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589107072:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619915.364992:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bd2e9c29/0x1cd7fe5368734977 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bdc7 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.365018:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd2e9c29/0x1cd7fe5368734977 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bdc7 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.365050:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752600, rc 0) 00000100:00100000:1.0:1788619915.365108:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571435136:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3787us (3944us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619915.365116:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 734 00000100:00100000:3.0:1788619915.368027:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619915.368208:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571435264 00000100:00100000:1.0:1788619915.368218:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571435264 00000100:00100000:1.0:1788619915.368281:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 735 00000100:00100000:1.0:1788619915.368291:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a8bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571435264:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619915.368309:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619915.368352:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f4c11a7a/0x1cd7fe536873497e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bdce expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619915.368411:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c7583800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589107328:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619915.368483:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c780 going to sleep for 11 seconds 00000100:00100000:1.0:1788619915.370536:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c7583800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589107328:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619915.370574:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f4c11a7a/0x1cd7fe536873497e lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bdce expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.370598:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f4c11a7a/0x1cd7fe536873497e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bdce expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.370611:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752000, rc 0) 00000100:00100000:1.0:1788619915.370680:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a8bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571435264:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2390us (2662us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619915.370704:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 735 00000100:00100000:0.0:1788619915.373068:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619915.373242:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571435392 00000100:00100000:3.0:1788619915.373251:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571435392 00002000:00100000:3.0:1788619915.373298:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5cd7480 x1875503571435392/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:11/0 lens 488/0 e 0 to 0 dl 1788619926 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619915.373309:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5cd7480 with x1875503571435392 00000100:00100000:3.0:1788619915.373376:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 582 00000100:00100000:3.0:1788619915.373389:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+14:2352:x1875503571435392:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619915.373649:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+14:2352:x1875503571435392:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 263us (589us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619915.373656:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 582 00002000:00100000:3.0:1788619915.373679:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5cd7480 x1875503571435392/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:11/0 lens 488/440 e 0 to 0 dl 1788619926 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619915.373686:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5cd7480 with x1875503571435392 00000100:00100000:2.0:1788619915.374247:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c7581180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589107200:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1788619915.376969:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619915.377181:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571435520 00000100:00100000:2.0:1788619915.377192:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571435520 00000100:00100000:2.0:1788619915.377257:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 736 00000100:00100000:2.0:1788619915.377268:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571435520:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619915.377287:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619915.377306:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000855993/0x1cd7fe5368734985 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bdd5 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619915.377396:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6d02300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589107456:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619915.377480:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccd80 going to sleep for 11 seconds 00000100:00100000:2.0:1788619915.379253:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6d02300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589107456:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619915.379289:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000000855993/0x1cd7fe5368734985 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bdd5 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.379313:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000855993/0x1cd7fe5368734985 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bdd5 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.379356:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079c00, rc 0) 00000100:00100000:2.0:1788619915.379407:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571435520:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2142us (2447us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619915.379416:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 736 00000100:00100000:0.0:1788619915.382841:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619915.382926:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571435648 00000100:00100000:3.0:1788619915.382936:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571435648 00000100:00100000:3.0:1788619915.382997:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 737 00000100:00100000:3.0:1788619915.383008:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571435648:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619915.383028:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619915.383047:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe536873498c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bddc expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619915.383121:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a8a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589107584:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619915.383214:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c2c5ea80 going to sleep for 11 seconds 00000100:00100000:3.0:1788619915.385626:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a8a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589107584:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619915.385678:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe536873498c lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bddc expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619915.385702:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe536873498c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bddc expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619915.385715:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683dc00, rc 0) 00000100:00100000:3.0:1788619915.385771:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571435648:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2765us (2940us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619915.385780:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 737 00000100:00100000:0.0:1788619915.388612:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619915.388715:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571435776 00000100:00100000:2.0:1788619915.388725:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571435776 00002000:00100000:2.0:1788619915.388776:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5cd5f80 x1875503571435776/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:11/0 lens 488/0 e 0 to 0 dl 1788619926 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619915.388806:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5cd5f80 with x1875503571435776 00000100:00100000:2.0:1788619915.388847:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 583 00000100:00100000:2.0:1788619915.388856:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+14:2352:x1875503571435776:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619915.389161:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+14:2352:x1875503571435776:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 305us (558us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619915.389168:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 583 00002000:00100000:2.0:1788619915.389176:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5cd5f80 x1875503571435776/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:11/0 lens 488/440 e 0 to 0 dl 1788619926 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619915.389184:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5cd5f80 with x1875503571435776 00000100:00100000:0.0:1788619915.392305:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619915.392511:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571435904 00000100:00100000:1.0:1788619915.392521:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571435904 00000100:00100000:1.0:1788619915.392581:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 738 00000100:00100000:1.0:1788619915.392591:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571435904:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619915.392608:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619915.392625:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f4c11a7a/0x1cd7fe5368734993 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bde3 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619915.392695:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c7583800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589107712:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619915.392776:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c780 going to sleep for 11 seconds 00000100:00100000:1.0:1788619915.394974:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c7583800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589107712:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619915.395011:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f4c11a7a/0x1cd7fe5368734993 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bde3 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.395035:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f4c11a7a/0x1cd7fe5368734993 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bde3 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.395048:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752000, rc 0) 00000100:00100000:1.0:1788619915.395113:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571435904:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2523us (2816us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619915.395122:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 738 00000100:00100000:0.0:1788619915.398422:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619915.398516:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571436032 00000100:00100000:1.0:1788619915.398527:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571436032 00000100:00100000:1.0:1788619915.398600:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 739 00000100:00100000:1.0:1788619915.398610:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c30700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571436032:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619915.398629:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619915.398655:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd2e9c29/0x1cd7fe536873499a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bdea expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619915.398712:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c7583800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589107840:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619915.398786:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c780 going to sleep for 11 seconds 00000100:00100000:1.0:1788619915.400614:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c7583800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589107840:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619915.400658:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bd2e9c29/0x1cd7fe536873499a lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bdea expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.400683:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd2e9c29/0x1cd7fe536873499a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bdea expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.400696:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752600, rc 0) 00000100:00100000:1.0:1788619915.400752:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c30700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571436032:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2143us (2339us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619915.400761:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 739 00000100:00100000:0.0:1788619915.403501:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619915.403578:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571436160 00000100:00100000:1.0:1788619915.403587:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571436160 00002000:00100000:1.0:1788619915.403645:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5c31500 x1875503571436160/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:11/0 lens 488/0 e 0 to 0 dl 1788619926 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619915.403657:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5c31500 with x1875503571436160 00000100:00100000:1.0:1788619915.403694:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 584 00000100:00100000:1.0:1788619915.403718:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c31500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+14:2353:x1875503571436160:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619915.403991:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c31500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+14:2353:x1875503571436160:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 274us (499us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619915.403999:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 584 00002000:00100000:1.0:1788619915.404005:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5c31500 x1875503571436160/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:11/0 lens 488/440 e 0 to 0 dl 1788619926 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619915.404012:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5c31500 with x1875503571436160 00000100:00100000:0.0:1788619915.407096:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619915.407186:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571436288 00000100:00100000:2.0:1788619915.407195:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571436288 00000100:00100000:2.0:1788619915.407251:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 740 00000100:00100000:2.0:1788619915.407262:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c31880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571436288:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619915.407280:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619915.407297:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000855993/0x1cd7fe53687349a1 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bdf1 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619915.407381:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6d02300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589107968:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619915.407446:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccd80 going to sleep for 11 seconds 00000100:00100000:2.0:1788619915.408971:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6d02300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589107968:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619915.409006:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000000855993/0x1cd7fe53687349a1 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bdf1 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.409049:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000855993/0x1cd7fe53687349a1 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bdf1 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.409062:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079c00, rc 0) 00000100:00100000:2.0:1788619915.409120:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c31880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571436288:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 1859us (2045us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619915.409129:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 740 00000100:00100000:0.0:1788619915.412145:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619915.412234:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571436416 00000100:00100000:3.0:1788619915.412256:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571436416 00000100:00100000:3.0:1788619915.412320:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 741 00000100:00100000:3.0:1788619915.412365:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c77e3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571436416:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619915.412381:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619915.412399:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe53687349a8 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bdf8 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619915.412456:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a8a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589108096:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619915.412524:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c2c5ea80 going to sleep for 11 seconds 00000100:00100000:3.0:1788619915.415163:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a8a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589108096:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619915.415199:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe53687349a8 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bdf8 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619915.415224:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe53687349a8 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bdf8 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619915.415237:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683dc00, rc 0) 00000100:00100000:3.0:1788619915.415457:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c77e3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571436416:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3093us (3319us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619915.415464:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 741 00000100:00100000:0.0:1788619915.417861:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619915.417933:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571436544 00000100:00100000:3.0:1788619915.417942:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571436544 00002000:00100000:3.0:1788619915.417989:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c77e3480 x1875503571436544/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:11/0 lens 488/0 e 0 to 0 dl 1788619926 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619915.418001:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c77e3480 with x1875503571436544 00000100:00100000:3.0:1788619915.418050:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 585 00000100:00100000:3.0:1788619915.418059:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c77e3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+14:2352:x1875503571436544:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619915.418356:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c77e3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+14:2352:x1875503571436544:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 296us (503us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619915.418363:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 585 00002000:00100000:3.0:1788619915.418371:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c77e3480 x1875503571436544/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:11/0 lens 488/440 e 0 to 0 dl 1788619926 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619915.418378:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c77e3480 with x1875503571436544 00000100:00100000:0.0:1788619915.421223:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619915.421315:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571436672 00000100:00100000:1.0:1788619915.421363:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571436672 00000100:00100000:1.0:1788619915.421423:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 742 00000100:00100000:1.0:1788619915.421433:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c77e3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571436672:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619915.421450:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619915.421467:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd2e9c29/0x1cd7fe53687349af lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bdff expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619915.421523:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c7583800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589108224:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619915.421591:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c780 going to sleep for 11 seconds 00000100:00100000:1.0:1788619915.424007:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c7583800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589108224:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619915.424044:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bd2e9c29/0x1cd7fe53687349af lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bdff expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.424068:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd2e9c29/0x1cd7fe53687349af lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bdff expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.424115:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752600, rc 0) 00000100:00100000:1.0:1788619915.424164:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c77e3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571436672:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2731us (2950us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619915.424172:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 742 00000100:00100000:0.0:1788619915.427479:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619915.427585:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571436800 00000100:00100000:2.0:1788619915.427594:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571436800 00002000:00100000:2.0:1788619915.427655:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c77e2d80 x1875503571436800/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:11/0 lens 488/0 e 0 to 0 dl 1788619926 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619915.427666:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c77e2d80 with x1875503571436800 00000100:00100000:2.0:1788619915.427703:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 586 00000100:00100000:2.0:1788619915.427714:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c77e2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+14:2352:x1875503571436800:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:2.0:1788619915.430671:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 172969 -> 729088 00000004:00010000:2.0:1788619915.430680:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619914 -> 1788619915 00000004:00010000:2.0:1788619915.430682:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619914 -> 1788619914 00000004:00010000:2.0:1788619915.430684:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619914 -> 1788619915 00000004:00010000:2.0:1788619915.430686:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1414 -> 1414 00000100:00100000:2.0:1788619915.430751:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c77e2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+16:2352:x1875503571436800:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 3034us (3277us total) trans 4294967655 rc 0/0 00000100:00100000:2.0:1788619915.430762:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 586 00002000:00100000:2.0:1788619915.430776:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c77e2d80 x1875503571436800/t4294967655(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:11/0 lens 488/448 e 0 to 0 dl 1788619926 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619915.430797:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c77e2d80 with x1875503571436800 00000100:00100000:0.0:1788619915.434233:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619915.434371:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571436928 00000100:00100000:2.0:1788619915.434380:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571436928 00000100:00100000:2.0:1788619915.434443:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 933 00000100:00100000:2.0:1788619915.434455:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5772300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+16:2353:x1875503571436928:12345-192.168.201.5@tcp:16:fsx.0 00000100:00100000:3.0:1788619915.462393:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5772300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+16:2353:x1875503571436928:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 27931us (28159us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619915.462408:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 933 00000001:00080000:0.0:1788619915.462485:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967653 is committed 00000001:00080000:0.0:1788619915.462496:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967650 is committed 00000001:00080000:0.0:1788619915.462504:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967654 is committed 00000001:00080000:0.0:1788619915.462516:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967655 is committed 00000100:00100000:0.0:1788619915.465540:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619915.465691:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571437056 00000100:00100000:2.0:1788619915.465701:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571437056 00000100:00100000:2.0:1788619915.465760:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 743 00000100:00100000:2.0:1788619915.465769:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5770000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571437056:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619915.465786:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619915.465805:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000855993/0x1cd7fe53687349b6 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1be06 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619915.465880:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6d02300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589108352:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619915.465996:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccb40 going to sleep for 11 seconds 00000100:00100000:1.0:1788619915.468233:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6d02300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589108352:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619915.468268:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000000855993/0x1cd7fe53687349b6 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1be06 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.468291:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000855993/0x1cd7fe53687349b6 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1be06 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.468304:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079c00, rc 0) 00000100:00100000:1.0:1788619915.468458:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5770000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571437056:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2688us (2926us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619915.468466:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 743 00000100:00100000:0.0:1788619915.470970:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619915.471138:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571437184 00000100:00100000:2.0:1788619915.471147:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571437184 00000100:00100000:2.0:1788619915.471238:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 744 00000100:00100000:2.0:1788619915.471249:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5771880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571437184:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619915.471266:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619915.471282:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008447b954/0x1cd7fe53687349bd lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1be0d expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619915.471361:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6d01f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589108480:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619915.471443:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccb40 going to sleep for 11 seconds 00000100:00100000:2.0:1788619915.473564:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6d01f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589108480:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619915.473601:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008447b954/0x1cd7fe53687349bd lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1be0d expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.473627:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008447b954/0x1cd7fe53687349bd lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1be0d expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.473641:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079000, rc 0) 00000100:00100000:2.0:1788619915.473699:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5771880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571437184:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2452us (2737us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619915.473707:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 744 00000100:00100000:3.0:1788619915.476600:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619915.476781:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571437312 00000100:00100000:0.0:1788619915.476791:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571437312 00000100:00100000:0.0:1788619915.476870:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 381 00000100:00100000:0.0:1788619915.476881:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571437312:12345-192.168.201.5@tcp:22:fsx.0 00002000:00010000:0.0:1788619915.477168:0:6702:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 2074055 -> 2074055 00002000:00010000:0.0:1788619915.477171:0:6702:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619914 -> 1788619914 00002000:00010000:0.0:1788619915.477174:0:6702:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619914 -> 1788619914 00002000:00010000:0.0:1788619915.477176:0:6702:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619914 -> 1788619914 00002000:00010000:0.0:1788619915.477179:0:6702:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 519 -> 519 00000100:00100000:0.0:1788619915.477242:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a8a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2352:x1875503571437312:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 363us (651us total) trans 4294967412 rc 0/0 00000100:00100000:0.0:1788619915.477250:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 381 00000100:00100000:0.0:1788619915.479788:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619915.479879:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571437440 00000100:00100000:3.0:1788619915.479888:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571437440 00000100:00100000:3.0:1788619915.479951:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 745 00000100:00100000:3.0:1788619915.479960:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5770700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571437440:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619915.479977:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619915.479993:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe53687349c4 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1be14 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619915.480054:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a88a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589108608:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619915.480157:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa240 going to sleep for 11 seconds 00002000:00010000:0.0:1788619915.482548:0:6696:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime: 1788619914 -> 1788619915 00000100:00100000:0.0:1788619915.482583:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a88a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589108608:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619915.482618:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe53687349c4 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1be14 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619915.482642:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe53687349c4 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1be14 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619915.482669:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683dc00, rc 0) 00000100:00100000:0.0:1788619915.482766:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5770700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571437440:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2806us (2984us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619915.482774:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 745 00000100:00100000:3.0:1788619915.485672:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619915.485748:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571437568 00000100:00100000:1.0:1788619915.485758:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571437568 00000100:00100000:1.0:1788619915.485826:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 746 00000100:00100000:1.0:1788619915.485836:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a89500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571437568:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619915.485854:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619915.485871:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd2e9c29/0x1cd7fe53687349cb lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1be1b expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619915.485963:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c7583800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589108736:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619915.486376:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c240 going to sleep for 11 seconds 00000100:00100000:1.0:1788619915.488508:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c7583800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589108736:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619915.488547:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bd2e9c29/0x1cd7fe53687349cb lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1be1b expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.488572:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd2e9c29/0x1cd7fe53687349cb lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1be1b expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.488586:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752600, rc 0) 00000100:00100000:1.0:1788619915.488644:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a89500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571437568:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2798us (2969us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619915.488654:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 746 00000100:00100000:0.0:1788619915.492311:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619915.492479:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571437696 00000100:00100000:3.0:1788619915.492491:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571437696 00000100:00100000:3.0:1788619915.492605:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 747 00000100:00100000:3.0:1788619915.492629:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6342a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571437696:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619915.492656:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619915.492676:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001391204f/0x1cd7fe53687349d2 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1be22 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619915.492733:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a89f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589108864:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619915.492825:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619915.495098:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a89f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589108864:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619915.495136:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001391204f/0x1cd7fe53687349d2 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1be22 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619915.495163:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001391204f/0x1cd7fe53687349d2 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1be22 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619915.495175:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683da00, rc 0) 00000100:00100000:3.0:1788619915.495229:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6342a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571437696:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2602us (2928us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619915.495238:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 747 00000100:00100000:0.0:1788619915.498152:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619915.498250:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571437824 00000100:00100000:2.0:1788619915.498262:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571437824 00000100:00100000:1.0:1788619915.498599:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 748 00000100:00100000:1.0:1788619915.498612:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6343800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571437824:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619915.498631:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619915.498746:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f4c11a7a/0x1cd7fe53687349d9 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1be29 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619915.498804:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c7583800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589108992:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619915.499015:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c240 going to sleep for 11 seconds 00000100:00100000:1.0:1788619915.501430:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c7583800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589108992:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619915.501480:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f4c11a7a/0x1cd7fe53687349d9 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1be29 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.501503:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f4c11a7a/0x1cd7fe53687349d9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1be29 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.501516:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752000, rc 0) 00000100:00100000:1.0:1788619915.501596:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6343800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571437824:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2986us (3453us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619915.501604:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 748 00000100:00100000:0.0:1788619915.504709:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619915.504828:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571437952 00000100:00100000:2.0:1788619915.504838:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571437952 00000100:00100000:2.0:1788619915.504891:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 749 00000100:00100000:2.0:1788619915.504901:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6342300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571437952:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619915.509383:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619915.509410:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008447b954/0x1cd7fe53687349e0 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1be30 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619915.509476:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6d01f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589109120:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619915.509569:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccb40 going to sleep for 11 seconds 00000100:00100000:2.0:1788619915.511696:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6d01f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589109120:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619915.511737:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008447b954/0x1cd7fe53687349e0 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1be30 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.511783:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008447b954/0x1cd7fe53687349e0 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1be30 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.511797:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079000, rc 0) 00000100:00100000:2.0:1788619915.511881:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6342300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571437952:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6980us (7180us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619915.511890:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 749 00000100:00100000:1.0:1788619915.514509:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619915.514582:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571438080 00000100:00100000:2.0:1788619915.514592:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571438080 00000100:00100000:2.0:1788619915.514654:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 750 00000100:00100000:2.0:1788619915.514663:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7583800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571438080:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619915.514681:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619915.514698:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000040c3172a/0x1cd7fe53687349e7 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1be37 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619915.514757:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6d01f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589109248:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619915.514842:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccb40 going to sleep for 11 seconds 00000100:00100000:2.0:1788619915.517306:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6d01f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589109248:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619915.517370:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000040c3172a/0x1cd7fe53687349e7 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1be37 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.517396:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000040c3172a/0x1cd7fe53687349e7 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1be37 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.517425:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079400, rc 0) 00000100:00100000:2.0:1788619915.517476:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7583800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571438080:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2813us (2975us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619915.517484:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 750 00000100:00100000:3.0:1788619915.520218:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619915.520375:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571438208 00000100:00100000:0.0:1788619915.520385:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571438208 00000100:00100000:0.0:1788619915.520447:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 934 00000100:00100000:0.0:1788619915.520458:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a89f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2353:x1875503571438208:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619915.520478:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619915.520497:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a600/0x1cd7fe53687349ee lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1be3e expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619915.520533:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079600/0x1cd7fe53687348dd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd2d expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619915.520572:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6341f80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589109376:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619915.520653:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78eab85e40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619915.522995:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6341f80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589109376:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619915.523022:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 729088 00010000:00010000:0.0:1788619915.523042:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a600/0x1cd7fe53687349ee lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1be3e expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619915.523239:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619915.523243:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619915.523250:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a600/0x1cd7fe53687349ee lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1be3e expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619915.523265:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767a600, rc 0) 00000100:00100000:0.0:1788619915.523689:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a89f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2353:x1875503571438208:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2868us (3114us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619915.523701:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 934 00000100:00100000:1.0:1788619915.527299:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619915.527450:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571438336 00000100:00100000:2.0:1788619915.527460:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571438336 00000100:00100000:2.0:1788619915.527507:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 935 00000100:00100000:2.0:1788619915.527517:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7580a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2352:x1875503571438336:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619915.527538:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619915.527560:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe53687349f5 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1be45 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.527596:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079600/0x1cd7fe53687348dd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd2d expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619915.527634:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6d01f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589109504:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619915.527728:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccb40 going to sleep for 11 seconds 00000100:00100000:2.0:1788619915.529805:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6d01f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589109504:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619915.529842:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 729088 00010000:00010000:2.0:1788619915.529860:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe53687349f5 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1be45 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.529884:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619915.529888:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619915.529912:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe53687349f5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1be45 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.529926:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079000, rc 0) 00000100:00100000:2.0:1788619915.529983:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7580a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2352:x1875503571438336:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2467us (2694us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619915.529991:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 935 00000100:00100000:0.0:1788619915.533113:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619915.533230:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571438464 00000100:00100000:3.0:1788619915.533240:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571438464 00000100:00100000:3.0:1788619915.533294:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 751 00000100:00100000:3.0:1788619915.533305:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6341f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571438464:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619915.533322:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619915.533357:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001391204f/0x1cd7fe53687349fc lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1be4c expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619915.533423:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a8b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589109632:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619915.533503:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa9c0 going to sleep for 11 seconds 00002000:00010000:3.0:1788619915.539502:0:6696:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime: 1788619914 -> 1788619915 00000100:00100000:3.0:1788619915.539536:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a8b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589109632:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619915.539572:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001391204f/0x1cd7fe53687349fc lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1be4c expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619915.539597:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001391204f/0x1cd7fe53687349fc lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1be4c expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619915.539610:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683da00, rc 0) 00000100:00100000:3.0:1788619915.539738:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6341f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571438464:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6433us (6632us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619915.539747:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 751 00000100:00100000:0.0:1788619915.545725:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619915.545824:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571438592 00000100:00100000:1.0:1788619915.545836:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571438592 00002000:00100000:1.0:1788619915.545891:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6340e00 x1875503571438592/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:11/0 lens 488/0 e 0 to 0 dl 1788619926 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619915.545902:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6340e00 with x1875503571438592 00000100:00100000:1.0:1788619915.545941:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 587 00000100:00100000:1.0:1788619915.545951:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6340e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571438592:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619915.546186:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6340e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571438592:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 236us (472us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619915.546194:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 587 00002000:00100000:1.0:1788619915.546201:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6340e00 x1875503571438592/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:11/0 lens 488/4536 e 0 to 0 dl 1788619926 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619915.546209:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6340e00 with x1875503571438592 00000100:00100000:0.0:1788619915.549833:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619915.550207:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571438720 00000100:00100000:2.0:1788619915.550217:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571438720 00002000:00100000:2.0:1788619915.550264:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6341500 x1875503571438720/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:11/0 lens 488/0 e 0 to 0 dl 1788619926 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619915.550274:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6341500 with x1875503571438720 00000100:00100000:2.0:1788619915.550311:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 588 00000100:00100000:2.0:1788619915.550321:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6341500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571438720:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619915.550607:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6341500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571438720:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 286us (782us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619915.550616:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 588 00002000:00100000:0.0:1788619915.550623:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6341500 x1875503571438720/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:11/0 lens 488/4536 e 0 to 0 dl 1788619926 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619915.550640:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6341500 with x1875503571438720 00000100:00100000:0.0:1788619915.553688:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619915.553771:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571438848 00000100:00100000:3.0:1788619915.553780:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571438848 00002000:00100000:3.0:1788619915.553835:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6343100 x1875503571438848/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:11/0 lens 488/0 e 0 to 0 dl 1788619926 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619915.553846:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6343100 with x1875503571438848 00000100:00100000:3.0:1788619915.553881:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 589 00000100:00100000:3.0:1788619915.553891:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6343100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571438848:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619915.554123:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6343100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571438848:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 232us (444us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619915.554147:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 589 00002000:00100000:3.0:1788619915.554156:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6343100 x1875503571438848/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:11/0 lens 488/4536 e 0 to 0 dl 1788619926 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619915.554163:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6343100 with x1875503571438848 00000100:00100000:3.0:1788619915.557476:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619915.557581:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571438976 00000100:00100000:1.0:1788619915.557589:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571438976 00002000:00100000:1.0:1788619915.557638:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6a8b800 x1875503571438976/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:11/0 lens 488/0 e 0 to 0 dl 1788619926 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619915.557649:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6a8b800 with x1875503571438976 00000100:00100000:1.0:1788619915.557681:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 590 00000100:00100000:1.0:1788619915.557692:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a8b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571438976:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619915.557920:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a8b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571438976:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 230us (456us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619915.557928:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 590 00002000:00100000:1.0:1788619915.557934:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6a8b800 x1875503571438976/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:11/0 lens 488/4536 e 0 to 0 dl 1788619926 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619915.557942:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6a8b800 with x1875503571438976 00000100:00100000:1.0:1788619915.561176:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619915.561277:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571439104 00000100:00100000:0.0:1788619915.561287:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571439104 00002000:00100000:0.0:1788619915.561373:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c7582d80 x1875503571439104/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:11/0 lens 488/0 e 0 to 0 dl 1788619926 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619915.561385:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c7582d80 with x1875503571439104 00000100:00100000:0.0:1788619915.561410:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 591 00000100:00100000:0.0:1788619915.561435:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7582d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571439104:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619915.561806:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7582d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571439104:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 371us (639us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619915.561824:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 591 00002000:00100000:0.0:1788619915.561831:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c7582d80 x1875503571439104/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:11/0 lens 488/4536 e 0 to 0 dl 1788619926 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619915.561838:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c7582d80 with x1875503571439104 00000100:00100000:2.0:1788619915.565623:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619915.565698:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571439232 00000100:00100000:3.0:1788619915.565709:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571439232 00002000:00100000:3.0:1788619915.565764:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6d01f80 x1875503571439232/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:11/0 lens 488/0 e 0 to 0 dl 1788619926 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619915.565775:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6d01f80 with x1875503571439232 00000100:00100000:3.0:1788619915.565825:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 592 00000100:00100000:3.0:1788619915.565836:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d01f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571439232:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619915.566359:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d01f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571439232:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 522us (743us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619915.566367:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 592 00002000:00100000:3.0:1788619915.566376:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6d01f80 x1875503571439232/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:11/0 lens 488/4536 e 0 to 0 dl 1788619926 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619915.566384:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6d01f80 with x1875503571439232 00000100:00100000:2.0:1788619915.569933:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619915.570028:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571439360 00000100:00100000:3.0:1788619915.570038:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571439360 00000100:00100000:3.0:1788619915.570103:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 936 00000100:00100000:3.0:1788619915.570114:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d01c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2353:x1875503571439360:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619915.570133:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619915.570153:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe5368734a03 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1be53 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619915.570207:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079600/0x1cd7fe53687348dd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd2d expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619915.570247:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a88a80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589109760:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619915.570319:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaacc0 going to sleep for 11 seconds 00000004:00010000:1.0:1788619915.572563:0:11397:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619914 -> 1788619915 00000100:00100000:1.0:1788619915.572597:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a88a80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589109760:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619915.572618:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 729088 00010000:00010000:1.0:1788619915.572636:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe5368734a03 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1be53 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.572728:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619915.572731:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619915.572737:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe5368734a03 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1be53 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.572752:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683da00, rc 0) 00000100:00100000:1.0:1788619915.572875:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d01c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2353:x1875503571439360:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2762us (2950us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619915.572884:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 936 00000100:00100000:1.0:1788619915.575607:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619915.575756:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571439488 00000100:00100000:0.0:1788619915.575766:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571439488 00000100:00100000:0.0:1788619915.575867:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 937 00000100:00100000:0.0:1788619915.575878:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7580000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2353:x1875503571439488:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619915.575897:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619915.575929:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767b800/0x1cd7fe5368734a0a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1be5a expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619915.575968:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079600/0x1cd7fe53687348dd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd2d expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619915.576015:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5dca680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589109888:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619915.576113:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78eab85780 going to sleep for 11 seconds 00000100:00100000:0.0:1788619915.578282:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5dca680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589109888:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619915.578310:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 729088 00010000:00010000:0.0:1788619915.578364:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767b800/0x1cd7fe5368734a0a lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1be5a expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619915.578385:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619915.578404:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619915.578411:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767b800/0x1cd7fe5368734a0a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1be5a expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619915.578426:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767b800, rc 0) 00000100:00100000:0.0:1788619915.578478:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7580000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2353:x1875503571439488:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2601us (2880us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619915.578486:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 937 00000100:00100000:2.0:1788619915.581368:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619915.581483:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571439616 00000100:00100000:3.0:1788619915.581493:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571439616 00000100:00100000:3.0:1788619915.581563:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 382 00000100:00100000:3.0:1788619915.581574:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d02a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2352:x1875503571439616:12345-192.168.201.5@tcp:22:fsx.0 00002000:00010000:3.0:1788619915.581841:0:6703:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 2074055 -> 2074055 00002000:00010000:3.0:1788619915.581845:0:6703:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619915 -> 1788619914 00002000:00010000:3.0:1788619915.581847:0:6703:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619914 -> 1788619914 00002000:00010000:3.0:1788619915.581849:0:6703:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619915 -> 1788619914 00002000:00010000:3.0:1788619915.581851:0:6703:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 519 -> 519 00000100:00100000:3.0:1788619915.581993:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d02a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2352:x1875503571439616:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 420us (632us total) trans 4294967413 rc 0/0 00000100:00100000:3.0:1788619915.582001:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 382 00000100:00100000:3.0:1788619915.585200:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619915.585290:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571439744 00000100:00100000:0.0:1788619915.585299:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571439744 00000100:00100000:0.0:1788619915.585387:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 938 00000100:00100000:0.0:1788619915.585396:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a88e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2352:x1875503571439744:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619915.585416:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619915.585436:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a400/0x1cd7fe5368734a11 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1be61 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619915.585472:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079600/0x1cd7fe53687348dd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd2d expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619915.585512:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5dca680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589110016:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619915.585591:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78eab85780 going to sleep for 11 seconds 00000100:00100000:0.0:1788619915.588098:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5dca680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589110016:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619915.588125:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 729088 00010000:00010000:0.0:1788619915.588144:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a400/0x1cd7fe5368734a11 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1be61 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619915.588165:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619915.588182:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619915.588189:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a400/0x1cd7fe5368734a11 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1be61 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619915.588203:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767a400, rc 0) 00000100:00100000:0.0:1788619915.588295:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a88e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2352:x1875503571439744:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2899us (3103us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619915.588305:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 938 00000100:00100000:1.0:1788619915.591976:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619915.592066:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571439872 00000100:00100000:2.0:1788619915.592082:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571439872 00000100:00100000:2.0:1788619915.592130:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 939 00000100:00100000:2.0:1788619915.592137:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7583b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2352:x1875503571439872:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619915.592155:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619915.592171:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079800/0x1cd7fe5368734a18 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1be68 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.592204:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079600/0x1cd7fe53687348dd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd2d expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619915.592263:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6d03800 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589110144:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619915.592370:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccb40 going to sleep for 11 seconds 00000100:00100000:1.0:1788619915.595202:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6d03800 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589110144:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619915.595226:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 729088 00010000:00010000:1.0:1788619915.595243:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079800/0x1cd7fe5368734a18 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1be68 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.595259:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619915.595262:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619915.595266:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079800/0x1cd7fe5368734a18 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1be68 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.595276:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079800, rc 0) 00000100:00100000:1.0:1788619915.595398:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7583b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2352:x1875503571439872:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3259us (3429us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619915.595407:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 939 00000100:00100000:3.0:1788619915.598151:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619915.598238:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571440000 00000100:00100000:1.0:1788619915.598251:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571440000 00000100:00100000:1.0:1788619915.598311:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 752 00000100:00100000:1.0:1788619915.598321:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571440000:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619915.598376:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619915.598396:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001786a1ee/0x1cd7fe5368734a1f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1be6f expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619915.598478:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c7583480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589110272:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619915.598561:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c300 going to sleep for 11 seconds 00002000:00010000:1.0:1788619915.603670:0:6697:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime: 1788619914 -> 1788619915 00002000:00010000:1.0:1788619915.603683:0:6697:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619914 -> 1788619915 00002000:00010000:1.0:1788619915.603686:0:6697:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime: 1788619914 -> 1788619915 00000100:00100000:1.0:1788619915.603717:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c7583480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589110272:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619915.603755:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001786a1ee/0x1cd7fe5368734a1f lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1be6f expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.603782:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001786a1ee/0x1cd7fe5368734a1f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1be6f expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.603795:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5753200, rc 0) 00000100:00100000:1.0:1788619915.603867:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571440000:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5547us (5726us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619915.603877:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 752 00000100:00100000:0.0:1788619915.606833:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619915.606940:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571440128 00000100:00100000:2.0:1788619915.606950:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571440128 00000100:00100000:2.0:1788619915.607008:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 753 00000100:00100000:2.0:1788619915.607019:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe139850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571440128:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619915.607035:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619915.607055:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008447b954/0x1cd7fe5368734a26 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1be76 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619915.607122:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6d00000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589110400:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619915.607242:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccd80 going to sleep for 11 seconds 00000100:00100000:2.0:1788619915.609624:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6d00000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589110400:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619915.609712:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008447b954/0x1cd7fe5368734a26 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1be76 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.609736:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008447b954/0x1cd7fe5368734a26 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1be76 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.609748:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079000, rc 0) 00000100:00100000:2.0:1788619915.609847:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe139850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571440128:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2830us (3023us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619915.609856:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 753 00000100:00100000:2.0:1788619915.617985:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619915.618146:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571440256 00000100:00100000:0.0:1788619915.618156:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571440256 00000100:00100000:0.0:1788619915.618220:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 940 00000100:00100000:0.0:1788619915.618231:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d00000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2353:x1875503571440256:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619915.618253:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619915.618274:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767b200/0x1cd7fe5368734a2d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1be7d expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619915.618314:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079600/0x1cd7fe53687348dd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd2d expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619915.618561:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5dca680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589110528:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619915.618761:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78eab85780 going to sleep for 11 seconds 00000100:00100000:3.0:1788619915.621177:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5dca680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589110528:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619915.621208:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 729088 00010000:00010000:3.0:1788619915.621230:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767b200/0x1cd7fe5368734a2d lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1be7d expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619915.621251:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619915.621269:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619915.621276:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767b200/0x1cd7fe5368734a2d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1be7d expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619915.621292:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767b200, rc 0) 00000100:00100000:3.0:1788619915.621504:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d00000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2353:x1875503571440256:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3274us (3526us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619915.621513:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 940 00000100:00100000:1.0:1788619915.624278:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619915.624467:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571440384 00000100:00100000:0.0:1788619915.624476:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571440384 00000100:00100000:0.0:1788619915.624571:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 941 00000100:00100000:0.0:1788619915.624582:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7583480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2352:x1875503571440384:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619915.624610:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619915.624661:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767bc00/0x1cd7fe5368734a34 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1be84 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619915.624711:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079600/0x1cd7fe53687348dd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd2d expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619915.624755:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5dc8700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589110656:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619915.624857:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78eab85e40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619915.627208:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5dc8700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589110656:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619915.627236:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 729088 00010000:00010000:0.0:1788619915.627255:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767bc00/0x1cd7fe5368734a34 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1be84 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619915.627275:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619915.627278:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619915.627284:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767bc00/0x1cd7fe5368734a34 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1be84 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619915.627300:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767bc00, rc 0) 00000100:00100000:0.0:1788619915.627476:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7583480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2352:x1875503571440384:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2895us (3206us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619915.627485:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 941 00000100:00100000:2.0:1788619915.630273:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619915.630422:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571440512 00000100:00100000:0.0:1788619915.630433:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571440512 00000100:00100000:0.0:1788619915.630485:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 754 00000100:00100000:0.0:1788619915.630497:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571440512:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619915.630515:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619915.630547:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ce178caa/0x1cd7fe5368734a3b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1be8b expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619915.630607:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5dc8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589110784:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619915.630697:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78eab85e40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619915.632604:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5dc8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589110784:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619915.632653:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ce178caa/0x1cd7fe5368734a3b lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1be8b expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619915.632677:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ce178caa/0x1cd7fe5368734a3b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1be8b expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619915.632690:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767ac00, rc 0) 00000100:00100000:0.0:1788619915.632747:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571440512:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2251us (2481us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619915.632756:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 754 00000100:00100000:2.0:1788619915.635489:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619915.635574:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571440640 00000100:00100000:3.0:1788619915.635584:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571440640 00000100:00100000:3.0:1788619915.635648:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 755 00000100:00100000:3.0:1788619915.635658:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d01880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571440640:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619915.635676:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619915.635693:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007736e663/0x1cd7fe5368734a42 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1be92 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619915.635751:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a89880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589110912:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619915.635843:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaacc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619915.638242:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a89880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589110912:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619915.638281:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007736e663/0x1cd7fe5368734a42 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1be92 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619915.638304:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007736e663/0x1cd7fe5368734a42 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1be92 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619915.638318:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d400, rc 0) 00000100:00100000:3.0:1788619915.638429:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d01880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571440640:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2773us (2948us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619915.638437:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 755 00000100:00100000:1.0:1788619915.641707:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619915.641847:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571440768 00000100:00100000:2.0:1788619915.641857:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571440768 00002000:00100000:2.0:1788619915.641918:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1294336->1298431) req@ffff8e78c7583800 x1875503571440768/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/0 e 0 to 0 dl 1788619926 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619915.641927:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7583800 with x1875503571440768 ext(1294336->1298431) 00002000:00010000:2.0:1788619915.641933:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c7583800 00000100:00100000:2.0:1788619915.641966:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 383 00000100:00100000:2.0:1788619915.641976:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7583800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571440768:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619915.642503:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7583800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571440768:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 526us (803us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619915.642511:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 383 00002000:00100000:2.0:1788619915.642517:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1294336->1298431) req@ffff8e78c7583800 x1875503571440768/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/4536 e 0 to 0 dl 1788619926 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619915.642526:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7583800 with x1875503571440768 ext(1294336->1298431) 00002000:00010000:2.0:1788619915.642530:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c7583800 00000100:00100000:2.0:1788619915.646488:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619915.646574:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571440896 00000100:00100000:3.0:1788619915.646585:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571440896 00002000:00100000:3.0:1788619915.646693:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1298432->1302527) req@ffff8e77c6d03b80 x1875503571440896/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/0 e 0 to 0 dl 1788619926 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619915.646705:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6d03b80 with x1875503571440896 ext(1298432->1302527) 00002000:00010000:3.0:1788619915.646711:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6d03b80 00000100:00100000:3.0:1788619915.646800:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 384 00000100:00100000:3.0:1788619915.646820:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d03b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2352:x1875503571440896:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619915.647101:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d03b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2352:x1875503571440896:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 291us (623us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619915.647108:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 384 00002000:00100000:3.0:1788619915.647116:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1298432->1302527) req@ffff8e77c6d03b80 x1875503571440896/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/4536 e 0 to 0 dl 1788619926 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619915.647123:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6d03b80 with x1875503571440896 ext(1298432->1302527) 00002000:00010000:3.0:1788619915.647127:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6d03b80 00000100:00100000:2.0:1788619915.649840:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619915.649952:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571441024 00000100:00100000:1.0:1788619915.649962:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571441024 00002000:00100000:1.0:1788619915.650011:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1302528->1306623) req@ffff8e77c6d01500 x1875503571441024/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/0 e 0 to 0 dl 1788619926 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619915.650022:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6d01500 with x1875503571441024 ext(1302528->1306623) 00002000:00010000:1.0:1788619915.650042:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6d01500 00000100:00100000:1.0:1788619915.650172:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 385 00000100:00100000:1.0:1788619915.650182:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d01500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571441024:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619915.650571:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d01500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571441024:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 389us (738us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619915.650579:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 385 00002000:00100000:1.0:1788619915.650586:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1302528->1306623) req@ffff8e77c6d01500 x1875503571441024/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/4536 e 0 to 0 dl 1788619926 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619915.650595:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6d01500 with x1875503571441024 ext(1302528->1306623) 00002000:00010000:1.0:1788619915.650598:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6d01500 00000100:00100000:1.0:1788619915.654267:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619915.654440:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571441152 00000100:00100000:3.0:1788619915.654450:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571441152 00002000:00100000:0.0:1788619915.654574:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1306624->1310719) req@ffff8e78c7581180 x1875503571441152/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/0 e 0 to 0 dl 1788619926 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619915.654585:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7581180 with x1875503571441152 ext(1306624->1310719) 00002000:00010000:0.0:1788619915.654590:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c7581180 00000100:00100000:0.0:1788619915.654674:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 386 00000100:00100000:0.0:1788619915.654685:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7581180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2352:x1875503571441152:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619915.654949:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7581180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2352:x1875503571441152:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 264us (691us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619915.654970:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 386 00002000:00100000:0.0:1788619915.654977:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1306624->1310719) req@ffff8e78c7581180 x1875503571441152/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/4536 e 0 to 0 dl 1788619926 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619915.654986:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7581180 with x1875503571441152 ext(1306624->1310719) 00002000:00010000:0.0:1788619915.654990:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c7581180 00000100:00100000:3.0:1788619915.657555:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619915.657934:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571441280 00000100:00100000:2.0:1788619915.657944:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571441280 00002000:00100000:2.0:1788619915.657989:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1310720->1314815) req@ffff8e77c6a89880 x1875503571441280/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/0 e 0 to 0 dl 1788619926 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619915.658000:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a89880 with x1875503571441280 ext(1310720->1314815) 00002000:00010000:2.0:1788619915.658017:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a89880 00000100:00100000:2.0:1788619915.658038:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 387 00000100:00100000:2.0:1788619915.658047:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a89880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571441280:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619915.658377:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a89880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571441280:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 330us (830us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619915.658385:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 387 00002000:00100000:2.0:1788619915.658392:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1310720->1314815) req@ffff8e77c6a89880 x1875503571441280/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/4536 e 0 to 0 dl 1788619926 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619915.658399:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a89880 with x1875503571441280 ext(1310720->1314815) 00002000:00010000:2.0:1788619915.658402:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a89880 00000100:00100000:2.0:1788619915.661586:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619915.661721:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571441408 00000100:00100000:0.0:1788619915.661731:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571441408 00002000:00100000:0.0:1788619915.661778:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1314816->1318911) req@ffff8e77c6d03100 x1875503571441408/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/0 e 0 to 0 dl 1788619926 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619915.661788:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6d03100 with x1875503571441408 ext(1314816->1318911) 00002000:00010000:0.0:1788619915.661793:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6d03100 00000100:00100000:0.0:1788619915.661832:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 388 00000100:00100000:0.0:1788619915.661843:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d03100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2352:x1875503571441408:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619915.662104:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d03100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2352:x1875503571441408:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 262us (528us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619915.662111:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 388 00002000:00100000:0.0:1788619915.662119:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1314816->1318911) req@ffff8e77c6d03100 x1875503571441408/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/4536 e 0 to 0 dl 1788619926 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619915.662126:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6d03100 with x1875503571441408 ext(1314816->1318911) 00002000:00010000:0.0:1788619915.662130:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6d03100 00000100:00100000:2.0:1788619915.664648:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619915.664875:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571441536 00000100:00100000:3.0:1788619915.664887:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571441536 00002000:00100000:3.0:1788619915.664933:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1318912->1323007) req@ffff8e77c6d00380 x1875503571441536/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/0 e 0 to 0 dl 1788619926 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619915.664964:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6d00380 with x1875503571441536 ext(1318912->1323007) 00002000:00010000:3.0:1788619915.664969:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6d00380 00000100:00100000:3.0:1788619915.665001:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 389 00000100:00100000:3.0:1788619915.665010:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d00380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571441536:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619915.665266:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d00380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571441536:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 256us (642us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619915.665273:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 389 00002000:00100000:3.0:1788619915.665280:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1318912->1323007) req@ffff8e77c6d00380 x1875503571441536/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/4536 e 0 to 0 dl 1788619926 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619915.665286:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6d00380 with x1875503571441536 ext(1318912->1323007) 00002000:00010000:3.0:1788619915.665289:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6d00380 00000100:00100000:2.0:1788619915.667861:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619915.668584:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571441664 00000100:00100000:0.0:1788619915.668595:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571441664 00002000:00100000:0.0:1788619915.668703:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1323008->1327103) req@ffff8e77c6d02680 x1875503571441664/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/0 e 0 to 0 dl 1788619926 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619915.668733:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6d02680 with x1875503571441664 ext(1323008->1327103) 00002000:00010000:0.0:1788619915.668739:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6d02680 00000100:00100000:0.0:1788619915.668818:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 390 00000100:00100000:0.0:1788619915.668828:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d02680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2352:x1875503571441664:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619915.669191:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d02680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2352:x1875503571441664:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 363us (1338us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619915.669199:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 390 00002000:00100000:0.0:1788619915.669208:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1323008->1327103) req@ffff8e77c6d02680 x1875503571441664/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/4536 e 0 to 0 dl 1788619926 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619915.669215:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6d02680 with x1875503571441664 ext(1323008->1327103) 00002000:00010000:0.0:1788619915.669220:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6d02680 00000100:00100000:2.0:1788619915.671778:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619915.671867:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571441792 00000100:00100000:0.0:1788619915.671875:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571441792 00002000:00100000:0.0:1788619915.671918:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1327104->1331199) req@ffff8e77c6d00a80 x1875503571441792/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/0 e 0 to 0 dl 1788619926 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619915.671928:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6d00a80 with x1875503571441792 ext(1327104->1331199) 00002000:00010000:0.0:1788619915.671933:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6d00a80 00000100:00100000:0.0:1788619915.672002:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 391 00000100:00100000:0.0:1788619915.672011:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d00a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571441792:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619915.672269:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d00a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571441792:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 259us (500us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619915.672277:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 391 00002000:00100000:0.0:1788619915.672298:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1327104->1331199) req@ffff8e77c6d00a80 x1875503571441792/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/4536 e 0 to 0 dl 1788619926 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619915.672307:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6d00a80 with x1875503571441792 ext(1327104->1331199) 00002000:00010000:0.0:1788619915.672310:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6d00a80 00000100:00100000:1.0:1788619915.675719:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619915.676082:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571441920 00000100:00100000:0.0:1788619915.676095:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571441920 00002000:00100000:0.0:1788619915.676147:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1331200->1335295) req@ffff8e78c7581f80 x1875503571441920/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/0 e 0 to 0 dl 1788619926 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619915.676158:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7581f80 with x1875503571441920 ext(1331200->1335295) 00002000:00010000:0.0:1788619915.676164:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c7581f80 00000100:00100000:0.0:1788619915.676199:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 392 00000100:00100000:0.0:1788619915.676208:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7581f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2352:x1875503571441920:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619915.676723:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7581f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2352:x1875503571441920:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 514us (1012us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619915.676731:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 392 00002000:00100000:0.0:1788619915.676739:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1331200->1335295) req@ffff8e78c7581f80 x1875503571441920/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/4536 e 0 to 0 dl 1788619926 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619915.676746:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7581f80 with x1875503571441920 ext(1331200->1335295) 00002000:00010000:0.0:1788619915.676750:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c7581f80 00000100:00100000:2.0:1788619915.679785:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619915.679881:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571442048 00000100:00100000:0.0:1788619915.679892:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571442048 00002000:00100000:0.0:1788619915.679952:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1335296->1339391) req@ffff8e77c6d03800 x1875503571442048/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/0 e 0 to 0 dl 1788619926 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619915.679964:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6d03800 with x1875503571442048 ext(1335296->1339391) 00002000:00010000:0.0:1788619915.679985:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6d03800 00000100:00100000:0.0:1788619915.680018:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 393 00000100:00100000:0.0:1788619915.680028:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d03800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571442048:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619915.680296:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d03800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571442048:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 268us (521us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619915.680305:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 393 00002000:00100000:0.0:1788619915.680312:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1335296->1339391) req@ffff8e77c6d03800 x1875503571442048/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/4536 e 0 to 0 dl 1788619926 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619915.680320:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6d03800 with x1875503571442048 ext(1335296->1339391) 00002000:00010000:0.0:1788619915.680352:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6d03800 00000100:00100000:2.0:1788619915.683758:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619915.683857:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571442176 00000100:00100000:1.0:1788619915.683867:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571442176 00000100:00100000:1.0:1788619915.683919:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 942 00000100:00100000:1.0:1788619915.683929:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d02300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2352:x1875503571442176:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619915.683950:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619915.683971:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752000/0x1cd7fe5368734a49 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1be99 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.684009:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079600/0x1cd7fe53687348dd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd2d expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619915.684053:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79008ac000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589111040:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619915.684145:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48ccc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619915.686206:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79008ac000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589111040:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619915.686231:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 729088 00010000:00010000:2.0:1788619915.686250:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752000/0x1cd7fe5368734a49 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1be99 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.686272:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619915.686275:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619915.686280:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752000/0x1cd7fe5368734a49 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1be99 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.686311:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752000, rc 0) 00000100:00100000:2.0:1788619915.686452:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d02300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2352:x1875503571442176:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2524us (2703us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619915.686460:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 942 00000100:00100000:2.0:1788619915.689619:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619915.689757:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571442304 00000100:00100000:3.0:1788619915.689768:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571442304 00000100:00100000:3.0:1788619915.689878:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 943 00000100:00100000:3.0:1788619915.689890:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5695f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2353:x1875503571442304:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619915.689922:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619915.689944:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368734a50 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bea0 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619915.689981:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079600/0x1cd7fe53687348dd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd2d expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619915.690023:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a8bb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589111168:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619915.690154:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:3.0:1788619915.692725:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a8bb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589111168:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619915.692752:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 729088 00010000:00010000:3.0:1788619915.692772:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368734a50 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bea0 expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619915.692792:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619915.692795:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619915.692800:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368734a50 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bea0 expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619915.692828:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d400, rc 0) 00000100:00100000:3.0:1788619915.692990:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5695f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2353:x1875503571442304:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3101us (3379us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619915.693000:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 943 00000100:00100000:2.0:1788619915.696484:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619915.696580:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571442432 00000100:00100000:0.0:1788619915.696589:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571442432 00000100:00100000:0.0:1788619915.696757:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 394 00000100:00100000:0.0:1788619915.696769:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5697b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571442432:12345-192.168.201.5@tcp:22:fsx.0 00002000:00010000:0.0:1788619915.697065:0:6703:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 2074055 -> 2074055 00002000:00010000:0.0:1788619915.697069:0:6703:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619915 -> 1788619914 00002000:00010000:0.0:1788619915.697225:0:6703:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619915 -> 1788619914 00002000:00010000:0.0:1788619915.697230:0:6703:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619915 -> 1788619914 00002000:00010000:0.0:1788619915.697232:0:6703:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 519 -> 519 00000100:00100000:0.0:1788619915.697448:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5697b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2353:x1875503571442432:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 679us (973us total) trans 4294967414 rc 0/0 00000100:00100000:0.0:1788619915.697456:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 394 00000100:00100000:2.0:1788619915.700533:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619915.700626:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571442560 00000100:00100000:0.0:1788619915.700704:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571442560 00000100:00100000:0.0:1788619915.700797:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 944 00000100:00100000:0.0:1788619915.700832:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5696a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2353:x1875503571442560:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619915.700853:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619915.700874:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767bc00/0x1cd7fe5368734a57 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bea7 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619915.700924:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079600/0x1cd7fe53687348dd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd2d expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619915.700966:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5dc8700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589111296:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619915.701095:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78eab85780 going to sleep for 11 seconds 00000100:00100000:0.0:1788619915.707742:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5dc8700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589111296:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619915.707772:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 729088 00010000:00010000:0.0:1788619915.707792:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767bc00/0x1cd7fe5368734a57 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bea7 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619915.707826:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619915.707829:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619915.707835:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767bc00/0x1cd7fe5368734a57 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bea7 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619915.707850:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767bc00, rc 0) 00000100:00100000:0.0:1788619915.707962:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5696a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2353:x1875503571442560:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7132us (7441us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619915.707972:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 944 00000100:00100000:2.0:1788619915.711000:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619915.711159:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571442688 00000100:00100000:1.0:1788619915.711168:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571442688 00000100:00100000:1.0:1788619915.711269:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 945 00000100:00100000:1.0:1788619915.711279:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b13480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2353:x1875503571442688:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619915.711298:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619915.711318:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752a00/0x1cd7fe5368734a5e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1beae expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.711384:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079600/0x1cd7fe53687348dd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd2d expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619915.711425:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79008ae680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589111424:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619915.711500:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c000 going to sleep for 11 seconds 00000100:00100000:1.0:1788619915.713403:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79008ae680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589111424:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619915.713429:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 729088 00010000:00010000:1.0:1788619915.713447:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752a00/0x1cd7fe5368734a5e lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1beae expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.713468:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619915.713471:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619915.713476:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752a00/0x1cd7fe5368734a5e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1beae expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.713490:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752a00, rc 0) 00000100:00100000:1.0:1788619915.713542:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b13480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2353:x1875503571442688:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2264us (2550us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619915.713550:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 945 00000100:00100000:2.0:1788619915.716742:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619915.716845:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571442816 00000100:00100000:3.0:1788619915.716856:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571442816 00000100:00100000:3.0:1788619915.716918:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 756 00000100:00100000:3.0:1788619915.716928:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c08a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571442816:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619915.716945:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619915.716963:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007736e663/0x1cd7fe5368734a65 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1beb5 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619915.717020:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a8bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589111552:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619915.717110:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00002000:00010000:3.0:1788619915.719817:0:6697:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime: 1788619914 -> 1788619915 00002000:00010000:3.0:1788619915.719826:0:6697:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619914 -> 1788619915 00002000:00010000:3.0:1788619915.719828:0:6697:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime: 1788619914 -> 1788619915 00000100:00100000:3.0:1788619915.719867:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a8bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589111552:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619915.719902:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007736e663/0x1cd7fe5368734a65 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1beb5 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619915.719925:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007736e663/0x1cd7fe5368734a65 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1beb5 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619915.719938:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d400, rc 0) 00000100:00100000:3.0:1788619915.719992:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c08a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571442816:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3065us (3259us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619915.720000:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 756 00000100:00100000:2.0:1788619915.722606:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619915.722710:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571442944 00000100:00100000:0.0:1788619915.722720:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571442944 00000100:00100000:0.0:1788619915.722775:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 757 00000100:00100000:0.0:1788619915.722799:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c09180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571442944:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619915.722828:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619915.722846:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006989f8b0/0x1cd7fe5368734a6c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bebc expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619915.722899:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5dc8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589111680:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619915.722974:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78eab85780 going to sleep for 11 seconds 00000100:00100000:0.0:1788619915.725184:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5dc8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589111680:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619915.725222:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000006989f8b0/0x1cd7fe5368734a6c lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bebc expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619915.725247:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006989f8b0/0x1cd7fe5368734a6c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bebc expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619915.725260:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767bc00, rc 0) 00000100:00100000:0.0:1788619915.725318:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c09180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571442944:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2520us (2720us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619915.725366:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 757 00000100:00100000:2.0:1788619915.729148:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619915.729398:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571443072 00000100:00100000:3.0:1788619915.729408:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571443072 00002000:00100000:3.0:1788619915.729458:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1449984->1454079) req@ffff8e78c5c09500 x1875503571443072/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/0 e 0 to 0 dl 1788619926 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619915.729469:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5c09500 with x1875503571443072 ext(1449984->1454079) 00002000:00010000:3.0:1788619915.729475:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5c09500 00000100:00100000:3.0:1788619915.729507:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 395 00000100:00100000:3.0:1788619915.729517:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c09500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2353:x1875503571443072:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619915.730034:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c09500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2353:x1875503571443072:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 515us (892us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619915.730043:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 395 00002000:00100000:3.0:1788619915.730050:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1449984->1454079) req@ffff8e78c5c09500 x1875503571443072/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/4536 e 0 to 0 dl 1788619926 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619915.730089:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5c09500 with x1875503571443072 ext(1449984->1454079) 00002000:00010000:3.0:1788619915.730094:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5c09500 00000100:00100000:2.0:1788619915.733476:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619915.733570:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571443200 00000100:00100000:0.0:1788619915.733580:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571443200 00002000:00100000:0.0:1788619915.733626:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1454080->1458175) req@ffff8e7900d5e300 x1875503571443200/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/0 e 0 to 0 dl 1788619926 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619915.733696:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e7900d5e300 with x1875503571443200 ext(1454080->1458175) 00002000:00010000:0.0:1788619915.733701:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e7900d5e300 00000100:00100000:0.0:1788619915.733770:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 396 00000100:00100000:0.0:1788619915.733794:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2353:x1875503571443200:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619915.734122:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2353:x1875503571443200:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 328us (654us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619915.734129:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 396 00002000:00100000:0.0:1788619915.734137:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1454080->1458175) req@ffff8e7900d5e300 x1875503571443200/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/4536 e 0 to 0 dl 1788619926 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619915.734144:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e7900d5e300 with x1875503571443200 ext(1454080->1458175) 00002000:00010000:0.0:1788619915.734148:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e7900d5e300 00000100:00100000:2.0:1788619915.737302:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619915.737406:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571443328 00000100:00100000:3.0:1788619915.737416:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571443328 00002000:00100000:3.0:1788619915.737465:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1458176->1462271) req@ffff8e7900d5df80 x1875503571443328/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/0 e 0 to 0 dl 1788619926 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619915.737477:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e7900d5df80 with x1875503571443328 ext(1458176->1462271) 00002000:00010000:3.0:1788619915.737482:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e7900d5df80 00000100:00100000:3.0:1788619915.737545:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 397 00000100:00100000:3.0:1788619915.737555:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2353:x1875503571443328:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619915.737820:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2353:x1875503571443328:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 265us (527us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619915.737828:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 397 00002000:00100000:3.0:1788619915.737834:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1458176->1462271) req@ffff8e7900d5df80 x1875503571443328/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/4536 e 0 to 0 dl 1788619926 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619915.737841:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e7900d5df80 with x1875503571443328 ext(1458176->1462271) 00002000:00010000:3.0:1788619915.737845:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e7900d5df80 00000100:00100000:1.0:1788619915.740935:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619915.741031:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571443456 00000100:00100000:0.0:1788619915.741041:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571443456 00002000:00100000:0.0:1788619915.741142:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1462272->1466367) req@ffff8e79008ae680 x1875503571443456/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/0 e 0 to 0 dl 1788619926 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619915.741153:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e79008ae680 with x1875503571443456 ext(1462272->1466367) 00002000:00010000:0.0:1788619915.741159:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e79008ae680 00000100:00100000:0.0:1788619915.741233:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 398 00000100:00100000:0.0:1788619915.741243:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79008ae680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2352:x1875503571443456:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619915.741538:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79008ae680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2352:x1875503571443456:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 295us (611us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619915.741545:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 398 00002000:00100000:0.0:1788619915.741552:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1462272->1466367) req@ffff8e79008ae680 x1875503571443456/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/4536 e 0 to 0 dl 1788619926 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619915.741559:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e79008ae680 with x1875503571443456 ext(1462272->1466367) 00002000:00010000:0.0:1788619915.741562:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e79008ae680 00000100:00100000:3.0:1788619915.744145:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619915.744493:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571443584 00000100:00100000:2.0:1788619915.744504:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571443584 00002000:00100000:2.0:1788619915.744553:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1466368->1470463) req@ffff8e77c6a8bb80 x1875503571443584/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/0 e 0 to 0 dl 1788619926 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619915.744564:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a8bb80 with x1875503571443584 ext(1466368->1470463) 00002000:00010000:2.0:1788619915.744570:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a8bb80 00000100:00100000:2.0:1788619915.744591:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 399 00000100:00100000:2.0:1788619915.744601:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a8bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2353:x1875503571443584:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619915.744884:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a8bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2353:x1875503571443584:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 284us (747us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619915.744892:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 399 00002000:00100000:2.0:1788619915.744913:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1466368->1470463) req@ffff8e77c6a8bb80 x1875503571443584/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/4536 e 0 to 0 dl 1788619926 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619915.744921:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a8bb80 with x1875503571443584 ext(1466368->1470463) 00002000:00010000:2.0:1788619915.744925:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a8bb80 00000100:00100000:3.0:1788619915.747371:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619915.747857:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571443712 00000100:00100000:2.0:1788619915.747867:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571443712 00002000:00100000:2.0:1788619915.747913:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1470464->1474559) req@ffff8e77c6a89500 x1875503571443712/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/0 e 0 to 0 dl 1788619926 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619915.747923:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a89500 with x1875503571443712 ext(1470464->1474559) 00002000:00010000:2.0:1788619915.747929:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a89500 00000100:00100000:2.0:1788619915.747958:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 400 00000100:00100000:2.0:1788619915.747967:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a89500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2353:x1875503571443712:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619915.748319:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a89500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2353:x1875503571443712:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 352us (957us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619915.748355:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 400 00002000:00100000:2.0:1788619915.748363:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1470464->1474559) req@ffff8e77c6a89500 x1875503571443712/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/4536 e 0 to 0 dl 1788619926 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619915.748371:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a89500 with x1875503571443712 ext(1470464->1474559) 00002000:00010000:2.0:1788619915.748374:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a89500 00000100:00100000:3.0:1788619915.750728:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619915.750888:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571443840 00000100:00100000:2.0:1788619915.750897:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571443840 00002000:00100000:2.0:1788619915.750942:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1474560->1478655) req@ffff8e77c6a88a80 x1875503571443840/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/0 e 0 to 0 dl 1788619926 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619915.750952:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a88a80 with x1875503571443840 ext(1474560->1478655) 00002000:00010000:2.0:1788619915.750979:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a88a80 00000100:00100000:2.0:1788619915.751048:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 401 00000100:00100000:2.0:1788619915.751059:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2353:x1875503571443840:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619915.751296:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a88a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2353:x1875503571443840:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 238us (575us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619915.751304:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 401 00002000:00100000:2.0:1788619915.751311:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1474560->1478655) req@ffff8e77c6a88a80 x1875503571443840/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/4536 e 0 to 0 dl 1788619926 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619915.751319:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a88a80 with x1875503571443840 ext(1474560->1478655) 00002000:00010000:2.0:1788619915.751323:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a88a80 00000100:00100000:3.0:1788619915.753564:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619915.753952:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571443968 00000100:00100000:0.0:1788619915.753961:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571443968 00002000:00100000:0.0:1788619915.754006:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1478656->1482751) req@ffff8e78c4a4f100 x1875503571443968/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/0 e 0 to 0 dl 1788619926 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619915.754018:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c4a4f100 with x1875503571443968 ext(1478656->1482751) 00002000:00010000:0.0:1788619915.754023:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c4a4f100 00000100:00100000:0.0:1788619915.754057:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 402 00000100:00100000:0.0:1788619915.754066:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2352:x1875503571443968:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619915.754549:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2352:x1875503571443968:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 483us (994us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619915.754557:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 402 00002000:00100000:0.0:1788619915.754564:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1478656->1482751) req@ffff8e78c4a4f100 x1875503571443968/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/4536 e 0 to 0 dl 1788619926 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619915.754572:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c4a4f100 with x1875503571443968 ext(1478656->1482751) 00002000:00010000:0.0:1788619915.754576:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c4a4f100 00000100:00100000:3.0:1788619915.757612:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619915.757712:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571444096 00000100:00100000:2.0:1788619915.757722:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571444096 00002000:00100000:2.0:1788619915.757770:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1482752->1486847) req@ffff8e78c4a4dc00 x1875503571444096/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/0 e 0 to 0 dl 1788619926 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619915.757781:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c4a4dc00 with x1875503571444096 ext(1482752->1486847) 00002000:00010000:2.0:1788619915.757787:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c4a4dc00 00000100:00100000:2.0:1788619915.757827:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 403 00000100:00100000:2.0:1788619915.757837:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2353:x1875503571444096:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619915.758246:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2353:x1875503571444096:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 409us (643us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619915.758254:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 403 00002000:00100000:2.0:1788619915.758261:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1482752->1486847) req@ffff8e78c4a4dc00 x1875503571444096/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/4536 e 0 to 0 dl 1788619926 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619915.758269:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c4a4dc00 with x1875503571444096 ext(1482752->1486847) 00002000:00010000:2.0:1788619915.758288:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c4a4dc00 00000100:00100000:2.0:1788619915.761775:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619915.761848:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571444224 00000100:00100000:3.0:1788619915.761857:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571444224 00000100:00100000:3.0:1788619915.761934:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 946 00000100:00100000:3.0:1788619915.761946:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2352:x1875503571444224:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619915.761970:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619915.761994:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368734a73 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bec3 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619915.762033:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079600/0x1cd7fe53687348dd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd2d expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619915.762084:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c4a4f480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589111808:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619915.762435:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa240 going to sleep for 11 seconds 00000100:00100000:2.0:1788619915.764569:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c4a4f480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589111808:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619915.764596:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 729088 00010000:00010000:2.0:1788619915.764615:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368734a73 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bec3 expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.764701:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619915.764705:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619915.764711:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368734a73 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bec3 expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.764727:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683de00, rc 0) 00000100:00100000:2.0:1788619915.764824:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2352:x1875503571444224:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2880us (3060us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619915.764833:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 946 00000100:00100000:0.0:1788619915.768704:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619915.768799:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571444352 00000100:00100000:2.0:1788619915.768819:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571444352 00000100:00100000:2.0:1788619915.768918:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 947 00000100:00100000:2.0:1788619915.768929:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dc8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2353:x1875503571444352:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619915.768951:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619915.768982:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe5368734a7a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1beca expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.769022:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079600/0x1cd7fe53687348dd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd2d expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619915.769063:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900d5d880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589111936:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619915.769178:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc780 going to sleep for 11 seconds 00000100:00100000:2.0:1788619915.771436:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900d5d880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589111936:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619915.771460:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 729088 00010000:00010000:2.0:1788619915.771479:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe5368734a7a lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1beca expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.771498:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619915.771502:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619915.771507:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe5368734a7a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1beca expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.771522:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079c00, rc 0) 00000100:00100000:2.0:1788619915.771581:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dc8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2353:x1875503571444352:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2654us (2950us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619915.771590:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 947 00000100:00100000:3.0:1788619915.774423:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619915.774533:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571444480 00000100:00100000:1.0:1788619915.774544:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571444480 00000100:00100000:1.0:1788619915.774605:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 758 00000100:00100000:1.0:1788619915.774618:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:14908:x1875503571444480:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619915.774637:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619915.774656:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000848526f6/0x1cd7fe5368734a81 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bed1 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.774690:0:7654:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000dc75705a/0x1cd7fe53687348c8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1884160->1888255) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd18 expref: 8 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.774699:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=1s v=(12 8) 00010000:00010000:1.0:1788619915.774704:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(1 12) 00010000:00010000:1.0:1788619915.774710:0:7654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000848526f6/0x1cd7fe5368734a81 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1290240->1327103] (req 1290240->1327103) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bed1 expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.774744:0:7654:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000dc75705a/0x1cd7fe53687348c8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1884160->1888255) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd18 expref: 8 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.774758:0:7654:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000dc75705a/0x1cd7fe53687348c8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1884160->1888255) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd18 expref: 8 pid: 6698 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619915.774772:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79008aed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589112064:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619915.774876:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c000 going to sleep for 11 seconds 00000100:00100000:0.0:1788619915.778155:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619915.778256:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79008aed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589112064:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619915.778305:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000848526f6/0x1cd7fe5368734a81 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1290240->1327103] (req 1290240->1327103) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bed1 expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.778355:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752800, rc 0) 00000100:00100000:1.0:1788619915.778411:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:14908:x1875503571444480:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3794us (3998us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619915.778421:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 758 00000100:00100000:3.0:1788619915.778854:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571444608 00000100:00100000:3.0:1788619915.778861:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571444608 00010000:00010000:3.0:1788619915.778893:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000dc75705a/0x1cd7fe53687348c8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1884160->1888255) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd18 expref: 9 pid: 6698 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619915.778940:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 317 00000100:00100000:3.0:1788619915.778950:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dcb100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2355:x1875503571444608:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619915.778964:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619915.778989:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000dc75705a/0x1cd7fe53687348c8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1884160->1888255) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd18 expref: 9 pid: 6698 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619915.779001:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000dc75705a/0x1cd7fe53687348c8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1884160->1888255) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd18 expref: 9 pid: 6698 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619915.779027:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dc75705a/0x1cd7fe53687348c8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1884160->1888255) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd18 expref: 9 pid: 6698 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619915.779048:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(1 12) 00010000:00010000:3.0:1788619915.779051:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(1 12) 00010000:00010000:3.0:1788619915.779059:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000848526f6/0x1cd7fe5368734a81 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1290240->1327103) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bed1 expref: 5 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619915.779081:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000848526f6/0x1cd7fe5368734a81 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1290240->1327103) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bed1 expref: 5 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619915.779117:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000848526f6/0x1cd7fe5368734a81 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1290240->1327103) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bed1 expref: 5 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619915.779130:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c4a4ca80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589112192:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619915.779198:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaab40 going to sleep for 11 seconds 00000100:00100000:3.0:1788619915.781969:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c4a4ca80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589112192:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619915.781996:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619915.782091:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dcb100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571444608:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3141us (3945us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619915.782102:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 317 00000100:00100000:2.0:1788619915.784582:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619915.784832:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571444736 00000100:00100000:1.0:1788619915.784844:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571444736 00000100:00100000:1.0:1788619915.784928:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 404 00000100:00100000:1.0:1788619915.784938:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571444736:12345-192.168.201.5@tcp:22:fsx.0 00002000:00010000:1.0:1788619915.785395:0:6701:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 2074055 -> 2074055 00002000:00010000:1.0:1788619915.785400:0:6701:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619915 -> 1788619914 00002000:00010000:1.0:1788619915.785403:0:6701:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619915 -> 1788619914 00002000:00010000:1.0:1788619915.785406:0:6701:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619915 -> 1788619914 00002000:00010000:1.0:1788619915.785408:0:6701:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 519 -> 519 00000100:00100000:1.0:1788619915.785477:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2352:x1875503571444736:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 538us (903us total) trans 4294967415 rc 0/0 00000100:00100000:1.0:1788619915.785485:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 404 00000100:00100000:0.0:1788619915.799883:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619915.799992:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571444864 00000100:00100000:2.0:1788619915.800005:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571444864 00002000:00100000:2.0:1788619915.800058:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1048576->2076671) req@ffff8e78c5dc9f80 x1875503571444864/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:11/0 lens 488/0 e 0 to 0 dl 1788619926 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619915.800069:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5dc9f80 with x1875503571444864 ext(1048576->2076671) 00002000:00010000:2.0:1788619915.800084:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5dc9f80 00000100:00100000:2.0:1788619915.800117:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 405 00000100:00100000:2.0:1788619915.800126:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dc9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2353:x1875503571444864:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619915.820466:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619915.820565:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571444992 00000100:00100000:0.0:1788619915.820576:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571444992 00002000:00100000:0.0:1788619915.820642:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900d5f480 x1875503571444992/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:11/0 lens 488/0 e 0 to 0 dl 1788619926 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619915.820654:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900d5f480 with x1875503571444992 00000100:00100000:0.0:1788619915.820698:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 593 00000100:00100000:0.0:1788619915.820710:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571444992:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619915.824758:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571444992:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 4037us (4292us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619915.824773:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 593 00002000:00100000:0.0:1788619915.824788:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900d5f480 x1875503571444992/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:11/0 lens 488/440 e 0 to 0 dl 1788619926 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619915.824798:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900d5f480 with x1875503571444992 00000100:00100000:0.0:1788619915.836838:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dc9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2353:x1875503571444864:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 36696us (36948us total) trans 0 rc 1028096/1028096 00000100:00100000:0.0:1788619915.836859:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 405 00002000:00100000:0.0:1788619915.836877:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1048576->2076671) req@ffff8e78c5dc9f80 x1875503571444864/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:11/0 lens 488/440 e 0 to 0 dl 1788619926 ref 1 fl Complete:/600/0 rc 1028096/1028096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619915.837058:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5dc9f80 with x1875503571444864 ext(1048576->2076671) 00002000:00010000:0.0:1788619915.837065:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5dc9f80 00000100:00100000:3.0:1788619915.848691:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619915.848786:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619915.848984:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571445248 00000100:00100000:0.0:1788619915.848996:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571445248 00000100:00100000:0.0:1788619915.849060:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 759 00000100:00100000:0.0:1788619915.849071:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571445248:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619915.849089:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619915.849108:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000af64f6e6/0x1cd7fe5368734a88 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bedf expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619915.849169:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5dcad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589112320:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619915.849258:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78eab85e40 going to sleep for 11 seconds 00000100:00100000:1.0:1788619915.850452:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571445120 00000100:00100000:1.0:1788619915.850461:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571445120 00000100:00100000:1.0:1788619915.850611:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 948 00000100:00100000:1.0:1788619915.850623:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2355:x1875503571445120:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619915.850640:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619915.850658:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752200/0x1cd7fe5368734a8f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bed8 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.850717:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079600/0x1cd7fe53687348dd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd2d expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619915.850751:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79008aed80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589112448:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619915.850830:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c000 going to sleep for 11 seconds 00002000:00010000:3.0:1788619915.852736:0:6696:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime: 1788619914 -> 1788619915 00002000:00010000:3.0:1788619915.852744:0:6696:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619914 -> 1788619915 00002000:00010000:3.0:1788619915.852746:0:6696:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime: 1788619914 -> 1788619915 00000100:00100000:3.0:1788619915.852769:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5dcad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589112320:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619915.852826:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000af64f6e6/0x1cd7fe5368734a88 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bedf expref: 9 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619915.852852:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000af64f6e6/0x1cd7fe5368734a88 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bedf expref: 9 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619915.852866:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767ae00, rc 0) 00000100:00100000:3.0:1788619915.853106:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571445248:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4035us (4317us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619915.853116:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 759 00000100:00100000:1.0:1788619915.861987:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79008aed80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589112448:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619915.862014:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 729088 00010000:00010000:1.0:1788619915.862033:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752200/0x1cd7fe5368734a8f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bed8 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.862054:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619915.862058:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619915.862063:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752200/0x1cd7fe5368734a8f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bed8 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.862234:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752200, rc 0) 00000100:00100000:1.0:1788619915.862296:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2355:x1875503571445120:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 11676us (13615us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619915.862305:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 948 00000100:00100000:1.0:1788619915.868211:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619915.868297:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619915.869376:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571445504 00000100:00100000:3.0:1788619915.869387:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571445504 00000100:00100000:3.0:1788619915.869457:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 760 00000100:00100000:3.0:1788619915.869468:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79008aea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571445504:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619915.869489:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619915.869517:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a72c5200/0x1cd7fe5368734a96 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1beed expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619915.869574:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c4a4c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589112576:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619915.869688:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaab40 going to sleep for 11 seconds 00000100:00100000:2.0:1788619915.871256:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571445376 00000100:00100000:2.0:1788619915.871282:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571445376 00000100:00100000:2.0:1788619915.871661:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 949 00000100:00100000:2.0:1788619915.871672:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79008aed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2353:x1875503571445376:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619915.871691:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619915.871708:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe5368734a9d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bee6 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.871742:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079600/0x1cd7fe53687348dd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd2d expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619915.871776:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900d5ca80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589112704:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619915.871862:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc780 going to sleep for 11 seconds 00000100:00100000:2.0:1788619915.877310:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c4a4c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589112576:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619915.877380:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a72c5200/0x1cd7fe5368734a96 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1beed expref: 9 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.877406:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a72c5200/0x1cd7fe5368734a96 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1beed expref: 9 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.877420:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c400, rc 0) 00000100:00100000:2.0:1788619915.877477:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79008aea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571445504:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8011us (9179us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619915.877486:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 760 00000100:00100000:2.0:1788619915.884629:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900d5ca80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589112704:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619915.884659:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 729088 00010000:00010000:2.0:1788619915.884700:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe5368734a9d lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bee6 expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.884726:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619915.884729:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619915.884736:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe5368734a9d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bee6 expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.884750:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079000, rc 0) 00000100:00100000:2.0:1788619915.884839:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79008aed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2353:x1875503571445376:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 13169us (16638us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619915.884848:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 949 00000100:00100000:1.0:1788619915.891186:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619915.891228:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619915.891375:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571445760 00000100:00100000:0.0:1788619915.891406:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571445760 00000100:00100000:0.0:1788619915.891461:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 761 00000100:00100000:0.0:1788619915.891473:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79008ac000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571445760:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619915.891489:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619915.891507:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d412ceb8/0x1cd7fe5368734aa4 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1befb expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619915.891576:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5dc9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589112832:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619915.891664:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78eab85780 going to sleep for 11 seconds 00000100:00100000:2.0:1788619915.892166:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571445632 00000100:00100000:2.0:1788619915.892173:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571445632 00000100:00100000:2.0:1788619915.892215:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 950 00000100:00100000:2.0:1788619915.892223:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79008ae300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571445632:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619915.892234:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619915.892248:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368734aab lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bef4 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.892277:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079600/0x1cd7fe53687348dd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd2d expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619915.892302:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900d5ca80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589112960:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619915.892365:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc780 going to sleep for 11 seconds 00000100:00100000:2.0:1788619915.902004:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900d5ca80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589112960:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619915.902033:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 729088 00010000:00010000:2.0:1788619915.902081:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368734aab lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bef4 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.902105:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619915.902108:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619915.902114:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368734aab lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bef4 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.902129:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079400, rc 0) 00000100:00100000:2.0:1788619915.902189:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79008ae300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571445632:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 9966us (11024us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619915.902198:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 950 00000100:00100000:0.0:1788619915.904840:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5dc9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589112832:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619915.904885:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d412ceb8/0x1cd7fe5368734aa4 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1befb expref: 9 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619915.904920:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d412ceb8/0x1cd7fe5368734aa4 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1befb expref: 9 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619915.904935:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767ba00, rc 0) 00000100:00100000:0.0:1788619915.905234:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79008ac000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571445760:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 13761us (14002us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619915.905243:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 761 00000100:00100000:2.0:1788619915.910520:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619915.910570:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619915.910622:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571446016 00000100:00100000:0.0:1788619915.910639:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571446016 00000100:00100000:0.0:1788619915.910688:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 762 00000100:00100000:0.0:1788619915.910696:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571446016:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619915.910713:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619915.910751:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000072940acc/0x1cd7fe5368734ab2 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bf09 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619915.910820:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5dc9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589113088:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619915.910889:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78eab85780 going to sleep for 11 seconds 00000100:00100000:3.0:1788619915.911040:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571445888 00000100:00100000:3.0:1788619915.911044:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571445888 00000100:00100000:3.0:1788619915.911086:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 951 00000100:00100000:3.0:1788619915.911093:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2353:x1875503571445888:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619915.911105:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619915.911131:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe5368734ab9 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bf02 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619915.911160:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079600/0x1cd7fe53687348dd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd2d expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619915.911182:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c4a4e300 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589113216:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619915.911206:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619915.919719:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5dc9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589113088:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619915.919761:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000072940acc/0x1cd7fe5368734ab2 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf09 expref: 9 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.919815:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000072940acc/0x1cd7fe5368734ab2 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf09 expref: 9 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619915.919831:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767b000, rc 0) 00000100:00100000:1.0:1788619915.919897:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571446016:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 9200us (9325us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619915.919907:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 762 00000100:00100000:3.0:1788619915.921956:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c4a4e300 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589113216:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619915.921983:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 729088 00010000:00010000:3.0:1788619915.922005:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe5368734ab9 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf02 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619915.922024:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619915.922027:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619915.922033:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe5368734ab9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf02 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619915.922046:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d800, rc 0) 00000100:00100000:3.0:1788619915.922153:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2353:x1875503571445888:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 11059us (11650us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619915.922162:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 951 00000100:00100000:0.0:1788619915.928991:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619915.929203:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571446144 00000100:00100000:2.0:1788619915.929213:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571446144 00000100:00100000:2.0:1788619915.929280:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 952 00000100:00100000:2.0:1788619915.929290:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dc9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2355:x1875503571446144:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619915.929312:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619915.929489:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368734ac0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bf10 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.929541:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079600/0x1cd7fe53687348dd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd2d expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619915.929582:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d57b80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589113344:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619915.929666:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc780 going to sleep for 11 seconds 00000100:00100000:1.0:1788619915.933817:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619915.933905:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571446272 00000100:00100000:3.0:1788619915.933916:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571446272 00000100:00100000:3.0:1788619915.933991:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 763 00000100:00100000:3.0:1788619915.934001:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6f86a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571446272:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619915.934019:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619915.934051:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001fc792ba/0x1cd7fe5368734ac7 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bf17 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619915.934122:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c4a4e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589113472:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619915.934215:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619915.937672:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d57b80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589113344:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619915.937707:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 729088 00010000:00010000:2.0:1788619915.937727:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368734ac0 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf10 expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.937747:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619915.937751:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619915.937756:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368734ac0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf10 expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.937771:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079400, rc 0) 00000100:00100000:2.0:1788619915.937829:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dc9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2355:x1875503571446144:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8539us (8846us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619915.937837:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 952 00000100:00100000:3.0:1788619915.941765:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c4a4e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589113472:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619915.941820:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001fc792ba/0x1cd7fe5368734ac7 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf17 expref: 9 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619915.941850:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001fc792ba/0x1cd7fe5368734ac7 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf17 expref: 9 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619915.941867:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d800, rc 0) 00000100:00100000:3.0:1788619915.941928:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6f86a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571446272:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7927us (8130us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619915.941937:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 763 00000100:00100000:1.0:1788619915.951141:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619915.951238:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571446400 00000100:00100000:2.0:1788619915.951249:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571446400 00000100:00100000:2.0:1788619915.951313:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 953 00000100:00100000:2.0:1788619915.951366:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6f86300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:14908:x1875503571446400:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619915.951388:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619915.951410:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368734ace lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bf1e expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.951440:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079600/0x1cd7fe53687348dd lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd2d expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.951452:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368734ace lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf1e expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.951487:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079600/0x1cd7fe53687348dd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd2d expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.951527:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079600/0x1cd7fe53687348dd lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd2d expref: 9 pid: 5846 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619915.951543:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d57b80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589113600:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619915.951640:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc780 going to sleep for 11 seconds 00000100:00100000:2.0:1788619915.954055:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d57b80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589113600:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619915.954102:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368734ace lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf1e expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.954121:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619915.954124:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619915.954129:0:5846:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368734ace lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf1e expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619915.954139:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079400, rc 0) 00000100:00100000:2.0:1788619915.954245:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6f86300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571446400:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2922us (3114us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619915.954255:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 953 00000100:00100000:1.0:1788619915.957155:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619915.958113:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571446528 00000100:00100000:0.0:1788619915.958126:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571446528 00010000:00010000:0.0:1788619915.958173:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079600/0x1cd7fe53687348dd lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd2d expref: 10 pid: 5846 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619915.958218:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 318 00000100:00100000:0.0:1788619915.958227:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6f84000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571446528:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619915.958240:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619915.958264:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079600/0x1cd7fe53687348dd lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd2d expref: 10 pid: 5846 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619915.958277:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079600/0x1cd7fe53687348dd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd2d expref: 10 pid: 5846 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619915.958299:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079600/0x1cd7fe53687348dd lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bd2d expref: 10 pid: 5846 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619915.958317:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619915.958323:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368734ace lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf1e expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619915.958498:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368734ace lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf1e expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619915.958533:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368734ace lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf1e expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619915.958573:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368734ace lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf1e expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619915.958585:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5dcb800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589113728:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619915.958667:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78eab85780 going to sleep for 11 seconds 00000100:00100000:1.0:1788619915.972998:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5dcb800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589113728:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619915.973028:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619915.973097:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6f84000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571446528:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 14869us (15953us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619915.973109:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 318 00000100:00100000:3.0:1788619915.973828:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619915.974793:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571446656 00000100:00100000:0.0:1788619915.974812:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571446656 00002000:00100000:0.0:1788619915.974866:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c4a4e300 x1875503571446656/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/0 e 0 to 0 dl 1788619926 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619915.974877:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c4a4e300 with x1875503571446656 00000100:00100000:0.0:1788619915.974915:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 594 00000100:00100000:0.0:1788619915.974924:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571446656:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619916.039123:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571446656:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 64186us (65284us total) trans 0 rc 729088/729088 00000100:00100000:0.0:1788619916.039138:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 594 00002000:00100000:0.0:1788619916.039152:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c4a4e300 x1875503571446656/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:11/0 lens 488/440 e 0 to 0 dl 1788619926 ref 1 fl Complete:/600/0 rc 729088/729088 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619916.039165:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c4a4e300 with x1875503571446656 00000100:00100000:3.0:1788619916.045970:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619916.046150:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571446784 00000100:00100000:0.0:1788619916.046161:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571446784 00002000:00100000:0.0:1788619916.046239:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c4a4df80 x1875503571446784/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/0 e 0 to 0 dl 1788619927 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619916.046263:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c4a4df80 with x1875503571446784 00000100:00100000:0.0:1788619916.046303:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 595 00000100:00100000:0.0:1788619916.046315:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571446784:12345-192.168.201.5@tcp:4:fsx.0 00000100:00080000:1.0:1788619916.046536:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000004:00010000:0.0:1788619916.048859:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 729088 -> 729088 00000004:00010000:0.0:1788619916.048867:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619915 -> 1788619915 00000004:00010000:0.0:1788619916.048870:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619915 -> 1788619914 00000004:00010000:0.0:1788619916.048872:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619915 -> 1788619915 00000004:00010000:0.0:1788619916.048874:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1414 -> 646 00000100:00100000:0.0:1788619916.048936:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2353:x1875503571446784:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 2620us (2984us total) trans 4294967656 rc 0/0 00000100:00100000:0.0:1788619916.048946:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 595 00002000:00100000:0.0:1788619916.048964:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c4a4df80 x1875503571446784/t4294967656(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/448 e 0 to 0 dl 1788619927 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619916.048973:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c4a4df80 with x1875503571446784 00000100:00100000:2.0:1788619916.052916:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619916.053048:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571446912 00000100:00100000:3.0:1788619916.053058:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571446912 00000100:00100000:3.0:1788619916.053127:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 954 00000100:00100000:3.0:1788619916.053138:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d57b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2353:x1875503571446912:12345-192.168.201.5@tcp:16:fsx.0 00010000:00010000:1.0:1788619916.071399:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1788619916.093347:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967656 is committed 00000100:00100000:2.0:1788619916.095052:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d57b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571446912:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 41901us (42130us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619916.095077:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 954 00000100:00100000:2.0:1788619916.103269:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619916.106539:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571447040 00000100:00100000:3.0:1788619916.106558:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571447040 00000100:00100000:3.0:1788619916.106665:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 764 00000100:00100000:3.0:1788619916.106676:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2352:x1875503571447040:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619916.106712:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619916.106733:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001fc792ba/0x1cd7fe5368734ad5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bf25 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619916.106812:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c4a4c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589113856:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619916.106915:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaab40 going to sleep for 11 seconds 00000100:00080000:0.0:1788619916.110165:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1788619916.110442:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c4a4c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589113856:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619916.110497:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001fc792ba/0x1cd7fe5368734ad5 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf25 expref: 9 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.110720:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001fc792ba/0x1cd7fe5368734ad5 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf25 expref: 9 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.110744:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d800, rc 0) 00000100:00100000:2.0:1788619916.110815:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2352:x1875503571447040:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4141us (7557us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619916.110824:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 764 00000100:00100000:2.0:1788619916.116694:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619916.116756:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571447168 00000100:00100000:0.0:1788619916.116764:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571447168 00000100:00100000:0.0:1788619916.116826:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 765 00000100:00100000:0.0:1788619916.116836:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571447168:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619916.116854:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619916.116874:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000496a39a3/0x1cd7fe5368734adc lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bf2c expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619916.116948:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5dc8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589113984:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619916.117034:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78eab85e40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619916.119059:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5dc8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589113984:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619916.119109:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000496a39a3/0x1cd7fe5368734adc lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf2c expref: 9 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.119136:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000496a39a3/0x1cd7fe5368734adc lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf2c expref: 9 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.119151:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767be00, rc 0) 00000100:00100000:0.0:1788619916.119262:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571447168:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2427us (2940us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619916.119271:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 765 00000100:00100000:3.0:1788619916.123762:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619916.123941:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571447296 00000100:00100000:2.0:1788619916.123952:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571447296 00000100:00100000:2.0:1788619916.124029:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 955 00000100:00100000:2.0:1788619916.124040:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571447296:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619916.124059:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619916.124089:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779bc00/0x1cd7fe5368734ae3 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bf33 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.124119:0:9417:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368734ace lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf1e expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.124129:0:9417:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779bc00/0x1cd7fe5368734ae3 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf33 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.124160:0:9417:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368734ace lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf1e expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.124186:0:9417:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368734ace lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf1e expref: 12 pid: 5846 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619916.124200:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5ff7100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589114112:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619916.124275:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc300 going to sleep for 11 seconds 00000100:00080000:0.0:1788619916.124347:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1788619916.127817:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5ff7100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589114112:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619916.127855:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779bc00/0x1cd7fe5368734ae3 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf33 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.127873:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619916.127876:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619916.127881:0:9417:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779bc00/0x1cd7fe5368734ae3 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf33 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.127892:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c779bc00, rc 0) 00000100:00100000:0.0:1788619916.127974:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:14908:x1875503571447296:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3935us (4221us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619916.127982:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 955 00000100:00100000:3.0:1788619916.132684:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619916.132855:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571447424 00000100:00100000:1.0:1788619916.132867:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571447424 00010000:00010000:1.0:1788619916.132922:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368734ace lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf1e expref: 13 pid: 5846 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619916.132963:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 319 00000100:00100000:1.0:1788619916.132972:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571447424:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619916.133007:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619916.133036:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368734ace lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf1e expref: 13 pid: 5846 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.133050:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368734ace lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf1e expref: 13 pid: 5846 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.133089:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368734ace lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf1e expref: 13 pid: 5846 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.133109:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619916.133115:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779bc00/0x1cd7fe5368734ae3 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf33 expref: 9 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.133129:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779bc00/0x1cd7fe5368734ae3 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf33 expref: 9 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.133138:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779bc00/0x1cd7fe5368734ae3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf33 expref: 9 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.133181:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779bc00/0x1cd7fe5368734ae3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf33 expref: 9 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619916.133195:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6f84a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589114240:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619916.133281:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619916.136079:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6f84a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589114240:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619916.136201:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619916.136261:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2355:x1875503571447424:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3288us (3591us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619916.136278:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 319 00000100:00100000:3.0:1788619916.136413:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619916.136529:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571447552 00000100:00100000:0.0:1788619916.136537:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571447552 00000100:00100000:0.0:1788619916.136595:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 596 00000100:00100000:0.0:1788619916.136604:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571447552:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:0.0:1788619916.136826:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 729088 -> 729088 00000004:00010000:0.0:1788619916.136831:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619915 -> 1788619915 00000004:00010000:0.0:1788619916.136833:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619914 -> 1788619914 00000004:00010000:0.0:1788619916.136839:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619915 -> 1788619915 00000004:00010000:0.0:1788619916.136841:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 646 -> 646 00000100:00100000:0.0:1788619916.136891:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571447552:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 288us (477us total) trans 4294967657 rc 0/0 00000100:00100000:0.0:1788619916.136898:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 596 00000100:00100000:3.0:1788619916.141216:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619916.142537:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571447680 00000100:00100000:1.0:1788619916.142551:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571447680 00000100:00100000:1.0:1788619916.142649:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 406 00000100:00100000:1.0:1788619916.142660:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2352:x1875503571447680:12345-192.168.201.5@tcp:22:fsx.0 00000100:00080000:0.0:1788619916.142686:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:1.0:1788619916.142921:0:6701:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 2074055 -> 2074055 00002000:00010000:1.0:1788619916.142925:0:6701:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619915 -> 1788619914 00002000:00010000:1.0:1788619916.142927:0:6701:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619915 -> 1788619914 00002000:00010000:1.0:1788619916.142930:0:6701:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619915 -> 1788619914 00002000:00010000:1.0:1788619916.142932:0:6701:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 519 -> 519 00000100:00100000:1.0:1788619916.143003:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+8:2352:x1875503571447680:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 345us (1796us total) trans 4294967416 rc 0/0 00000100:00100000:1.0:1788619916.143012:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 406 00000100:00100000:3.0:1788619916.148663:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619916.148730:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571447808 00000100:00100000:0.0:1788619916.148741:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571447808 00002000:00100000:0.0:1788619916.148790:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c4a4f480 x1875503571447808/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:12/0 lens 488/0 e 0 to 0 dl 1788619927 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619916.148814:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c4a4f480 with x1875503571447808 00000100:00100000:0.0:1788619916.148900:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 597 00000100:00100000:0.0:1788619916.148911:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571447808:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619916.149219:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571447808:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 308us (563us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619916.149228:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 597 00002000:00100000:0.0:1788619916.149248:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c4a4f480 x1875503571447808/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:12/0 lens 488/4536 e 0 to 0 dl 1788619927 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619916.149257:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c4a4f480 with x1875503571447808 00000100:00100000:3.0:1788619916.152747:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619916.153823:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571447936 00000100:00100000:0.0:1788619916.153837:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571447936 00002000:00100000:1.0:1788619916.155044:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b35f80 x1875503571447936/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:12/0 lens 488/0 e 0 to 0 dl 1788619927 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619916.155060:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b35f80 with x1875503571447936 00000100:00100000:1.0:1788619916.155122:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 598 00000100:00100000:1.0:1788619916.155134:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b35f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571447936:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619916.155493:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b35f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571447936:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 360us (2755us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619916.155511:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 598 00002000:00100000:1.0:1788619916.155520:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b35f80 x1875503571447936/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:12/0 lens 488/4536 e 0 to 0 dl 1788619927 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619916.155544:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b35f80 with x1875503571447936 00000100:00100000:2.0:1788619916.161948:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619916.162044:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571448064 00000100:00100000:1.0:1788619916.162066:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571448064 00002000:00100000:1.0:1788619916.162120:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5ff6300 x1875503571448064/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:12/0 lens 488/0 e 0 to 0 dl 1788619927 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619916.162131:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5ff6300 with x1875503571448064 00000100:00100000:1.0:1788619916.162178:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 599 00000100:00100000:1.0:1788619916.162188:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571448064:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619916.162527:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571448064:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 339us (589us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619916.162535:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 599 00002000:00100000:1.0:1788619916.162543:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5ff6300 x1875503571448064/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:12/0 lens 488/4536 e 0 to 0 dl 1788619927 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619916.162551:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5ff6300 with x1875503571448064 00000100:00100000:2.0:1788619916.167178:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619916.167689:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571448192 00000100:00100000:3.0:1788619916.167701:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571448192 00002000:00100000:3.0:1788619916.167750:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5ff5500 x1875503571448192/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:12/0 lens 12776/0 e 0 to 0 dl 1788619927 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619916.167761:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5ff5500 with x1875503571448192 00000100:00100000:3.0:1788619916.167810:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 600 00000100:00100000:3.0:1788619916.167821:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571448192:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:3.0:1788619916.168255:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 729088 -> 729088 00000004:00010000:3.0:1788619916.168259:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619915 -> 1788619916 00000004:00010000:3.0:1788619916.168261:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619914 -> 1788619914 00000004:00010000:3.0:1788619916.168263:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619915 -> 1788619916 00000004:00010000:3.0:1788619916.168265:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 646 -> 646 00000100:00100000:3.0:1788619916.168435:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+13:2353:x1875503571448192:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 615us (1271us total) trans 4294967658 rc 0/0 00000100:00100000:3.0:1788619916.168452:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 600 00002000:00100000:3.0:1788619916.168460:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5ff5500 x1875503571448192/t4294967658(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:12/0 lens 12776/448 e 0 to 0 dl 1788619927 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619916.168468:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5ff5500 with x1875503571448192 00000100:00100000:2.0:1788619916.178142:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619916.178315:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571448320 00000100:00100000:0.0:1788619916.178354:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571448320 00000100:00100000:0.0:1788619916.178494:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 956 00000100:00100000:0.0:1788619916.179989:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+13:2353:x1875503571448320:12345-192.168.201.5@tcp:16:fsx.0 00000001:00080000:3.0:1788619916.217030:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967657 is committed 00000001:00080000:3.0:1788619916.217053:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967658 is committed 00000100:00100000:2.0:1788619916.217303:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571448320:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 38793us (39168us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619916.217317:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 956 00000100:00100000:2.0:1788619916.222477:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619916.222720:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571448576 00000100:00100000:1.0:1788619916.222730:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571448576 00000100:00100000:1.0:1788619916.222855:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 957 00000100:00100000:1.0:1788619916.222868:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+10:14908:x1875503571448576:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:1.0:1788619916.222938:0:11397:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619916.222964:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767b600/0x1cd7fe5368734890 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bce0 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.222984:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619916.222990:0:11397:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788619916.223273:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753000/0x1cd7fe5368734aea lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.223290:0:11397:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a000/0x1cd7fe536873487b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bcd2 expref: 11 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.223299:0:11397:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753000/0x1cd7fe5368734aea lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.223379:0:11397:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a000/0x1cd7fe536873487b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bcd2 expref: 11 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.223393:0:11397:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a000/0x1cd7fe536873487b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bcd2 expref: 11 pid: 5844 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619916.223409:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6f84a80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589114368:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619916.223494:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c300 going to sleep for 11 seconds 00000100:00100000:3.0:1788619916.227097:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619916.227221:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571448704 00000100:00100000:2.0:1788619916.227230:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571448704 00010000:00010000:2.0:1788619916.227267:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a000/0x1cd7fe536873487b lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bcd2 expref: 12 pid: 5844 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619916.227311:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 320 00000100:00100000:2.0:1788619916.227320:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c855a050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571448704:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619916.227356:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619916.227362:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a000/0x1cd7fe536873487b lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bcd2 expref: 12 pid: 5844 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.227375:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a000/0x1cd7fe536873487b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bcd2 expref: 12 pid: 5844 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.227416:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619916.227422:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753000/0x1cd7fe5368734aea lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.227433:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753000/0x1cd7fe5368734aea lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.227441:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753000/0x1cd7fe5368734aea lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.227458:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619916.227516:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c855a050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571448704:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 197us (428us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619916.227524:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 320 00010000:00010000:1.0:1788619916.228045:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a000/0x1cd7fe536873487b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bcd2 expref: 11 pid: 5844 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619916.228073:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6f84a80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589114368:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619916.228092:0:11397:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753000/0x1cd7fe5368734aea lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.228101:0:11397:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753000/0x1cd7fe5368734aea lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.228107:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753000/0x1cd7fe5368734aea lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788619916.228246:0:11397:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967659 00000004:00010000:1.0:1788619916.228296:0:11397:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 729088 -> 729088 00000004:00010000:1.0:1788619916.228300:0:11397:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619916 -> 1788619916 00000004:00010000:1.0:1788619916.228302:0:11397:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619914 -> 1788619914 00000004:00010000:1.0:1788619916.228305:0:11397:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619916 -> 1788619916 00000004:00010000:1.0:1788619916.228307:0:11397:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 646 -> 902 00000004:00010000:1.0:1788619916.228314:0:11397:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000000b8b4380 reply state 00000000a0c1c0d4 transno 4294967660 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753000/0x1cd7fe5368734aea lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619916.228354:0:11397:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788619916.228459:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+12:14908:x1875503571448576:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 5592us (5991us total) trans 4294967660 rc 0/0 00000100:00100000:1.0:1788619916.228467:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 957 00000020:00080000:1.0:1788619916.228471:0:11397:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:2.0:1788619916.231449:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753000/0x1cd7fe5368734aea lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.231466:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753000/0x1cd7fe5368734aea lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.231474:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753000/0x1cd7fe5368734aea lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.231480:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c5753000 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:2.0:1788619916.231486:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753000/0x1cd7fe5368734aea lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.231494:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753000/0x1cd7fe5368734aea lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.231541:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619916.231544:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788619916.231550:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753000/0x1cd7fe5368734aea lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.231556:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753000/0x1cd7fe5368734aea lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619916.233203:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619916.233751:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571448832 00000100:00100000:0.0:1788619916.233761:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571448832 00002000:00010000:0.0:1788619916.233813:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1657461->18446744073709551615) 00002000:00100000:0.0:1788619916.233818:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6b35500 with x1875503571448832 ext(1657461->18446744073709551615) 00002000:00010000:0.0:1788619916.233829:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6b35500. 00000100:00100000:0.0:1788619916.233864:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 407 00000100:00100000:0.0:1788619916.233874:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b35500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+8:2355:x1875503571448832:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:0.0:1788619916.234176:0:6703:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 2074055 -> 1657461 00002000:00010000:0.0:1788619916.234180:0:6703:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619914 -> 1788619916 00002000:00010000:0.0:1788619916.234182:0:6703:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619914 -> 1788619914 00002000:00010000:0.0:1788619916.234184:0:6703:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619914 -> 1788619916 00002000:00010000:0.0:1788619916.234186:0:6703:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 519 -> 519 00000100:00100000:0.0:1788619916.234252:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b35500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571448832:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 379us (1058us total) trans 4294967417 rc 0/0 00000100:00100000:0.0:1788619916.234260:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 407 00002000:00010000:0.0:1788619916.234263:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1657461->18446744073709551615) 00002000:00100000:0.0:1788619916.234266:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6b35500 with x1875503571448832 ext(1657461->18446744073709551615) 00002000:00010000:0.0:1788619916.234284:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6b35500. 00000100:00100000:3.0:1788619916.238682:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619916.238926:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571448960 00000100:00100000:0.0:1788619916.238936:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571448960 00000100:00100000:0.0:1788619916.238993:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 958 00000100:00100000:0.0:1788619916.239002:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b36d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+11:14908:x1875503571448960:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619916.239021:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619916.239041:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767be00/0x1cd7fe5368734af1 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bf3a expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.239138:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767b400/0x1cd7fe5368734af8 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.239154:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767b400/0x1cd7fe5368734af8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.239166:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767b400/0x1cd7fe5368734af8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619916.239193:0:9417:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1657461 00000004:00200000:0.0:1788619916.239197:0:9417:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619916.239213:0:9417:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767b400/0x1cd7fe5368734af8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.239240:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767be00/0x1cd7fe5368734af1 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf3a expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.239258:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767b400/0x1cd7fe5368734af8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf3a expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.239267:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619916.239285:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619916.239289:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767b400, rc 0) 00000100:00100000:0.0:1788619916.239390:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b36d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+12:14908:x1875503571448960:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 388us (716us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619916.239397:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 958 00000100:00100000:3.0:1788619916.242404:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619916.242880:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571449088 00000100:00100000:2.0:1788619916.242889:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571449088 00000100:00100000:2.0:1788619916.242958:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 959 00000100:00100000:2.0:1788619916.242968:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b35c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2353:x1875503571449088:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619916.242988:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619916.243010:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779ae00/0x1cd7fe5368734aff lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bf41 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.243048:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779bc00/0x1cd7fe5368734ae3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf33 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619916.243100:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5ff4380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589114496:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619916.243189:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccd80 going to sleep for 11 seconds 00000100:00100000:3.0:1788619916.245559:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619916.245723:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571449216 00000100:00100000:2.0:1788619916.245733:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571449216 00000100:00100000:2.0:1788619916.245809:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 766 00000100:00100000:2.0:1788619916.245819:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b34e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571449216:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619916.245836:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619916.245853:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005b564562/0x1cd7fe5368734b06 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bf48 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619916.245944:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5ff4000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589114624:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619916.246039:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc0c0 going to sleep for 11 seconds 00000004:00010000:2.0:1788619916.250095:0:5846:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619914 -> 1788619915 00000100:00100000:2.0:1788619916.250132:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5ff4380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589114496:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619916.250165:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 729088 00010000:00010000:2.0:1788619916.250192:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779ae00/0x1cd7fe5368734aff lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf41 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.250214:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619916.250217:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619916.250223:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779ae00/0x1cd7fe5368734aff lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf41 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.250240:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c779ae00, rc 0) 00000100:00100000:2.0:1788619916.250316:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b35c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2353:x1875503571449088:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7349us (7922us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619916.250384:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 959 00002000:00010000:1.0:1788619916.252403:0:7654:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619914 -> 1788619915 00000100:00100000:0.0:1788619916.252577:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5ff4000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589114624:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619916.252615:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005b564562/0x1cd7fe5368734b06 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf48 expref: 9 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.252700:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005b564562/0x1cd7fe5368734b06 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf48 expref: 9 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.252716:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c779ac00, rc 0) 00000100:00100000:0.0:1788619916.252836:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b34e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571449216:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7016us (7283us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619916.252845:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 766 00000100:00100000:3.0:1788619916.257125:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619916.257170:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619916.257302:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571449344 00000100:00100000:1.0:1788619916.257312:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571449344 00000100:00100000:1.0:1788619916.257397:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 960 00000100:00100000:1.0:1788619916.257409:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b37100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2352:x1875503571449344:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619916.257428:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619916.257448:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734b0d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bf4f expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.257483:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779bc00/0x1cd7fe5368734ae3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf33 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619916.257535:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6f84a80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589114752:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619916.257642:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c300 going to sleep for 11 seconds 00000100:00100000:0.0:1788619916.257723:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571449472 00000100:00100000:0.0:1788619916.257728:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571449472 00000100:00100000:0.0:1788619916.257761:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 767 00000100:00100000:0.0:1788619916.257769:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b36300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2352:x1875503571449472:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619916.257778:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619916.257789:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f4734b2f/0x1cd7fe5368734b14 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bf56 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619916.257836:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5dc8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589114880:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619916.257878:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c3579000 going to sleep for 11 seconds 00000100:00100000:0.0:1788619916.260879:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5dc8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589114880:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619916.260919:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f4734b2f/0x1cd7fe5368734b14 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf56 expref: 9 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.260944:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f4734b2f/0x1cd7fe5368734b14 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf56 expref: 9 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.260957:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767a600, rc 0) 00000100:00100000:0.0:1788619916.261014:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b36300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2352:x1875503571449472:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3245us (3842us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619916.261023:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 767 00000100:00100000:1.0:1788619916.261542:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6f84a80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589114752:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619916.261557:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 729088 00010000:00010000:1.0:1788619916.261570:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734b0d lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf4f expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.261584:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619916.261587:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619916.261591:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734b0d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf4f expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.261602:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752e00, rc 0) 00000100:00100000:1.0:1788619916.261647:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b37100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2352:x1875503571449344:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4241us (4531us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619916.261653:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 960 00000100:00100000:3.0:1788619916.265725:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619916.265811:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571449600 00000100:00100000:1.0:1788619916.265821:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571449600 00000100:00100000:1.0:1788619916.265881:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 961 00000100:00100000:1.0:1788619916.265891:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b37800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571449600:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619916.265927:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619916.265948:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753200/0x1cd7fe5368734b1b lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bf5d expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.266036:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752400/0x1cd7fe5368734b22 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.266053:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752400/0x1cd7fe5368734b22 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.266091:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752400/0x1cd7fe5368734b22 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788619916.266119:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1657461 00000004:00200000:1.0:1788619916.266123:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788619916.266140:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752400/0x1cd7fe5368734b22 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.266167:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753200/0x1cd7fe5368734b1b lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf5d expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.266184:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752400/0x1cd7fe5368734b22 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf5d expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.266193:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619916.266196:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788619916.266199:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752400, rc 0) 00000100:00100000:1.0:1788619916.266254:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b37800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:14908:x1875503571449600:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 364us (539us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619916.266260:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 961 00000100:00100000:3.0:1788619916.270268:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619916.270456:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571449728 00000100:00100000:2.0:1788619916.270474:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571449728 00000100:00100000:2.0:1788619916.270535:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 962 00000100:00100000:2.0:1788619916.270544:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b35180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:14908:x1875503571449728:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619916.270564:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619916.270584:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779ae00/0x1cd7fe5368734b29 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bf64 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.270643:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779bc00/0x1cd7fe5368734ae3 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf33 expref: 11 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.270654:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779ae00/0x1cd7fe5368734b29 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf64 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.270686:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779bc00/0x1cd7fe5368734ae3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf33 expref: 11 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.270699:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779bc00/0x1cd7fe5368734ae3 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf33 expref: 11 pid: 9417 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619916.270712:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5ff4380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589115008:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619916.270783:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccd80 going to sleep for 11 seconds 00000100:00100000:3.0:1788619916.274279:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619916.274472:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571449856 00000100:00100000:0.0:1788619916.274482:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571449856 00010000:00010000:0.0:1788619916.274526:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779bc00/0x1cd7fe5368734ae3 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf33 expref: 12 pid: 9417 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619916.274567:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 321 00000100:00100000:0.0:1788619916.274576:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b34a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571449856:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619916.274591:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619916.274614:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779bc00/0x1cd7fe5368734ae3 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf33 expref: 12 pid: 9417 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.274637:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779bc00/0x1cd7fe5368734ae3 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf33 expref: 12 pid: 9417 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.274680:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619916.274686:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779ae00/0x1cd7fe5368734b29 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf64 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.274699:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779ae00/0x1cd7fe5368734b29 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf64 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.274709:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779ae00/0x1cd7fe5368734b29 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf64 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.274746:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779ae00/0x1cd7fe5368734b29 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf64 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619916.274759:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5dc8000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589115136:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619916.274852:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c3579000 going to sleep for 11 seconds 00010000:00010000:2.0:1788619916.275809:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779bc00/0x1cd7fe5368734ae3 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf33 expref: 12 pid: 9417 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619916.275830:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5ff4380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589115008:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619916.275857:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779ae00/0x1cd7fe5368734b29 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf64 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.275870:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619916.275872:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619916.275875:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c779ae00, rc 0) 00000100:00100000:2.0:1788619916.275938:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b35180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571449728:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5395us (5677us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619916.275957:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 962 00000100:00100000:0.0:1788619916.279988:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5dc8000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589115136:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619916.280016:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619916.280093:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b34a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571449856:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 5507us (5813us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619916.280106:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 321 00000100:00100000:3.0:1788619916.284540:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619916.285021:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571449984 00000100:00100000:0.0:1788619916.285032:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571449984 00002000:00100000:0.0:1788619916.285096:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b37b80 x1875503571449984/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/0 e 0 to 0 dl 1788619927 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619916.285106:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b37b80 with x1875503571449984 00000100:00100000:0.0:1788619916.285162:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 601 00000100:00100000:0.0:1788619916.285172:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b37b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571449984:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619916.286038:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b37b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571449984:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 865us (1507us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619916.286048:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 601 00002000:00100000:0.0:1788619916.286055:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b37b80 x1875503571449984/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/4536 e 0 to 0 dl 1788619927 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619916.286187:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b37b80 with x1875503571449984 00000100:00100000:3.0:1788619916.290828:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619916.291251:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571450112 00000100:00100000:2.0:1788619916.291262:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571450112 00000100:00100000:2.0:1788619916.291318:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 768 00000100:00100000:2.0:1788619916.291371:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b34000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2352:x1875503571450112:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619916.291391:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619916.291409:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f76e475a/0x1cd7fe5368734b30 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bf6b expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619916.291475:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5ff4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589115264:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619916.291555:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccd80 going to sleep for 11 seconds 00000100:00100000:3.0:1788619916.294833:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5ff4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589115264:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619916.294873:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f76e475a/0x1cd7fe5368734b30 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf6b expref: 9 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619916.294899:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f76e475a/0x1cd7fe5368734b30 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf6b expref: 9 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619916.294927:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c779b000, rc 0) 00000100:00100000:3.0:1788619916.294986:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b34000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2352:x1875503571450112:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3658us (4165us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619916.294994:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 768 00000100:00100000:3.0:1788619916.298957:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619916.299051:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571450240 00000100:00100000:1.0:1788619916.299069:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571450240 00000100:00100000:1.0:1788619916.299127:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 769 00000100:00100000:1.0:1788619916.299137:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b36680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571450240:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619916.299155:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619916.299173:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001786a1ee/0x1cd7fe5368734b37 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bf72 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619916.299249:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6f84a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589115392:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619916.299366:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c300 going to sleep for 11 seconds 00000100:00100000:0.0:1788619916.305081:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6f84a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589115392:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619916.305125:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001786a1ee/0x1cd7fe5368734b37 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf72 expref: 9 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.305154:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001786a1ee/0x1cd7fe5368734b37 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf72 expref: 9 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.305167:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5753200, rc 0) 00000100:00100000:0.0:1788619916.305233:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b36680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571450240:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6097us (6285us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619916.305242:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 769 00000100:00100000:2.0:1788619916.311306:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619916.311711:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571450368 00000100:00100000:1.0:1788619916.311721:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571450368 00000100:00100000:1.0:1788619916.311784:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 963 00000100:00100000:1.0:1788619916.311804:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571450368:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619916.311827:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619916.311850:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734b3e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bf79 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.311889:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779ae00/0x1cd7fe5368734b29 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf64 expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619916.311938:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6f84a80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589115520:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619916.312017:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c900 going to sleep for 11 seconds 00000100:00100000:1.0:1788619916.314728:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6f84a80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589115520:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619916.314757:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 729088 00010000:00010000:1.0:1788619916.314776:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734b3e lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf79 expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.314807:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619916.314811:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619916.314816:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734b3e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf79 expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.314853:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752e00, rc 0) 00000100:00100000:1.0:1788619916.314929:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571450368:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3135us (3635us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619916.314937:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 963 00000100:00100000:2.0:1788619916.319130:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619916.319285:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571450496 00000100:00100000:0.0:1788619916.319295:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571450496 00000100:00100000:0.0:1788619916.319390:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 964 00000100:00100000:0.0:1788619916.319401:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571450496:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619916.319421:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619916.319456:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a600/0x1cd7fe5368734b45 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bf80 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.319493:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779ae00/0x1cd7fe5368734b29 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf64 expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619916.319767:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5dc8000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589115648:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619916.319864:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c3579000 going to sleep for 11 seconds 00000100:00100000:0.0:1788619916.323312:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5dc8000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589115648:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619916.323377:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 729088 00010000:00010000:0.0:1788619916.323396:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a600/0x1cd7fe5368734b45 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf80 expref: 12 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.323418:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619916.323421:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619916.323427:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a600/0x1cd7fe5368734b45 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf80 expref: 12 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.323442:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767a600, rc 0) 00000100:00100000:0.0:1788619916.324187:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571450496:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4782us (5060us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619916.324197:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 964 00000100:00100000:3.0:1788619916.327522:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619916.327695:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571450752 00000100:00100000:2.0:1788619916.327707:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571450752 00000100:00100000:2.0:1788619916.327773:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 965 00000100:00100000:2.0:1788619916.327785:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b34700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571450752:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:2.0:1788619916.327851:0:5846:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619916.327878:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752400/0x1cd7fe5368734b22 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf5d expref: 11 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.327897:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619916.327917:0:5846:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788619916.328138:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779a800/0x1cd7fe5368734b4c lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.328170:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767b400/0x1cd7fe5368734af8 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf3a expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.328178:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779a800/0x1cd7fe5368734b4c lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.328212:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767b400/0x1cd7fe5368734af8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf3a expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.328225:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767b400/0x1cd7fe5368734af8 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf3a expref: 12 pid: 9417 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619916.328240:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5ff7100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589115776:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619916.328378:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccc00 going to sleep for 11 seconds 00000100:00100000:0.0:1788619916.335013:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5ff7100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589115776:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619916.335047:0:5846:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779a800/0x1cd7fe5368734b4c lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619916.336864:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619916.337058:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571450880 00000100:00100000:2.0:1788619916.337106:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571450880 00010000:00010000:2.0:1788619916.337140:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767b400/0x1cd7fe5368734af8 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf3a expref: 13 pid: 9417 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619916.337210:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 322 00000100:00100000:2.0:1788619916.337218:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b37480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571450880:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619916.337230:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619916.337235:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 1s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767b400/0x1cd7fe5368734af8 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf3a expref: 13 pid: 9417 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.337245:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767b400/0x1cd7fe5368734af8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf3a expref: 13 pid: 9417 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.337277:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767b400/0x1cd7fe5368734af8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf3a expref: 13 pid: 9417 timeout: 508 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.337289:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619916.337293:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779a800/0x1cd7fe5368734b4c lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.337300:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779a800/0x1cd7fe5368734b4c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.337306:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779a800/0x1cd7fe5368734b4c lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.337366:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619916.337405:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b37480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571450880:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 189us (551us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619916.337410:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 322 00010000:00010000:1.0:1788619916.337462:0:5846:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779a800/0x1cd7fe5368734b4c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.337470:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779a800/0x1cd7fe5368734b4c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788619916.337591:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967661 00000004:00010000:1.0:1788619916.337812:0:5846:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 729088 -> 729088 00000004:00010000:1.0:1788619916.337820:0:5846:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619916 -> 1788619916 00000004:00010000:1.0:1788619916.337823:0:5846:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619915 -> 1788619914 00000004:00010000:1.0:1788619916.337825:0:5846:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619916 -> 1788619916 00000004:00010000:1.0:1788619916.337828:0:5846:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 902 -> 902 00000004:00010000:1.0:1788619916.337836:0:5846:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000af47cebc reply state 000000000a6adf48 transno 4294967662 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779a800/0x1cd7fe5368734b4c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619916.337845:0:5846:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788619916.338740:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b34700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+13:14908:x1875503571450752:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 10953us (11234us total) trans 4294967662 rc 0/0 00000100:00100000:1.0:1788619916.338747:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 965 00000020:00080000:1.0:1788619916.338751:0:5846:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1788619916.340885:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00010000:00010000:0.0:1788619916.341125:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779a800/0x1cd7fe5368734b4c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.341150:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779a800/0x1cd7fe5368734b4c lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.341157:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779a800/0x1cd7fe5368734b4c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.341163:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c779a800 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788619916.341169:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779a800/0x1cd7fe5368734b4c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.341178:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779a800/0x1cd7fe5368734b4c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.341194:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619916.341197:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788619916.341203:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779a800/0x1cd7fe5368734b4c lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.341209:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779a800/0x1cd7fe5368734b4c lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619916.342014:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571451008 00000100:00100000:1.0:1788619916.342028:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571451008 00000100:00100000:1.0:1788619916.342096:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 966 00000100:00100000:1.0:1788619916.342118:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6859f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+12:14908:x1875503571451008:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619916.342143:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619916.342162:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734b53 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bf87 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.342193:0:11397:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779ae00/0x1cd7fe5368734b29 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf64 expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.342204:0:11397:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734b53 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf87 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.342237:0:11397:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779ae00/0x1cd7fe5368734b29 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf64 expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.342251:0:11397:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779ae00/0x1cd7fe5368734b29 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf64 expref: 11 pid: 5846 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619916.342265:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6f84a80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589115904:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619916.342376:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c240 going to sleep for 11 seconds 00000100:00100000:1.0:1788619916.347233:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6f84a80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589115904:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619916.347270:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734b53 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf87 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.347289:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619916.347291:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619916.347296:0:11397:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734b53 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf87 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.347405:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752e00, rc 0) 00000100:00100000:1.0:1788619916.347682:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6859f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+13:14908:x1875503571451008:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5565us (6804us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619916.347691:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 966 00000100:00100000:0.0:1788619916.349658:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619916.349749:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571451136 00000100:00100000:1.0:1788619916.349760:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571451136 00002000:00100000:1.0:1788619916.350018:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5dc8000 x1875503571451136/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/0 e 0 to 0 dl 1788619927 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619916.350029:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5dc8000 with x1875503571451136 00010000:00010000:1.0:1788619916.350040:0:5858:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779ae00/0x1cd7fe5368734b29 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf64 expref: 12 pid: 5846 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.350052:0:5858:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779ae00/0x1cd7fe5368734b29 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf64 expref: 12 pid: 5846 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788619916.350059:0:5858:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e78c5dc8000 00000100:00100000:1.0:1788619916.350123:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 22 00000100:00100000:1.0:1788619916.350135:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dc8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2355:x1875503571451136:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:1.0:1788619916.360806:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 729088 -> 729088 00000004:00010000:1.0:1788619916.360815:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619916 -> 1788619916 00000004:00010000:1.0:1788619916.360817:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619914 -> 1788619914 00000004:00010000:1.0:1788619916.360819:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619916 -> 1788619916 00000004:00010000:1.0:1788619916.360821:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 902 -> 902 00000100:00100000:1.0:1788619916.360962:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dc8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571451136:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 10825us (11308us total) trans 4294967663 rc 0/0 00000100:00100000:1.0:1788619916.360984:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 22 00002000:00100000:1.0:1788619916.360997:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5dc8000 x1875503571451136/t4294967663(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/448 e 0 to 0 dl 1788619927 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619916.361007:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5dc8000 with x1875503571451136 00010000:00010000:1.0:1788619916.361017:0:5858:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779ae00/0x1cd7fe5368734b29 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf64 expref: 14 pid: 5846 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.361036:0:5858:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779ae00/0x1cd7fe5368734b29 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf64 expref: 14 pid: 5846 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788619916.361044:0:5858:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e78c5dc8000 00000100:00100000:0.0:1788619916.369004:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619916.369169:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571451264 00000100:00100000:1.0:1788619916.369179:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571451264 00010000:00010000:1.0:1788619916.369223:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779ae00/0x1cd7fe5368734b29 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf64 expref: 14 pid: 5846 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619916.369267:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 323 00000100:00100000:1.0:1788619916.369278:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dc8a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571451264:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619916.369293:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619916.369319:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779ae00/0x1cd7fe5368734b29 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf64 expref: 14 pid: 5846 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.369371:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779ae00/0x1cd7fe5368734b29 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf64 expref: 14 pid: 5846 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.369393:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779ae00/0x1cd7fe5368734b29 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf64 expref: 14 pid: 5846 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.369425:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619916.369432:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734b53 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf87 expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.369443:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734b53 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf87 expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.369452:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734b53 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf87 expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.369491:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734b53 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf87 expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619916.369518:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6f84a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589116032:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619916.369594:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c240 going to sleep for 11 seconds 00000100:00100000:0.0:1788619916.374949:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619916.375076:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6f84a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589116032:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619916.375099:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619916.375156:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dc8a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571451264:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 5881us (6162us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619916.375168:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 323 00000100:00100000:3.0:1788619916.378709:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571451392 00000100:00100000:3.0:1788619916.378727:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571451392 00002000:00010000:3.0:1788619916.378774:0:6701:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (2088103->18446744073709551615) 00002000:00100000:3.0:1788619916.378780:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5dcbb80 with x1875503571451392 ext(2088103->18446744073709551615) 00002000:00010000:3.0:1788619916.379050:0:6701:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5dcbb80. 00000100:00100000:3.0:1788619916.379158:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 408 00000100:00100000:3.0:1788619916.379169:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dcbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571451392:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:3.0:1788619916.381305:0:6701:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1657461 -> 2088103 00002000:00010000:3.0:1788619916.381315:0:6701:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619916 -> 1788619916 00002000:00010000:3.0:1788619916.381318:0:6701:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619915 -> 1788619914 00002000:00010000:3.0:1788619916.381320:0:6701:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619916 -> 1788619916 00002000:00010000:3.0:1788619916.381336:0:6701:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 519 -> 519 00000100:00100000:3.0:1788619916.381421:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dcbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571451392:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 2249us (6476us total) trans 4294967418 rc 0/0 00000100:00100000:3.0:1788619916.381431:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 408 00002000:00010000:3.0:1788619916.381437:0:6701:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (2088103->18446744073709551615) 00002000:00100000:3.0:1788619916.381452:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5dcbb80 with x1875503571451392 ext(2088103->18446744073709551615) 00002000:00010000:3.0:1788619916.381460:0:6701:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5dcbb80. 00000100:00100000:0.0:1788619916.384665:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619916.384766:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571451520 00000100:00100000:1.0:1788619916.384775:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571451520 00000100:00100000:1.0:1788619916.384848:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 967 00000100:00100000:1.0:1788619916.384859:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dcaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571451520:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619916.384882:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619916.384916:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752400/0x1cd7fe5368734b5a lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bf8e expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.385000:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753200/0x1cd7fe5368734b61 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.385015:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753200/0x1cd7fe5368734b61 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.385027:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753200/0x1cd7fe5368734b61 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788619916.385054:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 2088103 00000004:00200000:1.0:1788619916.385058:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788619916.385076:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753200/0x1cd7fe5368734b61 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.385118:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752400/0x1cd7fe5368734b5a lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf8e expref: 15 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.385137:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753200/0x1cd7fe5368734b61 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf8e expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.385146:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619916.385149:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788619916.385153:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5753200, rc 0) 00000100:00100000:1.0:1788619916.385218:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dcaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571451520:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 361us (566us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619916.385225:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 967 00000100:00100000:2.0:1788619916.404900:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619916.404948:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619916.405048:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571451648 00000100:00100000:1.0:1788619916.405084:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571451648 00000100:00100000:1.0:1788619916.405144:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 968 00000100:00100000:1.0:1788619916.405156:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c0a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571451648:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619916.405178:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619916.405200:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752400/0x1cd7fe5368734b68 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bf95 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.405238:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734b53 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf87 expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619916.405280:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6f84a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589116160:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619916.405395:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c240 going to sleep for 11 seconds 00000100:00100000:0.0:1788619916.405694:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571451776 00000100:00100000:0.0:1788619916.405699:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571451776 00000100:00100000:0.0:1788619916.405742:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 770 00000100:00100000:0.0:1788619916.405750:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571451776:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619916.405758:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619916.405776:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000496a39a3/0x1cd7fe5368734b6f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bf9c expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619916.405821:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5dc9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589116288:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619916.405847:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c3579000 going to sleep for 11 seconds 00000004:00010000:1.0:1788619916.415949:0:5846:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619914 -> 1788619915 00002000:00010000:0.0:1788619916.415962:0:7654:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619914 -> 1788619915 00000100:00100000:1.0:1788619916.415985:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6f84a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589116160:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619916.415985:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5dc9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589116288:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619916.416007:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 729088 00010000:00010000:1.0:1788619916.416020:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752400/0x1cd7fe5368734b68 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf95 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.416020:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000496a39a3/0x1cd7fe5368734b6f lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf9c expref: 9 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.416041:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619916.416043:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000496a39a3/0x1cd7fe5368734b6f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf9c expref: 9 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.416044:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619916.416048:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752400/0x1cd7fe5368734b68 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf95 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.416056:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767be00, rc 0) 00010000:00010000:1.0:1788619916.416057:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752400, rc 0) 00000100:00100000:1.0:1788619916.416112:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c0a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571451648:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 10958us (11163us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619916.416121:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 968 00000100:00100000:0.0:1788619916.417350:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571451776:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 11593us (12452us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619916.417750:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 770 00000100:00100000:3.0:1788619916.420713:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619916.420758:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619916.420814:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571452032 00000100:00100000:0.0:1788619916.420825:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571452032 00000100:00100000:1.0:1788619916.420878:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571451904 00000100:00100000:1.0:1788619916.420883:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571451904 00000100:00100000:1.0:1788619916.420935:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 969 00000100:00100000:1.0:1788619916.420945:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c685b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571451904:12345-192.168.201.5@tcp:101:fsx.0 00000100:00100000:0.0:1788619916.420953:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 771 00000100:00100000:0.0:1788619916.420965:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6858380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571452032:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619916.420966:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619916.420979:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619916.420990:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753000/0x1cd7fe5368734b76 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bfa3 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.420992:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007f892434/0x1cd7fe5368734b7d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bfaa expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.421043:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734b53 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf87 expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619916.421072:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5dc9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589116416:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619916.421089:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6f84a80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589116544:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619916.421190:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c3579000 going to sleep for 11 seconds 00000100:00100000:1.0:1788619916.430291:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c240 going to sleep for 11 seconds 00000100:00100000:0.0:1788619916.436016:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5dc9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589116416:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619916.436064:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007f892434/0x1cd7fe5368734b7d lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bfaa expref: 9 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.436094:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f892434/0x1cd7fe5368734b7d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bfaa expref: 9 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.436110:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767a000, rc 0) 00000100:00100000:0.0:1788619916.436193:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6858380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571452032:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 15229us (15432us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619916.436202:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 771 00000100:00100000:1.0:1788619916.438811:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6f84a80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589116544:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619916.438836:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 729088 00010000:00010000:1.0:1788619916.438854:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753000/0x1cd7fe5368734b76 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bfa3 expref: 15 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.438875:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619916.438878:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619916.438883:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753000/0x1cd7fe5368734b76 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bfa3 expref: 15 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.438896:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5753000, rc 0) 00000100:00100000:1.0:1788619916.439213:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c685b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571451904:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 18269us (18508us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619916.439222:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 969 00000100:00100000:2.0:1788619916.449777:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619916.453361:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571452160 00000100:00100000:1.0:1788619916.453380:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571452160 00000100:00100000:0.0:1788619916.453560:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 602 00000100:00100000:0.0:1788619916.453572:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c09880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+13:2355:x1875503571452160:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:0.0:1788619916.453875:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 729088 -> 729088 00000004:00010000:0.0:1788619916.453880:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619916 -> 1788619916 00000004:00010000:0.0:1788619916.453883:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619915 -> 1788619914 00000004:00010000:0.0:1788619916.453885:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619916 -> 1788619916 00000004:00010000:0.0:1788619916.453888:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 902 -> 902 00000100:00100000:0.0:1788619916.453967:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c09880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+14:2355:x1875503571452160:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 396us (4200us total) trans 4294967664 rc 0/0 00000100:00100000:0.0:1788619916.453975:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 602 00000100:00100000:0.0:1788619916.457030:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619916.457081:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619916.457189:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571452416 00000100:00100000:2.0:1788619916.457198:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571452416 00000100:00100000:2.0:1788619916.457251:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 772 00000100:00100000:2.0:1788619916.457262:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dc9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571452416:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619916.457280:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619916.457298:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008901cab6/0x1cd7fe5368734b84 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bfb8 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619916.457395:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900d5d180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589116672:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619916.457476:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc780 going to sleep for 11 seconds 00000100:00100000:3.0:1788619916.458824:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571452288 00000100:00100000:3.0:1788619916.458832:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571452288 00000100:00100000:3.0:1788619916.458889:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 970 00000100:00100000:3.0:1788619916.458900:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dc9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571452288:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619916.458914:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619916.458932:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368734b8b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bfb1 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619916.458965:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734b53 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf87 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619916.458997:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6859c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589116800:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619916.459138:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa240 going to sleep for 11 seconds 00000100:00100000:2.0:1788619916.460203:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900d5d180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589116672:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619916.460232:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008901cab6/0x1cd7fe5368734b84 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bfb8 expref: 9 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.460254:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008901cab6/0x1cd7fe5368734b84 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bfb8 expref: 9 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.460268:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c779a600, rc 0) 00000100:00100000:2.0:1788619916.460315:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dc9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571452416:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3054us (3232us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619916.460357:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 772 00000004:00010000:3.0:1788619916.462892:0:5845:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619914 -> 1788619915 00000100:00100000:3.0:1788619916.462922:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6859c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589116800:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619916.462940:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 729088 00010000:00010000:3.0:1788619916.462957:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368734b8b lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bfb1 expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619916.462974:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619916.462977:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619916.462983:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368734b8b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bfb1 expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619916.463016:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683ca00, rc 0) 00000100:00100000:3.0:1788619916.463084:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dc9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571452288:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4186us (6063us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619916.463094:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 970 00000100:00100000:1.0:1788619916.465895:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619916.465936:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619916.466044:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571452672 00000100:00100000:2.0:1788619916.466053:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571452672 00000100:00100000:2.0:1788619916.466121:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 773 00000100:00100000:2.0:1788619916.466132:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6f84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571452672:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619916.466150:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619916.466167:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000528c47b1/0x1cd7fe5368734b92 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bfc6 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619916.466223:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900d5d180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589116928:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619916.466299:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc780 going to sleep for 11 seconds 00000100:00100000:0.0:1788619916.466807:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571452544 00000100:00100000:0.0:1788619916.466813:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571452544 00000100:00100000:0.0:1788619916.466855:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 971 00000100:00100000:0.0:1788619916.466864:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6f84000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571452544:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619916.466876:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619916.466890:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a800/0x1cd7fe5368734b99 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bfbf expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.466943:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734b53 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf87 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619916.466976:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5dcb100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589117056:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619916.467077:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c3579000 going to sleep for 11 seconds 00000100:00100000:2.0:1788619916.471687:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900d5d180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589116928:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619916.471727:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000528c47b1/0x1cd7fe5368734b92 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bfc6 expref: 9 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.471753:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000528c47b1/0x1cd7fe5368734b92 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bfc6 expref: 9 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.471765:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c779aa00, rc 0) 00000100:00100000:2.0:1788619916.471841:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6f84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571452672:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5711us (5955us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619916.471850:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 773 00000100:00100000:0.0:1788619916.472562:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5dcb100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589117056:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619916.472581:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 729088 00010000:00010000:0.0:1788619916.472596:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a800/0x1cd7fe5368734b99 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bfbf expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.472613:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619916.472616:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619916.472622:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a800/0x1cd7fe5368734b99 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bfbf expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.472634:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767a800, rc 0) 00000100:00100000:0.0:1788619916.472695:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6f84000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571452544:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5833us (6758us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619916.472703:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 971 00000100:00100000:1.0:1788619916.477078:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619916.477157:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571452800 00000100:00100000:2.0:1788619916.477166:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571452800 00000100:00100000:2.0:1788619916.477228:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 972 00000100:00100000:2.0:1788619916.477238:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d5df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+14:14908:x1875503571452800:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619916.477262:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619916.477282:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779a600/0x1cd7fe5368734ba0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bfcd expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.477410:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779ba00/0x1cd7fe5368734ba7 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.477427:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779ba00/0x1cd7fe5368734ba7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.477438:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779ba00/0x1cd7fe5368734ba7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788619916.477483:0:11397:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 2088103 00000004:00200000:2.0:1788619916.477488:0:11397:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788619916.477507:0:11397:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779ba00/0x1cd7fe5368734ba7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.477533:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779a600/0x1cd7fe5368734ba0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bfcd expref: 16 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.477551:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779ba00/0x1cd7fe5368734ba7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bfcd expref: 15 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.477559:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619916.477562:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788619916.477566:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c779ba00, rc 0) 00000100:00100000:2.0:1788619916.477839:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d5df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+15:14908:x1875503571452800:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 379us (559us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619916.477849:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 972 00000100:00100000:1.0:1788619916.484526:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619916.484771:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571452928 00000100:00100000:2.0:1788619916.484780:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571452928 00002000:00100000:2.0:1788619916.484840:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1572864->1601535) req@ffff8e77c5d5ea00 x1875503571452928/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:12/0 lens 488/0 e 0 to 0 dl 1788619927 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619916.484851:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5d5ea00 with x1875503571452928 ext(1572864->1601535) 00002000:00010000:2.0:1788619916.484856:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5d5ea00 00000100:00100000:2.0:1788619916.484895:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 409 00000100:00100000:2.0:1788619916.484906:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d5ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571452928:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:2.0:1788619916.489835:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d5ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2352:x1875503571452928:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 4921us (5306us total) trans 4294967419 rc 0/0 00000100:00100000:2.0:1788619916.489857:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 409 00002000:00100000:2.0:1788619916.489871:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1572864->1601535) req@ffff8e77c5d5ea00 x1875503571452928/t4294967419(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:12/0 lens 488/448 e 0 to 0 dl 1788619927 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619916.489882:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5d5ea00 with x1875503571452928 ext(1572864->1601535) 00002000:00010000:2.0:1788619916.489889:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5d5ea00 00000100:00100000:1.0:1788619916.493184:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619916.493422:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571453056 00000100:00100000:2.0:1788619916.493434:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571453056 00000100:00100000:2.0:1788619916.493494:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 774 00000100:00100000:2.0:1788619916.493505:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d5ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571453056:12345-192.168.201.5@tcp:16:fsx.0 00000001:00080000:3.0:1788619916.526882:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967417 is committed 00000001:00080000:3.0:1788619916.526915:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967414 is committed 00000001:00080000:3.0:1788619916.526932:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967419 is committed 00000100:00100000:1.0:1788619916.527299:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d5ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571453056:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 33790us (34121us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619916.527312:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 774 00000100:00100000:3.0:1788619916.530939:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619916.531071:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571453184 00000100:00100000:0.0:1788619916.531081:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571453184 00000100:00100000:0.0:1788619916.531154:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 973 00000100:00100000:0.0:1788619916.531165:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6859c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571453184:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619916.531186:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619916.531207:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a800/0x1cd7fe5368734bae lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bfd4 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.531244:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734b53 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf87 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619916.531285:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5dcb100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589117184:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619916.531395:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78ce7f3a80 going to sleep for 11 seconds 00000100:00100000:3.0:1788619916.532961:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619916.533045:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571453312 00000100:00100000:0.0:1788619916.533052:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571453312 00000100:00100000:0.0:1788619916.533110:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 775 00000100:00100000:0.0:1788619916.533119:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c685b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571453312:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619916.533135:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619916.533150:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000085e95fb9/0x1cd7fe5368734bb5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bfdb expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619916.533202:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5dcb800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589117312:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619916.533273:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78ce7f3540 going to sleep for 11 seconds 00000004:00010000:0.0:1788619916.535189:0:5845:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619915 -> 1788619916 00000100:00100000:0.0:1788619916.535220:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5dcb100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589117184:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619916.535244:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 729088 00010000:00010000:0.0:1788619916.535277:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a800/0x1cd7fe5368734bae lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bfd4 expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.535298:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619916.535301:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619916.535306:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a800/0x1cd7fe5368734bae lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bfd4 expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.535320:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767a800, rc 0) 00000100:00100000:0.0:1788619916.535647:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6859c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571453184:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4482us (4716us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619916.535656:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 973 00002000:00010000:0.0:1788619916.537710:0:6696:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619915 -> 1788619916 00000100:00100000:0.0:1788619916.537739:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5dcb800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589117312:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619916.537774:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000085e95fb9/0x1cd7fe5368734bb5 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bfdb expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.537811:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000085e95fb9/0x1cd7fe5368734bb5 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bfdb expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.537823:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767a400, rc 0) 00000100:00100000:0.0:1788619916.537918:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c685b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571453312:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4799us (4963us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619916.537928:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 775 00000100:00100000:1.0:1788619916.542030:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619916.542125:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571453440 00000100:00100000:3.0:1788619916.542135:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571453440 00000100:00100000:3.0:1788619916.542196:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 776 00000100:00100000:3.0:1788619916.542208:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:14908:x1875503571453440:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619916.542226:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619916.542245:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe5368734bbc lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bfe2 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619916.542276:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000848526f6/0x1cd7fe5368734a81 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1290240->1327103) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bed1 expref: 5 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619916.542286:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=1s v=(1 12) 00010000:00010000:3.0:1788619916.542290:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 12) 00010000:00010000:3.0:1788619916.542296:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe5368734bbc lrc: 4/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1839104->1843199] (req 1839104->1843199) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bfe2 expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619916.542368:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000848526f6/0x1cd7fe5368734a81 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1290240->1327103) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bed1 expref: 5 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619916.542398:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000848526f6/0x1cd7fe5368734a81 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1290240->1327103) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bed1 expref: 5 pid: 7654 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619916.542414:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6858a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589117440:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619916.542496:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa240 going to sleep for 11 seconds 00000100:00100000:2.0:1788619916.546299:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619916.546424:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571453568 00000100:00100000:0.0:1788619916.546434:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571453568 00010000:00010000:0.0:1788619916.546476:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000848526f6/0x1cd7fe5368734a81 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1290240->1327103) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bed1 expref: 6 pid: 7654 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619916.546536:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 324 00000100:00100000:0.0:1788619916.546546:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571453568:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619916.546560:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619916.546583:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000848526f6/0x1cd7fe5368734a81 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1290240->1327103) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bed1 expref: 6 pid: 7654 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.546596:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000848526f6/0x1cd7fe5368734a81 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1290240->1327103) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bed1 expref: 6 pid: 7654 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.546635:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 12) 00010000:00010000:0.0:1788619916.546642:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 12) 00010000:00010000:0.0:1788619916.546652:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe5368734bbc lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1839104->1843199) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bfe2 expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.546663:0:5831:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe5368734bbc lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1839104->1843199) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bfe2 expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.546704:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe5368734bbc lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1839104->1843199) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bfe2 expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619916.546718:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5dcb800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5831:1875503589117568:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619916.546788:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78ce7f3540 going to sleep for 11 seconds 00010000:00010000:3.0:1788619916.547130:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000848526f6/0x1cd7fe5368734a81 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1290240->1327103) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bed1 expref: 6 pid: 7654 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619916.547144:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6858a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589117440:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619916.547167:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe5368734bbc lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1839104->1843199) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bfe2 expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619916.547178:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683ca00, rc 0) 00000100:00100000:3.0:1788619916.547226:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:14908:x1875503571453440:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5021us (5208us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619916.547234:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 776 00000100:00100000:0.0:1788619916.550090:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5dcb800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5831:1875503589117568:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619916.550131:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619916.550892:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571453568:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 4343us (4600us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619916.550904:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 324 00000100:00100000:0.0:1788619916.553997:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619916.554104:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571453696 00000100:00100000:2.0:1788619916.554115:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571453696 00002000:00100000:2.0:1788619916.554173:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1839104->1843199) req@ffff8e78c5dcb800 x1875503571453696/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/0 e 0 to 0 dl 1788619927 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619916.554185:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5dcb800 with x1875503571453696 ext(1839104->1843199) 00002000:00010000:2.0:1788619916.554192:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5dcb800 00000100:00100000:2.0:1788619916.554225:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 410 00000100:00100000:2.0:1788619916.554239:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dcb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571453696:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619916.554533:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dcb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571453696:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 298us (542us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619916.554541:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 410 00002000:00100000:2.0:1788619916.554547:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1839104->1843199) req@ffff8e78c5dcb800 x1875503571453696/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/4536 e 0 to 0 dl 1788619927 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619916.554575:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5dcb800 with x1875503571453696 ext(1839104->1843199) 00002000:00010000:2.0:1788619916.554579:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5dcb800 00000100:00100000:2.0:1788619916.567729:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619916.567854:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571453824 00000100:00100000:0.0:1788619916.567865:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571453824 00002000:00100000:0.0:1788619916.567917:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1843200->1847295) req@ffff8e78c4a4ed80 x1875503571453824/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/0 e 0 to 0 dl 1788619927 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619916.567928:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c4a4ed80 with x1875503571453824 ext(1843200->1847295) 00002000:00010000:0.0:1788619916.567934:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c4a4ed80 00000100:00100000:0.0:1788619916.567972:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 411 00000100:00100000:0.0:1788619916.567982:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571453824:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619916.568266:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571453824:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 284us (548us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619916.568275:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 411 00002000:00100000:0.0:1788619916.568282:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1843200->1847295) req@ffff8e78c4a4ed80 x1875503571453824/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/4536 e 0 to 0 dl 1788619927 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619916.568290:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c4a4ed80 with x1875503571453824 ext(1843200->1847295) 00002000:00010000:0.0:1788619916.568293:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c4a4ed80 00000100:00100000:1.0:1788619916.576891:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619916.577238:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571453952 00000100:00100000:0.0:1788619916.577248:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571453952 00002000:00100000:0.0:1788619916.577300:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1847296->1851391) req@ffff8e77c5d5a300 x1875503571453952/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/0 e 0 to 0 dl 1788619927 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619916.577311:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5d5a300 with x1875503571453952 ext(1847296->1851391) 00002000:00010000:0.0:1788619916.577375:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5d5a300 00000100:00100000:0.0:1788619916.577413:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 412 00000100:00100000:0.0:1788619916.577424:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d5a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571453952:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619916.578085:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d5a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571453952:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 660us (1200us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619916.578095:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 412 00002000:00100000:1.0:1788619916.578103:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1847296->1851391) req@ffff8e77c5d5a300 x1875503571453952/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/4536 e 0 to 0 dl 1788619927 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619916.578112:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5d5a300 with x1875503571453952 ext(1847296->1851391) 00002000:00010000:1.0:1788619916.578117:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5d5a300 00000100:00100000:1.0:1788619916.583079:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619916.583157:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571454080 00000100:00100000:2.0:1788619916.583166:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571454080 00002000:00100000:2.0:1788619916.583207:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1851392->1855487) req@ffff8e77c5d58e00 x1875503571454080/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/0 e 0 to 0 dl 1788619927 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619916.583215:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5d58e00 with x1875503571454080 ext(1851392->1855487) 00002000:00010000:2.0:1788619916.583220:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5d58e00 00000100:00100000:2.0:1788619916.583250:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 413 00000100:00100000:2.0:1788619916.583257:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d58e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571454080:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619916.583510:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d58e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571454080:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 252us (439us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619916.583519:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 413 00002000:00100000:2.0:1788619916.583525:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1851392->1855487) req@ffff8e77c5d58e00 x1875503571454080/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/4536 e 0 to 0 dl 1788619927 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619916.583533:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5d58e00 with x1875503571454080 ext(1851392->1855487) 00002000:00010000:2.0:1788619916.583537:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5d58e00 00000100:00100000:2.0:1788619916.587103:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619916.587208:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571454208 00000100:00100000:1.0:1788619916.587218:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571454208 00002000:00100000:1.0:1788619916.587264:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1855488->1859583) req@ffff8e77c5cd4380 x1875503571454208/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/0 e 0 to 0 dl 1788619927 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619916.587271:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5cd4380 with x1875503571454208 ext(1855488->1859583) 00002000:00010000:1.0:1788619916.587276:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5cd4380 00000100:00100000:1.0:1788619916.587300:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 414 00000100:00100000:1.0:1788619916.587318:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571454208:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619916.587590:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571454208:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 272us (495us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619916.587596:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 414 00002000:00100000:1.0:1788619916.587601:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1855488->1859583) req@ffff8e77c5cd4380 x1875503571454208/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/4536 e 0 to 0 dl 1788619927 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619916.587606:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5cd4380 with x1875503571454208 ext(1855488->1859583) 00002000:00010000:1.0:1788619916.587609:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5cd4380 00000100:00100000:1.0:1788619916.590307:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619916.590475:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571454336 00000100:00100000:0.0:1788619916.590487:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571454336 00002000:00100000:0.0:1788619916.590544:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1859584->1863679) req@ffff8e77c5d5b480 x1875503571454336/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/0 e 0 to 0 dl 1788619927 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619916.590557:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5d5b480 with x1875503571454336 ext(1859584->1863679) 00002000:00010000:0.0:1788619916.590563:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5d5b480 00000100:00100000:0.0:1788619916.590593:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 415 00000100:00100000:0.0:1788619916.590602:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571454336:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619916.590940:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571454336:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 337us (640us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619916.590953:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 415 00002000:00100000:0.0:1788619916.590962:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1859584->1863679) req@ffff8e77c5d5b480 x1875503571454336/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/4536 e 0 to 0 dl 1788619927 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619916.590970:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5d5b480 with x1875503571454336 ext(1859584->1863679) 00002000:00010000:0.0:1788619916.590974:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5d5b480 00000100:00100000:0.0:1788619916.593759:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619916.593837:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571454464 00000100:00100000:2.0:1788619916.593848:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571454464 00002000:00100000:2.0:1788619916.593891:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1863680->1867775) req@ffff8e78c5dcb100 x1875503571454464/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/0 e 0 to 0 dl 1788619927 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619916.593914:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5dcb100 with x1875503571454464 ext(1863680->1867775) 00002000:00010000:2.0:1788619916.593919:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5dcb100 00000100:00100000:2.0:1788619916.593946:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 416 00000100:00100000:2.0:1788619916.593955:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dcb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571454464:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619916.594238:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dcb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571454464:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 284us (487us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619916.594244:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 416 00002000:00100000:2.0:1788619916.594249:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1863680->1867775) req@ffff8e78c5dcb100 x1875503571454464/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/4536 e 0 to 0 dl 1788619927 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619916.594254:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5dcb100 with x1875503571454464 ext(1863680->1867775) 00002000:00010000:2.0:1788619916.594257:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5dcb100 00000100:00100000:2.0:1788619916.596636:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619916.596749:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571454592 00000100:00100000:1.0:1788619916.596757:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571454592 00002000:00100000:1.0:1788619916.596813:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1867776->1871871) req@ffff8e77c5cd6d80 x1875503571454592/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/0 e 0 to 0 dl 1788619927 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619916.596824:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5cd6d80 with x1875503571454592 ext(1867776->1871871) 00002000:00010000:1.0:1788619916.596829:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5cd6d80 00000100:00100000:1.0:1788619916.596864:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 417 00000100:00100000:1.0:1788619916.596874:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571454592:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619916.597191:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571454592:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 313us (559us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619916.597200:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 417 00002000:00100000:1.0:1788619916.597207:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1867776->1871871) req@ffff8e77c5cd6d80 x1875503571454592/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/4536 e 0 to 0 dl 1788619927 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619916.597214:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5cd6d80 with x1875503571454592 ext(1867776->1871871) 00002000:00010000:1.0:1788619916.597217:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5cd6d80 00000100:00100000:1.0:1788619916.599055:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619916.599184:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571454720 00000100:00100000:2.0:1788619916.599193:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571454720 00002000:00100000:2.0:1788619916.599236:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1871872->1875967) req@ffff8e77c5d5b100 x1875503571454720/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/0 e 0 to 0 dl 1788619927 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619916.599246:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5d5b100 with x1875503571454720 ext(1871872->1875967) 00002000:00010000:2.0:1788619916.599252:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5d5b100 00000100:00100000:2.0:1788619916.599284:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 418 00000100:00100000:2.0:1788619916.599293:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d5b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571454720:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619916.599746:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d5b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571454720:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 452us (698us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619916.599754:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 418 00002000:00100000:2.0:1788619916.599773:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1871872->1875967) req@ffff8e77c5d5b100 x1875503571454720/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/4536 e 0 to 0 dl 1788619927 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619916.599781:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5d5b100 with x1875503571454720 ext(1871872->1875967) 00002000:00010000:2.0:1788619916.599785:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5d5b100 00000100:00100000:0.0:1788619916.606229:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619916.606692:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571454848 00000100:00100000:1.0:1788619916.606704:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571454848 00002000:00100000:1.0:1788619916.606763:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1875968->1880063) req@ffff8e78c5dc9180 x1875503571454848/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/0 e 0 to 0 dl 1788619927 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619916.606776:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5dc9180 with x1875503571454848 ext(1875968->1880063) 00002000:00010000:1.0:1788619916.606781:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5dc9180 00000100:00100000:1.0:1788619916.606865:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 419 00000100:00100000:1.0:1788619916.606883:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dc9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571454848:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619916.607149:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dc9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571454848:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 276us (933us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619916.607155:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 419 00002000:00100000:1.0:1788619916.607160:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1875968->1880063) req@ffff8e78c5dc9180 x1875503571454848/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/4536 e 0 to 0 dl 1788619927 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619916.607166:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5dc9180 with x1875503571454848 ext(1875968->1880063) 00002000:00010000:1.0:1788619916.607169:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5dc9180 00000100:00100000:3.0:1788619916.613572:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619916.614317:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571454976 00000100:00100000:0.0:1788619916.614343:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571454976 00002000:00100000:0.0:1788619916.614406:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1880064->1884159) req@ffff8e77c6858a80 x1875503571454976/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/0 e 0 to 0 dl 1788619927 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619916.614440:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6858a80 with x1875503571454976 ext(1880064->1884159) 00002000:00010000:0.0:1788619916.614446:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6858a80 00000100:00100000:0.0:1788619916.614478:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 420 00000100:00100000:0.0:1788619916.614488:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6858a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571454976:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619916.614938:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6858a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571454976:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 449us (1375us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619916.614946:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 420 00002000:00100000:0.0:1788619916.614953:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1880064->1884159) req@ffff8e77c6858a80 x1875503571454976/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/4536 e 0 to 0 dl 1788619927 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619916.614961:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6858a80 with x1875503571454976 ext(1880064->1884159) 00002000:00010000:0.0:1788619916.614964:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6858a80 00000100:00100000:2.0:1788619916.623905:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619916.624821:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571455104 00000100:00100000:0.0:1788619916.624836:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571455104 00002000:00100000:0.0:1788619916.624902:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1884160->1888255) req@ffff8e77c5cd4e00 x1875503571455104/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/0 e 0 to 0 dl 1788619927 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619916.624914:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5cd4e00 with x1875503571455104 ext(1884160->1888255) 00002000:00010000:0.0:1788619916.624920:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5cd4e00 00000100:00100000:0.0:1788619916.624960:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 421 00000100:00100000:0.0:1788619916.624970:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571455104:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619916.625251:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571455104:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 281us (1357us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619916.625259:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 421 00002000:00100000:0.0:1788619916.625279:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1884160->1888255) req@ffff8e77c5cd4e00 x1875503571455104/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/4536 e 0 to 0 dl 1788619927 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619916.625287:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5cd4e00 with x1875503571455104 ext(1884160->1888255) 00002000:00010000:0.0:1788619916.625291:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5cd4e00 00000100:00100000:3.0:1788619916.634229:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619916.634423:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571455232 00000100:00100000:2.0:1788619916.634434:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571455232 00002000:00100000:2.0:1788619916.634569:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1888256->1892351) req@ffff8e77c685ad80 x1875503571455232/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/0 e 0 to 0 dl 1788619927 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619916.634583:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c685ad80 with x1875503571455232 ext(1888256->1892351) 00002000:00010000:2.0:1788619916.634590:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c685ad80 00000100:00100000:2.0:1788619916.634635:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 422 00000100:00100000:2.0:1788619916.634647:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c685ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571455232:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619916.636045:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c685ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571455232:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 1391us (1817us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619916.636058:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 422 00002000:00100000:2.0:1788619916.636068:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1888256->1892351) req@ffff8e77c685ad80 x1875503571455232/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/4536 e 0 to 0 dl 1788619927 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619916.636078:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c685ad80 with x1875503571455232 ext(1888256->1892351) 00002000:00010000:2.0:1788619916.636083:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c685ad80 00000100:00100000:3.0:1788619916.643614:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619916.643956:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571455360 00000100:00100000:1.0:1788619916.643968:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571455360 00002000:00100000:1.0:1788619916.644017:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1892352->1896447) req@ffff8e77c6858700 x1875503571455360/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/0 e 0 to 0 dl 1788619927 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619916.644029:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6858700 with x1875503571455360 ext(1892352->1896447) 00002000:00010000:1.0:1788619916.644035:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6858700 00000100:00100000:1.0:1788619916.644134:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 423 00000100:00100000:1.0:1788619916.644144:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6858700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571455360:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619916.644555:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6858700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571455360:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 411us (950us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619916.644563:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 423 00002000:00100000:1.0:1788619916.644570:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1892352->1896447) req@ffff8e77c6858700 x1875503571455360/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/4536 e 0 to 0 dl 1788619927 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619916.644579:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6858700 with x1875503571455360 ext(1892352->1896447) 00002000:00010000:1.0:1788619916.644583:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6858700 00000100:00100000:1.0:1788619916.648452:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619916.648722:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571455488 00000100:00100000:2.0:1788619916.648780:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571455488 00002000:00100000:2.0:1788619916.648842:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1896448->1900543) req@ffff8e77c5d58380 x1875503571455488/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/0 e 0 to 0 dl 1788619927 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619916.648874:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5d58380 with x1875503571455488 ext(1896448->1900543) 00002000:00010000:2.0:1788619916.648881:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5d58380 00000100:00100000:2.0:1788619916.648919:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 424 00000100:00100000:2.0:1788619916.648929:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571455488:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619916.649203:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571455488:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 274us (760us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619916.649211:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 424 00002000:00100000:2.0:1788619916.649218:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1896448->1900543) req@ffff8e77c5d58380 x1875503571455488/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/4536 e 0 to 0 dl 1788619927 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619916.649226:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5d58380 with x1875503571455488 ext(1896448->1900543) 00002000:00010000:2.0:1788619916.649230:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5d58380 00000100:00100000:3.0:1788619916.653755:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619916.654259:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571455616 00000100:00100000:1.0:1788619916.654271:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571455616 00002000:00100000:1.0:1788619916.654319:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1900544->1904639) req@ffff8e77c6858000 x1875503571455616/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/0 e 0 to 0 dl 1788619927 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619916.654364:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6858000 with x1875503571455616 ext(1900544->1904639) 00002000:00010000:1.0:1788619916.654371:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6858000 00000100:00100000:1.0:1788619916.654395:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 425 00000100:00100000:1.0:1788619916.654405:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6858000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571455616:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619916.654727:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6858000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571455616:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 323us (980us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619916.654735:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 425 00002000:00100000:1.0:1788619916.654742:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1900544->1904639) req@ffff8e77c6858000 x1875503571455616/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/4536 e 0 to 0 dl 1788619927 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619916.654751:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6858000 with x1875503571455616 ext(1900544->1904639) 00002000:00010000:1.0:1788619916.654755:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6858000 00000100:00100000:2.0:1788619916.659930:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619916.660110:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571455744 00000100:00100000:1.0:1788619916.660120:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571455744 00002000:00100000:1.0:1788619916.660166:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1904640->1908735) req@ffff8e77c5cd5880 x1875503571455744/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/0 e 0 to 0 dl 1788619927 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619916.660177:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5cd5880 with x1875503571455744 ext(1904640->1908735) 00002000:00010000:1.0:1788619916.660183:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5cd5880 00000100:00100000:1.0:1788619916.660275:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 426 00000100:00100000:1.0:1788619916.660284:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571455744:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619916.660589:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571455744:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 305us (668us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619916.660598:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 426 00002000:00100000:1.0:1788619916.660605:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1904640->1908735) req@ffff8e77c5cd5880 x1875503571455744/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/4536 e 0 to 0 dl 1788619927 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619916.660625:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5cd5880 with x1875503571455744 ext(1904640->1908735) 00002000:00010000:1.0:1788619916.660628:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5cd5880 00000100:00100000:2.0:1788619916.665419:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619916.665692:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571455872 00000100:00100000:0.0:1788619916.665702:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571455872 00000100:00100000:0.0:1788619916.665814:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 777 00000100:00100000:0.0:1788619916.665824:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571455872:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619916.665843:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619916.665862:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d412ceb8/0x1cd7fe5368734bc3 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bfe9 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.665887:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 12) 00010000:00010000:0.0:1788619916.665892:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 12) 00010000:00010000:0.0:1788619916.665901:0:6698:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d412ceb8/0x1cd7fe5368734bc3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bfe9 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.665917:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d412ceb8/0x1cd7fe5368734bc3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bfe9 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.665929:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767ba00, rc 0) 00000100:00100000:0.0:1788619916.665986:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571455872:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 164us (577us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619916.665991:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 777 00000100:00100000:3.0:1788619916.669815:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619916.670262:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571456000 00000100:00100000:1.0:1788619916.670271:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571456000 00000100:00100000:1.0:1788619916.670439:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 778 00000100:00100000:1.0:1788619916.670450:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6858e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:14908:x1875503571456000:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619916.670479:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619916.670497:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c03a8843/0x1cd7fe5368734bca lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bff0 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.670530:0:7654:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe5368734bbc lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1839104->1843199) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bfe2 expref: 5 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.670559:0:7654:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000d412ceb8/0x1cd7fe5368734bc3 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bfe9 expref: 7 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.670567:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 12) 00010000:00010000:1.0:1788619916.670572:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 12) 00010000:00010000:1.0:1788619916.670577:0:7654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c03a8843/0x1cd7fe5368734bca lrc: 5/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [1544192->1613823] (req 1544192->1613823) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bff0 expref: 7 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.670623:0:7654:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000d412ceb8/0x1cd7fe5368734bc3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bfe9 expref: 7 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.670639:0:7654:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000d412ceb8/0x1cd7fe5368734bc3 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bfe9 expref: 7 pid: 6698 timeout: 509 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619916.670654:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589117696:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619916.670755:0:7654:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe5368734bbc lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1839104->1843199) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bfe2 expref: 5 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.670766:0:7654:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe5368734bbc lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1839104->1843199) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bfe2 expref: 5 pid: 6697 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619916.670776:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589117824:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619916.670809:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c300 going to sleep for 11 seconds 00000100:00100000:2.0:1788619916.676289:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619916.676397:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571456128 00000100:00100000:0.0:1788619916.676407:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571456128 00010000:00010000:0.0:1788619916.676474:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe5368734bbc lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1839104->1843199) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bfe2 expref: 6 pid: 6697 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619916.676511:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 325 00000100:00100000:0.0:1788619916.676521:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c33480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571456128:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619916.676538:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619916.676561:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe5368734bbc lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1839104->1843199) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bfe2 expref: 6 pid: 6697 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.676574:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe5368734bbc lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1839104->1843199) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bfe2 expref: 6 pid: 6697 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.676601:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619916.676662:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c33480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571456128:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 142us (383us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619916.676668:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 325 00000100:00100000:1.0:1788619916.677825:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589117696:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619916.677844:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe5368734bbc lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1839104->1843199) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bfe2 expref: 5 pid: 6697 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619916.677861:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589117824:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619916.677888:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c03a8843/0x1cd7fe5368734bca lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1544192->1613823] (req 1544192->1613823) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bff0 expref: 7 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.677913:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752a00, rc 0) 00000100:00100000:1.0:1788619916.677974:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6858e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+7:14908:x1875503571456000:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7525us (8169us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619916.677982:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 778 00000100:00100000:2.0:1788619916.679803:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619916.680024:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571456256 00000100:00100000:0.0:1788619916.680033:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571456256 00010000:00010000:0.0:1788619916.680082:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000d412ceb8/0x1cd7fe5368734bc3 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bfe9 expref: 7 pid: 6698 timeout: 509 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619916.680128:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 326 00000100:00100000:0.0:1788619916.680137:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c30700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571456256:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619916.680152:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619916.680178:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d412ceb8/0x1cd7fe5368734bc3 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bfe9 expref: 7 pid: 6698 timeout: 509 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.680212:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d412ceb8/0x1cd7fe5368734bc3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bfe9 expref: 7 pid: 6698 timeout: 509 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.680236:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d412ceb8/0x1cd7fe5368734bc3 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bfe9 expref: 7 pid: 6698 timeout: 509 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.680253:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 12) 00010000:00010000:0.0:1788619916.680257:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 12) 00010000:00010000:0.0:1788619916.680267:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000c03a8843/0x1cd7fe5368734bca lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1544192->1613823) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bff0 expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.680277:0:13330:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c03a8843/0x1cd7fe5368734bca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1544192->1613823) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bff0 expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.680314:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000c03a8843/0x1cd7fe5368734bca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1544192->1613823) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bff0 expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619916.680371:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5dcad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589117952:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619916.680451:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78ce7f39c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619916.685366:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5dcad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589117952:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619916.685393:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619916.685454:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c30700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571456256:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 5317us (5666us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619916.685475:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 326 00000100:00100000:2.0:1788619916.690004:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619916.690122:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571456384 00000100:00100000:0.0:1788619916.690136:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571456384 00002000:00100000:0.0:1788619916.690201:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1544192->2088959) req@ffff8e78c5c31880 x1875503571456384/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:12/0 lens 504/0 e 0 to 0 dl 1788619927 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619916.690214:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5c31880 with x1875503571456384 ext(1544192->2088959) 00002000:00010000:0.0:1788619916.690220:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5c31880 00000100:00100000:0.0:1788619916.690263:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 427 00000100:00100000:0.0:1788619916.690273:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c31880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571456384:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619916.694250:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c31880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571456384:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 3966us (4249us total) trans 0 rc 16384/16384 00000100:00100000:0.0:1788619916.694263:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 427 00002000:00100000:0.0:1788619916.694276:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1544192->2088959) req@ffff8e78c5c31880 x1875503571456384/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:12/0 lens 504/440 e 0 to 0 dl 1788619927 ref 1 fl Complete:/600/0 rc 16384/16384 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619916.694287:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5c31880 with x1875503571456384 ext(1544192->2088959) 00002000:00010000:0.0:1788619916.694292:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5c31880 00000100:00100000:3.0:1788619916.697777:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619916.697944:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571456512 00000100:00100000:1.0:1788619916.697954:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571456512 00002000:00100000:1.0:1788619916.698008:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1609728->1613823) req@ffff8e77c6859180 x1875503571456512/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:12/0 lens 488/0 e 0 to 0 dl 1788619927 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619916.698020:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6859180 with x1875503571456512 ext(1609728->1613823) 00002000:00010000:1.0:1788619916.698026:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6859180 00000100:00100000:1.0:1788619916.698076:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 428 00000100:00100000:1.0:1788619916.698087:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6859180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571456512:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619916.698300:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6859180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571456512:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 213us (532us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619916.698307:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 428 00002000:00100000:1.0:1788619916.698314:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1609728->1613823) req@ffff8e77c6859180 x1875503571456512/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:12/0 lens 488/4536 e 0 to 0 dl 1788619927 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619916.698391:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6859180 with x1875503571456512 ext(1609728->1613823) 00002000:00010000:1.0:1788619916.698396:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6859180 00000100:00100000:2.0:1788619916.703734:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619916.703849:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571456768 00000100:00100000:1.0:1788619916.703859:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571456768 00000100:00100000:1.0:1788619916.703924:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 974 00000100:00100000:1.0:1788619916.703935:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c77e3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+15:14908:x1875503571456768:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:1.0:1788619916.703988:0:5846:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619916.704016:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779ba00/0x1cd7fe5368734ba7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bfcd expref: 15 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.704035:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619916.704040:0:5846:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788619916.704268:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753a00/0x1cd7fe5368734bd1 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.704284:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753200/0x1cd7fe5368734b61 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf8e expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.704294:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753a00/0x1cd7fe5368734bd1 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.704497:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753200/0x1cd7fe5368734b61 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf8e expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.704514:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753200/0x1cd7fe5368734b61 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf8e expref: 13 pid: 5844 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619916.704549:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501ce00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589118080:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619916.704635:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c600 going to sleep for 11 seconds 00000100:00100000:2.0:1788619916.708296:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619916.708401:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571456896 00000100:00100000:0.0:1788619916.708410:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571456896 00010000:00010000:0.0:1788619916.708453:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753200/0x1cd7fe5368734b61 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf8e expref: 14 pid: 5844 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619916.708497:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 327 00000100:00100000:0.0:1788619916.708508:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8559c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571456896:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619916.708522:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619916.708529:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753200/0x1cd7fe5368734b61 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf8e expref: 14 pid: 5844 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.708564:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753200/0x1cd7fe5368734b61 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf8e expref: 14 pid: 5844 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.708588:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619916.708594:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753a00/0x1cd7fe5368734bd1 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.708606:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753a00/0x1cd7fe5368734bd1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.708625:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753a00/0x1cd7fe5368734bd1 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.708646:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619916.708701:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8559c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571456896:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 195us (416us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619916.708709:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 327 00010000:00010000:1.0:1788619916.709092:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753200/0x1cd7fe5368734b61 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf8e expref: 13 pid: 5844 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619916.709111:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501ce00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589118080:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619916.709131:0:5846:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753a00/0x1cd7fe5368734bd1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.709141:0:5846:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753a00/0x1cd7fe5368734bd1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.709148:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753a00/0x1cd7fe5368734bd1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788619916.709268:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967665 00000004:00010000:1.0:1788619916.709317:0:5846:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 729088 -> 729088 00000004:00010000:1.0:1788619916.709321:0:5846:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619916 -> 1788619916 00000004:00010000:1.0:1788619916.709350:0:5846:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619916 -> 1788619914 00000004:00010000:1.0:1788619916.709353:0:5846:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619916 -> 1788619916 00000004:00010000:1.0:1788619916.709356:0:5846:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 902 -> 902 00000004:00010000:1.0:1788619916.709363:0:5846:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000001b344dd reply state 000000006377be87 transno 4294967666 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753a00/0x1cd7fe5368734bd1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619916.709372:0:5846:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788619916.709456:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c77e3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+17:14908:x1875503571456768:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 5523us (5732us total) trans 4294967666 rc 0/0 00000100:00100000:1.0:1788619916.709475:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 974 00000020:00080000:1.0:1788619916.709479:0:5846:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1788619916.712812:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619916.712848:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619916.713072:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571457152 00000100:00100000:0.0:1788619916.713083:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571457152 00002000:00010000:0.0:1788619916.713124:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:0.0:1788619916.713130:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5770000 with x1875503571457152 ext(0->18446744073709551615) 00002000:00010000:0.0:1788619916.713141:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5770000. 00000100:00100000:0.0:1788619916.713179:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 429 00000100:00100000:0.0:1788619916.713189:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5770000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571457152:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:0.0:1788619916.713554:0:6702:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 2088103 -> 0 00002000:00010000:0.0:1788619916.713558:0:6702:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619916 -> 1788619916 00002000:00010000:0.0:1788619916.713560:0:6702:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619916 -> 1788619915 00002000:00010000:0.0:1788619916.713576:0:6702:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619916 -> 1788619916 00002000:00010000:0.0:1788619916.713579:0:6702:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 519 -> 391 00000100:00100000:0.0:1788619916.713649:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5770000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571457152:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 460us (798us total) trans 4294967420 rc 0/0 00000100:00100000:0.0:1788619916.713658:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 429 00002000:00010000:0.0:1788619916.713662:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:0.0:1788619916.713664:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5770000 with x1875503571457152 ext(0->18446744073709551615) 00002000:00010000:0.0:1788619916.713669:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5770000. 00010000:00010000:3.0:1788619916.714169:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753a00/0x1cd7fe5368734bd1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619916.714196:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753a00/0x1cd7fe5368734bd1 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619916.714203:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753a00/0x1cd7fe5368734bd1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619916.714210:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c5753a00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:3.0:1788619916.714214:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753a00/0x1cd7fe5368734bd1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619916.714222:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753a00/0x1cd7fe5368734bd1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619916.714238:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619916.714241:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788619916.714246:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753a00/0x1cd7fe5368734bd1 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619916.714253:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753a00/0x1cd7fe5368734bd1 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619916.715106:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571457024 00000100:00100000:1.0:1788619916.715116:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571457024 00002000:00010000:1.0:1788619916.715156:0:5858:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:1.0:1788619916.715162:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c77e3b80 with x1875503571457024 00000100:00100000:1.0:1788619916.715191:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 603 00000100:00100000:1.0:1788619916.715201:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c77e3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571457024:12345-192.168.201.5@tcp:10:fsx.0 00000004:00010000:1.0:1788619916.715592:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 729088 -> 149756 00000004:00010000:1.0:1788619916.715597:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619916 -> 1788619916 00000004:00010000:1.0:1788619916.715599:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619914 -> 1788619914 00000004:00010000:1.0:1788619916.715602:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619916 -> 1788619916 00000004:00010000:1.0:1788619916.715605:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 902 -> 902 00000100:00100000:1.0:1788619916.715670:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c77e3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+17:2355:x1875503571457024:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 469us (2866us total) trans 4294967667 rc 0/0 00000100:00100000:1.0:1788619916.715677:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 603 00002000:00010000:1.0:1788619916.715682:0:5858:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:1.0:1788619916.715684:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c77e3b80 with x1875503571457024 00000100:00100000:2.0:1788619916.719942:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619916.720009:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571457280 00000100:00100000:1.0:1788619916.720020:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571457280 00000100:00100000:1.0:1788619916.720084:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 975 00000100:00100000:1.0:1788619916.720094:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5771880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571457280:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619916.720116:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619916.720138:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752000/0x1cd7fe5368734bd8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bff7 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.720236:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752600/0x1cd7fe5368734bdf lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.720252:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752600/0x1cd7fe5368734bdf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.720262:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752600/0x1cd7fe5368734bdf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788619916.720286:0:11397:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 149756 00000004:00200000:1.0:1788619916.720290:0:11397:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788619916.720305:0:11397:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752600/0x1cd7fe5368734bdf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.720375:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752000/0x1cd7fe5368734bd8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bff7 expref: 15 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.720392:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752600/0x1cd7fe5368734bdf lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bff7 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.720402:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619916.720404:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788619916.720408:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752600, rc 0) 00000100:00100000:1.0:1788619916.720466:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5771880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571457280:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 373us (533us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619916.720473:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 975 00000100:00100000:2.0:1788619916.724251:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619916.724366:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571457408 00000100:00100000:0.0:1788619916.724376:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571457408 00000100:00100000:0.0:1788619916.724436:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 976 00000100:00100000:0.0:1788619916.724447:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5770700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571457408:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619916.724471:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619916.724492:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767ba00/0x1cd7fe5368734be6 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1bffe expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.724531:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734b53 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf87 expref: 16 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619916.724573:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5dc8a80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589118208:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619916.724828:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78ce7f3000 going to sleep for 11 seconds 00000100:00100000:3.0:1788619916.727248:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000004:00010000:1.0:1788619916.727393:0:5845:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619914 -> 1788619916 00000100:00100000:1.0:1788619916.727414:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5dc8a80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589118208:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619916.727449:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 149756 00010000:00010000:1.0:1788619916.727467:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767ba00/0x1cd7fe5368734be6 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bffe expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.727485:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619916.727488:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619916.727494:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767ba00/0x1cd7fe5368734be6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bffe expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.727509:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767ba00, rc 0) 00000100:00100000:1.0:1788619916.727559:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5770700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571457408:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3113us (3321us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619916.727566:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 976 00000100:00100000:0.0:1788619916.728105:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571457536 00000100:00100000:0.0:1788619916.728113:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571457536 00000100:00100000:0.0:1788619916.728160:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 779 00000100:00100000:0.0:1788619916.728169:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c685bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571457536:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619916.728182:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619916.728202:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000072940acc/0x1cd7fe5368734bed lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c005 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619916.728250:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5dcad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589118336:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619916.728315:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78ce7f3b40 going to sleep for 11 seconds 00002000:00010000:0.0:1788619916.731015:0:6696:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619915 -> 1788619916 00000100:00100000:0.0:1788619916.731047:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5dcad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589118336:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619916.731092:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000072940acc/0x1cd7fe5368734bed lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c005 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.731116:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000072940acc/0x1cd7fe5368734bed lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c005 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.731129:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767b000, rc 0) 00000100:00100000:0.0:1788619916.731186:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c685bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571457536:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3017us (3945us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619916.731194:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 779 00000100:00100000:1.0:1788619916.735744:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619916.735800:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619916.735878:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571457792 00000100:00100000:3.0:1788619916.735888:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571457792 00000100:00100000:3.0:1788619916.735955:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 780 00000100:00100000:3.0:1788619916.735966:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571457792:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619916.735983:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619916.736001:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe5368734bf4 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c013 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619916.736091:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c685a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589118464:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619916.736176:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa240 going to sleep for 11 seconds 00000100:00100000:0.0:1788619916.736681:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571457664 00000100:00100000:0.0:1788619916.736687:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571457664 00000100:00100000:0.0:1788619916.736718:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 977 00000100:00100000:0.0:1788619916.736726:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571457664:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619916.736736:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619916.736750:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767bc00/0x1cd7fe5368734bfb lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c00c expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.736781:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734b53 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf87 expref: 16 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619916.736825:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5dcad80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589118592:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619916.736873:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78ce7f3b40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619916.739943:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5dcad80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589118592:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619916.739978:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 149756 00010000:00010000:0.0:1788619916.739997:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767bc00/0x1cd7fe5368734bfb lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c00c expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.740019:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619916.740021:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619916.740026:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767bc00/0x1cd7fe5368734bfb lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c00c expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.740040:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767bc00, rc 0) 00000100:00100000:0.0:1788619916.740111:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571457664:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3385us (4378us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619916.740119:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 977 00000100:00100000:3.0:1788619916.742882:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c685a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589118464:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619916.742938:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe5368734bf4 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c013 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619916.742962:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe5368734bf4 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c013 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619916.742977:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683dc00, rc 0) 00000100:00100000:3.0:1788619916.743042:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571457792:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7078us (7251us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619916.743050:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 780 00000100:00100000:2.0:1788619916.746848:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619916.746943:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571457920 00000100:00100000:0.0:1788619916.746953:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571457920 00000100:00100000:0.0:1788619916.747016:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 781 00000100:00100000:0.0:1788619916.747027:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6342a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:14908:x1875503571457920:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619916.747044:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619916.747073:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000072940acc/0x1cd7fe5368734c02 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c01a expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.747109:0:6698:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000c03a8843/0x1cd7fe5368734bca lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1544192->1613823) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bff0 expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.747131:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 12) 00010000:00010000:0.0:1788619916.747136:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(3 12) 00010000:00010000:0.0:1788619916.747143:0:6698:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000072940acc/0x1cd7fe5368734c02 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1077248->1134591] (req 1077248->1134591) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c01a expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.747177:0:6698:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000c03a8843/0x1cd7fe5368734bca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1544192->1613823) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bff0 expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.747190:0:6698:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000c03a8843/0x1cd7fe5368734bca lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1544192->1613823) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bff0 expref: 6 pid: 7654 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619916.747204:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5dcad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589118720:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619916.747289:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78ce7f3b40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619916.749586:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5dcad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589118720:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619916.749630:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000072940acc/0x1cd7fe5368734c02 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1077248->1134591] (req 1077248->1134591) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c01a expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.749652:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767b000, rc 0) 00000100:00100000:0.0:1788619916.749708:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6342a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:14908:x1875503571457920:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2682us (2870us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619916.749716:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 781 00000100:00100000:2.0:1788619916.752858:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619916.752971:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571458048 00000100:00100000:1.0:1788619916.752982:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571458048 00010000:00010000:1.0:1788619916.753021:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000c03a8843/0x1cd7fe5368734bca lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1544192->1613823) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bff0 expref: 7 pid: 7654 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619916.753076:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 328 00000100:00100000:1.0:1788619916.753086:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6343800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2353:x1875503571458048:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619916.753105:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619916.753131:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000c03a8843/0x1cd7fe5368734bca lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1544192->1613823) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bff0 expref: 7 pid: 7654 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.753147:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000c03a8843/0x1cd7fe5368734bca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1544192->1613823) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bff0 expref: 7 pid: 7654 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.753186:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c03a8843/0x1cd7fe5368734bca lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1544192->1613823) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bff0 expref: 7 pid: 7654 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.753208:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(3 12) 00010000:00010000:1.0:1788619916.753213:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(3 12) 00010000:00010000:1.0:1788619916.753223:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000072940acc/0x1cd7fe5368734c02 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1077248->1134591) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c01a expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.753235:0:5831:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000072940acc/0x1cd7fe5368734c02 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1077248->1134591) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c01a expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.753279:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000072940acc/0x1cd7fe5368734c02 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1077248->1134591) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c01a expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619916.753295:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501d500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5831:1875503589118848:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619916.753422:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c600 going to sleep for 11 seconds 00000100:00100000:2.0:1788619916.758504:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619916.758589:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571458176 00000100:00100000:3.0:1788619916.758598:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571458176 00000100:00100000:3.0:1788619916.758664:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 978 00000100:00100000:3.0:1788619916.758674:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6342300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+17:14908:x1875503571458176:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619916.758696:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619916.758716:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368734c09 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c021 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619916.758816:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368734c10 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619916.758847:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368734c10 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619916.758858:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368734c10 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788619916.758891:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 149756 00000004:00200000:3.0:1788619916.758895:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788619916.758914:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368734c10 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619916.758940:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368734c09 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c021 expref: 19 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619916.758957:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368734c10 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c021 expref: 18 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619916.758965:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619916.758968:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788619916.758972:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c800, rc 0) 00000100:00100000:3.0:1788619916.759031:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6342300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+18:14908:x1875503571458176:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 359us (538us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619916.759038:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 978 00000100:00100000:1.0:1788619916.759547:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501d500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5831:1875503589118848:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619916.759571:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619916.759609:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6343800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571458048:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 6525us (6761us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619916.759618:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 328 00000100:00100000:2.0:1788619916.762968:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619916.763092:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571458304 00000100:00100000:1.0:1788619916.763101:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571458304 00000100:00100000:1.0:1788619916.763199:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 782 00000100:00100000:1.0:1788619916.763234:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6341f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571458304:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619916.763254:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619916.763271:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c03a8843/0x1cd7fe5368734c17 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c028 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619916.763352:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501d500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589118976:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619916.763423:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c600 going to sleep for 11 seconds 00002000:00010000:1.0:1788619916.765956:0:7654:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size: 0 -> 1133889 00000100:00100000:1.0:1788619916.765989:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501d500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589118976:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619916.766025:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c03a8843/0x1cd7fe5368734c17 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c028 expref: 7 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.766063:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c03a8843/0x1cd7fe5368734c17 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c028 expref: 7 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.766077:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752a00, rc 0) 00000100:00100000:1.0:1788619916.766135:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6341f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571458304:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2902us (3175us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619916.766143:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 782 00000100:00100000:0.0:1788619916.768777:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619916.768863:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571458432 00000100:00100000:3.0:1788619916.768871:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571458432 00000100:00100000:3.0:1788619916.768928:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 783 00000100:00100000:3.0:1788619916.768938:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dcad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571458432:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619916.768956:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619916.768987:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe5368734c1e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c02f expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619916.769043:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c685a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589119104:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619916.769136:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa240 going to sleep for 11 seconds 00000100:00100000:3.0:1788619916.771814:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c685a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589119104:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619916.771855:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe5368734c1e lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c02f expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619916.771882:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe5368734c1e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c02f expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619916.771896:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683dc00, rc 0) 00000100:00100000:3.0:1788619916.772026:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dcad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571458432:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3088us (3258us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619916.772035:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 783 00000100:00100000:2.0:1788619916.775718:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619916.776257:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571458688 00000100:00100000:1.0:1788619916.776269:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571458688 00000100:00100000:1.0:1788619916.776437:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 979 00000100:00100000:1.0:1788619916.776448:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571458688:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:1.0:1788619916.776500:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619916.776526:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752600/0x1cd7fe5368734bdf lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bff7 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.776558:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619916.776564:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788619916.776776:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752a00/0x1cd7fe5368734c25 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.776801:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368734c10 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c021 expref: 17 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.776810:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752a00/0x1cd7fe5368734c25 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.776842:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368734c10 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c021 expref: 17 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.776856:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368734c10 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c021 expref: 17 pid: 5844 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619916.776868:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501d500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589119232:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619916.777201:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c600 going to sleep for 11 seconds 00000100:00100000:0.0:1788619916.779671:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619916.779752:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571458816 00000100:00100000:3.0:1788619916.779763:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571458816 00010000:00010000:3.0:1788619916.779820:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368734c10 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c021 expref: 18 pid: 5844 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619916.779871:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 329 00000100:00100000:3.0:1788619916.779882:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dca680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+18:2355:x1875503571458816:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619916.779897:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619916.779921:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368734c10 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c021 expref: 18 pid: 5844 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619916.779953:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368734c10 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c021 expref: 18 pid: 5844 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619916.779976:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619916.779981:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752a00/0x1cd7fe5368734c25 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619916.779991:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752a00/0x1cd7fe5368734c25 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619916.779999:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752a00/0x1cd7fe5368734c25 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619916.780019:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619916.780082:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dca680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+18:2355:x1875503571458816:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 201us (420us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619916.780090:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 329 00010000:00010000:1.0:1788619916.780358:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368734c10 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c021 expref: 17 pid: 5844 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619916.780376:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501d500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589119232:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619916.780397:0:5845:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752a00/0x1cd7fe5368734c25 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.780406:0:5845:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752a00/0x1cd7fe5368734c25 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.780419:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752a00/0x1cd7fe5368734c25 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788619916.780533:0:5845:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967668 00000004:00010000:1.0:1788619916.780585:0:5845:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 149756 -> 149756 00000004:00010000:1.0:1788619916.780588:0:5845:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619916 -> 1788619916 00000004:00010000:1.0:1788619916.780591:0:5845:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619916 -> 1788619914 00000004:00010000:1.0:1788619916.780594:0:5845:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619916 -> 1788619916 00000004:00010000:1.0:1788619916.780596:0:5845:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 902 -> 902 00000004:00010000:1.0:1788619916.780603:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000ed5f3dca reply state 000000007f0655bd transno 4294967669 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752a00/0x1cd7fe5368734c25 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619916.780612:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788619916.780677:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:14908:x1875503571458688:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 4231us (4968us total) trans 4294967669 rc 0/0 00000100:00100000:1.0:1788619916.780683:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 979 00000020:00080000:1.0:1788619916.780686:0:5845:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788619916.783801:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752a00/0x1cd7fe5368734c25 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.783833:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752a00/0x1cd7fe5368734c25 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.783841:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752a00/0x1cd7fe5368734c25 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.783847:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c5752a00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788619916.783853:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752a00/0x1cd7fe5368734c25 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.783862:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752a00/0x1cd7fe5368734c25 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.783880:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619916.783883:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788619916.783889:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752a00/0x1cd7fe5368734c25 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.783895:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752a00/0x1cd7fe5368734c25 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619916.784954:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619916.785011:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571458944 00000100:00100000:1.0:1788619916.785020:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571458944 00000100:00100000:1.0:1788619916.785076:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 980 00000100:00100000:1.0:1788619916.785085:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dc8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:14908:x1875503571458944:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619916.785108:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619916.785125:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752c00/0x1cd7fe5368734c2c lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c036 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.785156:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734b53 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf87 expref: 16 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.785177:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752c00/0x1cd7fe5368734c2c lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c036 expref: 16 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.785208:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734b53 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf87 expref: 16 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.785220:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734b53 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf87 expref: 16 pid: 11397 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619916.785234:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501d500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589119360:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619916.785308:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c900 going to sleep for 11 seconds 00000100:00100000:2.0:1788619916.788667:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619916.788723:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571459072 00000100:00100000:0.0:1788619916.788733:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571459072 00010000:00010000:0.0:1788619916.788771:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734b53 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf87 expref: 17 pid: 11397 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619916.788821:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 330 00000100:00100000:0.0:1788619916.788830:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc1500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+17:2355:x1875503571459072:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619916.788858:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619916.788882:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734b53 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf87 expref: 17 pid: 11397 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.788896:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734b53 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf87 expref: 17 pid: 11397 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.788927:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619916.788932:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752c00/0x1cd7fe5368734c2c lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c036 expref: 16 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.788943:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752c00/0x1cd7fe5368734c2c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c036 expref: 16 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.788953:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752c00/0x1cd7fe5368734c2c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c036 expref: 16 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.788989:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752c00/0x1cd7fe5368734c2c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c036 expref: 16 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619916.789001:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b34a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589119488:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619916.789080:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78ce7f3b40 going to sleep for 11 seconds 00010000:00010000:1.0:1788619916.789282:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734b53 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1bf87 expref: 17 pid: 11397 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619916.789298:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501d500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589119360:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619916.789319:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752c00/0x1cd7fe5368734c2c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c036 expref: 16 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.789361:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619916.789363:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619916.789366:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752c00, rc 0) 00000100:00100000:1.0:1788619916.789402:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dc8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:14908:x1875503571458944:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4318us (4455us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619916.789410:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 980 00000100:00100000:2.0:1788619916.797970:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619916.798132:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b34a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589119488:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619916.798156:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619916.798213:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc1500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571459072:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 9384us (9555us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619916.798242:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 330 00000100:00100000:1.0:1788619916.798805:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571459200 00000100:00100000:1.0:1788619916.798821:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571459200 00002000:00010000:1.0:1788619916.798860:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1566350->18446744073709551615) 00002000:00100000:1.0:1788619916.798865:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78cdcc1180 with x1875503571459200 ext(1566350->18446744073709551615) 00002000:00010000:1.0:1788619916.798875:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78cdcc1180. 00000100:00100000:1.0:1788619916.798917:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 430 00000100:00100000:1.0:1788619916.798925:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571459200:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:1.0:1788619916.799173:0:6703:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1133889 -> 1566350 00002000:00010000:1.0:1788619916.799178:0:6703:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619916 -> 1788619916 00002000:00010000:1.0:1788619916.799181:0:6703:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619916 -> 1788619915 00002000:00010000:1.0:1788619916.799183:0:6703:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619916 -> 1788619916 00002000:00010000:1.0:1788619916.799185:0:6703:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 391 -> 391 00000100:00100000:1.0:1788619916.799232:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571459200:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 308us (1271us total) trans 4294967421 rc 0/0 00000100:00100000:1.0:1788619916.799240:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 430 00002000:00010000:1.0:1788619916.799243:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1566350->18446744073709551615) 00002000:00100000:1.0:1788619916.799245:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78cdcc1180 with x1875503571459200 ext(1566350->18446744073709551615) 00002000:00010000:1.0:1788619916.799250:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78cdcc1180. 00000100:00100000:2.0:1788619916.806530:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619916.806586:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619916.806675:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571459456 00000100:00100000:0.0:1788619916.806685:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571459456 00002000:00100000:0.0:1788619916.806735:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78cdcc3100 x1875503571459456/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/0 e 0 to 0 dl 1788619927 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619916.806747:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78cdcc3100 with x1875503571459456 00000100:00100000:0.0:1788619916.806797:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 604 00000100:00100000:0.0:1788619916.806809:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2352:x1875503571459456:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619916.808248:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571459328 00000100:00100000:1.0:1788619916.808260:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571459328 00002000:00100000:1.0:1788619916.808304:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1568767) req@ffff8e78cdcc3b80 x1875503571459328/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 504/0 e 0 to 0 dl 1788619927 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619916.808315:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78cdcc3b80 with x1875503571459328 ext(1048576->1568767) 00002000:00010000:1.0:1788619916.808321:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78cdcc3b80 00000100:00100000:1.0:1788619916.808383:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 431 00000100:00100000:1.0:1788619916.808392:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2352:x1875503571459328:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619916.810992:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2352:x1875503571459456:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 4175us (4396us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619916.811006:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 604 00002000:00100000:0.0:1788619916.811018:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78cdcc3100 x1875503571459456/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/440 e 0 to 0 dl 1788619927 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619916.811028:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78cdcc3100 with x1875503571459456 00000100:00100000:1.0:1788619916.822966:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2352:x1875503571459328:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 14564us (16435us total) trans 0 rc 462848/462848 00000100:00100000:1.0:1788619916.822982:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 431 00002000:00100000:1.0:1788619916.822995:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1568767) req@ffff8e78cdcc3b80 x1875503571459328/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 504/440 e 0 to 0 dl 1788619927 ref 1 fl Complete:/600/0 rc 462848/462848 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619916.823006:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78cdcc3b80 with x1875503571459328 ext(1048576->1568767) 00002000:00010000:1.0:1788619916.823012:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78cdcc3b80 00000100:00100000:3.0:1788619916.827454:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619916.827555:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571459584 00000100:00100000:0.0:1788619916.827565:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571459584 00002000:00100000:0.0:1788619916.827626:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1077248->1566349) req@ffff8e77c685a300 x1875503571459584/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 504/0 e 0 to 0 dl 1788619927 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619916.827637:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c685a300 with x1875503571459584 ext(1077248->1566349) 00002000:00010000:0.0:1788619916.827642:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c685a300 00000100:00100000:0.0:1788619916.827682:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 432 00000100:00100000:0.0:1788619916.827692:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c685a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2352:x1875503571459584:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:1.0:1788619916.838173:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c685a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2352:x1875503571459584:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 10471us (10718us total) trans 4294967422 rc 0/0 00000100:00100000:1.0:1788619916.838188:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 432 00002000:00100000:1.0:1788619916.838201:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1077248->1566349) req@ffff8e77c685a300 x1875503571459584/t4294967422(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 504/448 e 0 to 0 dl 1788619927 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619916.838212:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c685a300 with x1875503571459584 ext(1077248->1566349) 00002000:00010000:1.0:1788619916.838219:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c685a300 00000100:00100000:0.0:1788619916.841747:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619916.841845:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571459712 00000100:00100000:3.0:1788619916.841856:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571459712 00000100:00100000:3.0:1788619916.841915:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 784 00000100:00100000:3.0:1788619916.841926:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b34a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571459712:12345-192.168.201.5@tcp:16:fsx.0 00000100:00100000:3.0:1788619916.863510:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b34a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571459712:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 21575us (21763us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619916.863524:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 784 00000001:00080000:3.0:1788619916.863565:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967420 is committed 00000001:00080000:3.0:1788619916.863573:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967422 is committed 00000100:00100000:3.0:1788619916.867161:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619916.867345:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571459840 00000100:00100000:2.0:1788619916.867353:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571459840 00000100:00100000:2.0:1788619916.867400:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 981 00000100:00100000:2.0:1788619916.867408:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c685a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+16:14908:x1875503571459840:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619916.867424:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619916.867440:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779aa00/0x1cd7fe5368734c33 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c03d expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.867516:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779a600/0x1cd7fe5368734c3a lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.867531:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779a600/0x1cd7fe5368734c3a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.867543:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779a600/0x1cd7fe5368734c3a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788619916.867570:0:9417:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1566350 00000004:00200000:2.0:1788619916.867573:0:9417:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788619916.867590:0:9417:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779a600/0x1cd7fe5368734c3a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.867630:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779aa00/0x1cd7fe5368734c33 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c03d expref: 18 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.867662:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779a600/0x1cd7fe5368734c3a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c03d expref: 17 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.867671:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619916.867674:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788619916.867678:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c779a600, rc 0) 00000100:00100000:2.0:1788619916.867724:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c685a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+17:14908:x1875503571459840:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 316us (569us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619916.867731:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 981 00000100:00100000:2.0:1788619916.870414:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619916.870442:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619916.870523:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571459968 00000100:00100000:1.0:1788619916.870532:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571459968 00000100:00100000:1.0:1788619916.870586:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 982 00000100:00100000:1.0:1788619916.870596:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+17:2355:x1875503571459968:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619916.870625:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619916.870646:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734c41 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c044 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.870679:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752c00/0x1cd7fe5368734c2c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c036 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619916.870738:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501d500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589119616:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619916.870831:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a900 going to sleep for 11 seconds 00000100:00100000:0.0:1788619916.871377:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571460096 00000100:00100000:0.0:1788619916.871391:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571460096 00000100:00100000:0.0:1788619916.871436:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 785 00000100:00100000:0.0:1788619916.871445:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571460096:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619916.871457:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619916.871469:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006989f8b0/0x1cd7fe5368734c48 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c04b expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619916.871507:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b37480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589119744:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619916.871558:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78ce7f3a80 going to sleep for 11 seconds 00000004:00010000:1.0:1788619916.874869:0:5845:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619914 -> 1788619916 00000100:00100000:1.0:1788619916.874909:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501d500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589119616:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619916.874935:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 149756 00010000:00010000:1.0:1788619916.874956:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734c41 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c044 expref: 18 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.874979:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619916.874982:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619916.874987:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734c41 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c044 expref: 18 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.875002:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752e00, rc 0) 00000100:00100000:1.0:1788619916.875069:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+17:2355:x1875503571459968:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4474us (4625us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619916.875078:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 982 00002000:00010000:0.0:1788619916.875752:0:6698:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619915 -> 1788619916 00000100:00100000:0.0:1788619916.875772:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b37480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589119744:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619916.875806:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000006989f8b0/0x1cd7fe5368734c48 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c04b expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.875824:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006989f8b0/0x1cd7fe5368734c48 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c04b expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.875834:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767bc00, rc 0) 00000100:00100000:0.0:1788619916.875868:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571460096:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4425us (5464us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619916.875876:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 785 00000100:00100000:2.0:1788619916.879090:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619916.879166:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571460224 00000100:00100000:1.0:1788619916.879175:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571460224 00000100:00100000:1.0:1788619916.879229:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 983 00000100:00100000:1.0:1788619916.879238:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+17:2352:x1875503571460224:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619916.879256:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619916.879275:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753000/0x1cd7fe5368734c4f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c052 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.879366:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752c00/0x1cd7fe5368734c2c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c036 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619916.879404:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501d500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589119872:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619916.879480:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a900 going to sleep for 11 seconds 00000100:00100000:1.0:1788619916.881690:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619916.881829:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571460352 00000100:00100000:2.0:1788619916.881837:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571460352 00000100:00100000:2.0:1788619916.881889:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 786 00000100:00100000:2.0:1788619916.881899:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571460352:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619916.881913:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619916.881929:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c4e5601d/0x1cd7fe5368734c56 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c059 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619916.881977:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78cdcc3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589120000:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619916.882051:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc240 going to sleep for 11 seconds 00000100:00100000:1.0:1788619916.884729:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501d500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589119872:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619916.884754:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 149756 00010000:00010000:1.0:1788619916.884773:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753000/0x1cd7fe5368734c4f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c052 expref: 18 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.884805:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619916.884808:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619916.884814:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753000/0x1cd7fe5368734c4f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c052 expref: 18 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.884829:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5753000, rc 0) 00000100:00100000:1.0:1788619916.884878:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+17:2352:x1875503571460224:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5641us (5797us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619916.884886:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 983 00000100:00100000:2.0:1788619916.885205:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78cdcc3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589120000:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619916.885224:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c4e5601d/0x1cd7fe5368734c56 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c059 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.885260:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c4e5601d/0x1cd7fe5368734c56 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c059 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.885270:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c779b800, rc 0) 00000100:00100000:2.0:1788619916.885305:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571460352:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3408us (3623us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619916.885311:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 786 00000100:00100000:3.0:1788619916.890076:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619916.890122:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619916.890207:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571460480 00000100:00100000:1.0:1788619916.890216:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571460480 00000100:00100000:1.0:1788619916.890275:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 984 00000100:00100000:1.0:1788619916.890284:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6859880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+17:2353:x1875503571460480:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619916.890305:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619916.890387:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734c5d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c060 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.890428:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752c00/0x1cd7fe5368734c2c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c036 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619916.890472:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501d500 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589120128:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619916.890555:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a900 going to sleep for 11 seconds 00000100:00100000:0.0:1788619916.890643:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571460608 00000100:00100000:0.0:1788619916.890649:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571460608 00000100:00100000:0.0:1788619916.890690:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 787 00000100:00100000:0.0:1788619916.890698:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c685b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571460608:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619916.890715:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619916.890727:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000033d84a7d/0x1cd7fe5368734c64 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c067 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619916.890772:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b37480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589120256:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619916.890816:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78ce7f3a80 going to sleep for 11 seconds 00000100:00100000:1.0:1788619916.893552:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501d500 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589120128:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619916.893585:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 149756 00010000:00010000:1.0:1788619916.893606:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734c5d lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c060 expref: 18 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.893642:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619916.893645:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619916.893651:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734c5d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c060 expref: 18 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.893667:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752e00, rc 0) 00000100:00100000:1.0:1788619916.893736:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6859880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+17:2353:x1875503571460480:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3452us (3681us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619916.893745:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 984 00000100:00100000:3.0:1788619916.895400:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b37480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589120256:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619916.895442:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000033d84a7d/0x1cd7fe5368734c64 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c067 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619916.895466:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000033d84a7d/0x1cd7fe5368734c64 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c067 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619916.895480:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767b200, rc 0) 00000100:00100000:3.0:1788619916.895551:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c685b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571460608:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4854us (5428us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619916.895560:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 787 00000100:00100000:0.0:1788619916.898675:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619916.898717:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619916.898800:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571460736 00000100:00100000:1.0:1788619916.898810:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571460736 00000100:00100000:1.0:1788619916.898869:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 985 00000100:00100000:1.0:1788619916.898879:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b37480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+17:2353:x1875503571460736:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619916.898899:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619916.898949:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734c6b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c06e expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.898984:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752c00/0x1cd7fe5368734c2c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c036 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619916.899020:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501d500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589120384:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619916.899115:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a900 going to sleep for 11 seconds 00000100:00100000:3.0:1788619916.900728:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571460864 00000100:00100000:3.0:1788619916.900740:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571460864 00000100:00100000:3.0:1788619916.900865:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 788 00000100:00100000:3.0:1788619916.900876:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571460864:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619916.900891:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619916.900921:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe5368734c72 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c075 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619916.900974:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c685aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589120512:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619916.901048:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa240 going to sleep for 11 seconds 00000100:00100000:1.0:1788619916.903439:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501d500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589120384:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619916.903475:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 149756 00010000:00010000:1.0:1788619916.903498:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734c6b lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c06e expref: 18 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.903521:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619916.903525:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619916.903530:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734c6b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c06e expref: 18 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.903548:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752e00, rc 0) 00000100:00100000:1.0:1788619916.903620:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b37480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+17:2353:x1875503571460736:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4730us (4941us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619916.903630:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 985 00000100:00100000:3.0:1788619916.905683:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c685aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589120512:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619916.905736:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe5368734c72 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c075 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619916.905760:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe5368734c72 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c075 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619916.905772:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c200, rc 0) 00000100:00100000:3.0:1788619916.905835:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571460864:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4961us (7115us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619916.905843:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 788 00000100:00100000:0.0:1788619916.913230:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619916.913320:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571460992 00000100:00100000:1.0:1788619916.913475:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571460992 00000100:00100000:1.0:1788619916.913573:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 986 00000100:00100000:1.0:1788619916.913586:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c33480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:14908:x1875503571460992:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619916.913680:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619916.913724:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753000/0x1cd7fe5368734c79 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c07c expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.913826:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752400/0x1cd7fe5368734c80 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.913842:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752400/0x1cd7fe5368734c80 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.913854:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752400/0x1cd7fe5368734c80 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788619916.913883:0:5846:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1566350 00000004:00200000:1.0:1788619916.913887:0:5846:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788619916.914020:0:5846:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752400/0x1cd7fe5368734c80 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.914059:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753000/0x1cd7fe5368734c79 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c07c expref: 18 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.914080:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752400/0x1cd7fe5368734c80 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c07c expref: 17 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.914090:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619916.914094:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788619916.914098:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752400, rc 0) 00000100:00100000:1.0:1788619916.914208:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c33480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+17:14908:x1875503571460992:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 623us (987us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619916.914215:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 986 00000100:00100000:0.0:1788619916.918527:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619916.918597:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619916.918686:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571461248 00000100:00100000:2.0:1788619916.918697:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571461248 00000100:00100000:2.0:1788619916.918754:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 789 00000100:00100000:2.0:1788619916.918765:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571461248:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619916.918794:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619916.918813:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005b564562/0x1cd7fe5368734c87 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c08a expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619916.918876:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78cdcc3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589120640:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619916.918980:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc240 going to sleep for 11 seconds 00000100:00100000:2.0:1788619916.921389:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78cdcc3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589120640:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619916.921427:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005b564562/0x1cd7fe5368734c87 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c08a expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.921455:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005b564562/0x1cd7fe5368734c87 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c08a expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.921491:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c779ac00, rc 0) 00000100:00100000:2.0:1788619916.921544:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571461248:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2780us (2945us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619916.921552:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 789 00000100:00100000:1.0:1788619916.925502:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571461120 00000100:00100000:1.0:1788619916.925535:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571461120 00000100:00100000:1.0:1788619916.925661:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 987 00000100:00100000:1.0:1788619916.925674:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c30700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+17:2353:x1875503571461120:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619916.925695:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619916.925716:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734c8e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c083 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.925757:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752c00/0x1cd7fe5368734c2c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c036 expref: 16 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619916.925810:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501d500 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589120768:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619916.925911:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a900 going to sleep for 11 seconds 00000100:00100000:1.0:1788619916.928593:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501d500 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589120768:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619916.928631:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 149756 00010000:00010000:1.0:1788619916.928650:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734c8e lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c083 expref: 18 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.928672:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619916.928675:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619916.928681:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734c8e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c083 expref: 18 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.928697:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752e00, rc 0) 00000100:00100000:1.0:1788619916.928756:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c30700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+17:2353:x1875503571461120:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3084us (10237us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619916.928766:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 987 00000100:00100000:3.0:1788619916.931357:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619916.931407:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619916.931439:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571461504 00000100:00100000:3.0:1788619916.931448:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571461504 00000100:00100000:3.0:1788619916.931505:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 790 00000100:00100000:3.0:1788619916.931516:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571461504:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619916.931533:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619916.931551:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe5368734c95 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c098 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619916.931618:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c71f2680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589120896:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619916.931689:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa240 going to sleep for 11 seconds 00000100:00100000:2.0:1788619916.932131:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571461376 00000100:00100000:2.0:1788619916.932138:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571461376 00000100:00100000:2.0:1788619916.932182:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 988 00000100:00100000:2.0:1788619916.932190:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c685aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+17:2353:x1875503571461376:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619916.932202:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619916.932218:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779ba00/0x1cd7fe5368734c9c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c091 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.932248:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752c00/0x1cd7fe5368734c2c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c036 expref: 16 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619916.932274:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78cdcc3480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589121024:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619916.932311:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc240 going to sleep for 11 seconds 00000100:00100000:1.0:1788619916.936612:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78cdcc3480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589121024:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619916.936655:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 149756 00010000:00010000:1.0:1788619916.936676:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779ba00/0x1cd7fe5368734c9c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c091 expref: 18 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.936698:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619916.936701:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619916.936707:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779ba00/0x1cd7fe5368734c9c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c091 expref: 18 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.936722:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c779ba00, rc 0) 00000100:00100000:1.0:1788619916.936799:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c685aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+17:2353:x1875503571461376:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4597us (5474us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619916.936809:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 988 00000100:00100000:2.0:1788619916.936951:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c71f2680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589120896:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619916.936974:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe5368734c95 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c098 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.936993:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe5368734c95 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c098 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619916.937016:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683dc00, rc 0) 00000100:00100000:2.0:1788619916.937043:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571461504:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5529us (5635us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619916.937050:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 790 00000100:00100000:2.0:1788619916.941475:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619916.941534:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571461760 00000100:00100000:1.0:1788619916.941543:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571461760 00000100:00100000:1.0:1788619916.941588:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 989 00000100:00100000:1.0:1788619916.941596:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+17:14908:x1875503571461760:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:1.0:1788619916.941648:0:5846:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619916.941671:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752400/0x1cd7fe5368734c80 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c07c expref: 17 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.941689:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619916.941695:0:5846:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788619916.941914:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734ca3 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.941939:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779a600/0x1cd7fe5368734c3a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c03d expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.941959:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734ca3 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.941989:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779a600/0x1cd7fe5368734c3a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c03d expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.942003:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779a600/0x1cd7fe5368734c3a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c03d expref: 16 pid: 9417 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619916.942018:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501d500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589121152:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619916.942098:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a900 going to sleep for 11 seconds 00000100:00100000:1.0:1788619916.944432:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501d500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589121152:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619916.944466:0:5846:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734ca3 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619916.945372:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619916.945447:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571461888 00000100:00100000:3.0:1788619916.945456:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571461888 00010000:00010000:3.0:1788619916.945494:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779a600/0x1cd7fe5368734c3a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c03d expref: 17 pid: 9417 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619916.945545:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 331 00000100:00100000:3.0:1788619916.945554:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc0380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+17:2354:x1875503571461888:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619916.945567:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619916.945572:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779a600/0x1cd7fe5368734c3a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c03d expref: 17 pid: 9417 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619916.945594:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779a600/0x1cd7fe5368734c3a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c03d expref: 17 pid: 9417 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619916.945622:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779a600/0x1cd7fe5368734c3a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c03d expref: 17 pid: 9417 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619916.945641:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619916.945646:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734ca3 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619916.945658:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734ca3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619916.945666:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734ca3 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619916.945698:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619916.945770:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc0380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+16:2354:x1875503571461888:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 215us (402us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619916.945777:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 331 00010000:00010000:1.0:1788619916.945831:0:5846:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734ca3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.945841:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734ca3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788619916.945987:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967670 00000004:00010000:1.0:1788619916.946052:0:5846:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 149756 -> 149756 00000004:00010000:1.0:1788619916.946056:0:5846:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619916 -> 1788619916 00000004:00010000:1.0:1788619916.946059:0:5846:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619916 -> 1788619914 00000004:00010000:1.0:1788619916.946061:0:5846:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619916 -> 1788619916 00000004:00010000:1.0:1788619916.946064:0:5846:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 902 -> 902 00000004:00010000:1.0:1788619916.946079:0:5846:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000b8e7eee1 reply state 000000007375f346 transno 4294967671 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734ca3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619916.946089:0:5846:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788619916.946175:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+19:14908:x1875503571461760:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 4578us (4708us total) trans 4294967671 rc 0/0 00000100:00100000:1.0:1788619916.946184:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 989 00000020:00080000:1.0:1788619916.946190:0:5846:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:3.0:1788619916.947747:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734ca3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619916.947793:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734ca3 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619916.947800:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734ca3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619916.947806:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c5752e00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:3.0:1788619916.947811:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734ca3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619916.947819:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734ca3 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619916.947833:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619916.947836:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788619916.947841:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734ca3 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619916.947847:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734ca3 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619916.950605:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619916.950696:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619916.950800:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571462144 00000100:00100000:1.0:1788619916.950810:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571462144 00002000:00010000:1.0:1788619916.950855:0:6701:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:1.0:1788619916.950860:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78cdcc0700 with x1875503571462144 ext(0->18446744073709551615) 00002000:00010000:1.0:1788619916.950871:0:6701:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78cdcc0700. 00000100:00100000:1.0:1788619916.950908:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 433 00000100:00100000:1.0:1788619916.950918:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571462144:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:1.0:1788619916.951244:0:6701:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1566350 -> 0 00002000:00010000:1.0:1788619916.951248:0:6701:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619916 -> 1788619916 00002000:00010000:1.0:1788619916.951250:0:6701:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619916 -> 1788619916 00002000:00010000:1.0:1788619916.951253:0:6701:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619916 -> 1788619916 00002000:00010000:1.0:1788619916.951261:0:6701:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 391 -> 391 00000100:00100000:1.0:1788619916.951319:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2352:x1875503571462144:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 402us (621us total) trans 4294967423 rc 0/0 00000100:00100000:1.0:1788619916.951355:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 433 00002000:00010000:1.0:1788619916.951359:0:6701:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:1.0:1788619916.951362:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78cdcc0700 with x1875503571462144 ext(0->18446744073709551615) 00002000:00010000:1.0:1788619916.951367:0:6701:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78cdcc0700. 00000100:00100000:0.0:1788619916.952475:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571462016 00000100:00100000:0.0:1788619916.952485:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571462016 00002000:00010000:0.0:1788619916.952518:0:5857:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:0.0:1788619916.952523:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78cdcc0000 with x1875503571462016 00000100:00100000:0.0:1788619916.952597:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 605 00000100:00100000:0.0:1788619916.952606:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2353:x1875503571462016:12345-192.168.201.5@tcp:10:fsx.0 00000004:00010000:0.0:1788619916.952764:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 149756 -> 316080 00000004:00010000:0.0:1788619916.952768:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619916 -> 1788619916 00000004:00010000:0.0:1788619916.952770:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619914 -> 1788619914 00000004:00010000:0.0:1788619916.952773:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619916 -> 1788619916 00000004:00010000:0.0:1788619916.952775:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 902 -> 902 00000100:00100000:0.0:1788619916.952850:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2353:x1875503571462016:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 245us (2254us total) trans 4294967672 rc 0/0 00000100:00100000:0.0:1788619916.952858:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 605 00002000:00010000:0.0:1788619916.952861:0:5857:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:0.0:1788619916.952876:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78cdcc0000 with x1875503571462016 00000100:00100000:2.0:1788619916.956257:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619916.956311:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571462272 00000100:00100000:3.0:1788619916.956319:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571462272 00000100:00100000:3.0:1788619916.956392:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 990 00000100:00100000:3.0:1788619916.956403:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+19:14908:x1875503571462272:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619916.956424:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619916.956441:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368734caa lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c09f expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619916.956520:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368734cb1 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619916.956548:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368734cb1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619916.956559:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368734cb1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788619916.956581:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 316080 00000004:00200000:3.0:1788619916.956585:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788619916.956600:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368734cb1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619916.956620:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368734caa lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c09f expref: 21 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619916.956633:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368734cb1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c09f expref: 20 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619916.956639:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619916.956641:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788619916.956643:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683de00, rc 0) 00000100:00100000:3.0:1788619916.956692:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+20:14908:x1875503571462272:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 293us (443us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619916.956697:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 990 00000100:00100000:3.0:1788619916.959039:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619916.959679:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571462528 00000100:00100000:1.0:1788619916.959703:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571462528 00000100:00100000:1.0:1788619916.959753:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 991 00000100:00100000:1.0:1788619916.959763:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+20:14908:x1875503571462528:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:1.0:1788619916.959828:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619916.959854:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368734cb1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c09f expref: 20 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.959875:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619916.959880:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788619916.960257:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752400/0x1cd7fe5368734cb8 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.960273:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752400/0x1cd7fe5368734cb8 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619916.960282:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752400/0x1cd7fe5368734cb8 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788619916.960473:0:5845:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967673 00000004:00010000:1.0:1788619916.960502:0:5845:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 316080 -> 316080 00000004:00010000:1.0:1788619916.960506:0:5845:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619916 -> 1788619916 00000004:00010000:1.0:1788619916.960509:0:5845:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619914 -> 1788619914 00000004:00010000:1.0:1788619916.960511:0:5845:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619916 -> 1788619916 00000004:00010000:1.0:1788619916.960513:0:5845:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 902 -> 902 00000004:00010000:1.0:1788619916.960521:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000ff0ad605 reply state 0000000002cf6641 transno 4294967674 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752400/0x1cd7fe5368734cb8 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619916.960530:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788619916.960733:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+22:14908:x1875503571462528:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 970us (1701us total) trans 4294967674 rc 0/0 00000100:00100000:1.0:1788619916.960749:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 991 00000020:00080000:1.0:1788619916.960755:0:5845:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788619916.962522:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752400/0x1cd7fe5368734cb8 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.962538:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752400/0x1cd7fe5368734cb8 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.962545:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752400/0x1cd7fe5368734cb8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.962551:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c5752400 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788619916.962557:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752400/0x1cd7fe5368734cb8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.962565:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752400/0x1cd7fe5368734cb8 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.962581:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619916.962584:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788619916.962590:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752400/0x1cd7fe5368734cb8 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619916.962596:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752400/0x1cd7fe5368734cb8 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619916.965090:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619916.965133:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619916.965465:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571462784 00000100:00100000:1.0:1788619916.965475:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571462784 00002000:00010000:1.0:1788619916.965521:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:1.0:1788619916.965526:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c71f1180 with x1875503571462784 ext(0->18446744073709551615) 00002000:00010000:1.0:1788619916.965539:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c71f1180. 00000100:00100000:1.0:1788619916.965568:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 434 00000100:00100000:1.0:1788619916.965579:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2352:x1875503571462784:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:1.0:1788619916.965807:0:6702:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:1.0:1788619916.965811:0:6702:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619916 -> 1788619916 00002000:00010000:1.0:1788619916.965814:0:6702:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619916 -> 1788619916 00002000:00010000:1.0:1788619916.965816:0:6702:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619916 -> 1788619916 00002000:00010000:1.0:1788619916.965819:0:6702:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 391 -> 391 00000100:00100000:1.0:1788619916.965870:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2352:x1875503571462784:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 292us (789us total) trans 4294967424 rc 0/0 00000100:00100000:1.0:1788619916.965879:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 434 00002000:00010000:1.0:1788619916.965882:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:1.0:1788619916.965885:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c71f1180 with x1875503571462784 ext(0->18446744073709551615) 00002000:00010000:1.0:1788619916.965890:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c71f1180. 00000100:00100000:0.0:1788619916.966300:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571462656 00000100:00100000:0.0:1788619916.966306:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571462656 00002000:00010000:0.0:1788619916.966363:0:5859:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:0.0:1788619916.966368:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c71f2300 with x1875503571462656 00000100:00100000:0.0:1788619916.966394:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 606 00000100:00100000:0.0:1788619916.966402:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2353:x1875503571462656:12345-192.168.201.5@tcp:10:fsx.0 00000004:00010000:0.0:1788619916.966522:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 316080 -> 1032771 00000004:00010000:0.0:1788619916.966539:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619916 -> 1788619916 00000004:00010000:0.0:1788619916.966542:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619914 -> 1788619914 00000004:00010000:0.0:1788619916.966544:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619916 -> 1788619916 00000004:00010000:0.0:1788619916.966546:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 902 -> 902 00000100:00100000:0.0:1788619916.966597:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2353:x1875503571462656:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 195us (1462us total) trans 4294967675 rc 0/0 00000100:00100000:0.0:1788619916.966604:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 606 00002000:00010000:0.0:1788619916.966607:0:5859:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:0.0:1788619916.966609:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c71f2300 with x1875503571462656 00000100:00100000:3.0:1788619916.969581:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619916.969706:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571462912 00000100:00100000:0.0:1788619916.969719:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571462912 00002000:00100000:0.0:1788619916.969761:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c71f0000 x1875503571462912/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/0 e 0 to 0 dl 1788619927 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619916.969774:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c71f0000 with x1875503571462912 00000100:00100000:0.0:1788619916.969819:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 607 00000100:00100000:0.0:1788619916.969827:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2354:x1875503571462912:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619916.970229:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2354:x1875503571462912:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 402us (658us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619916.970235:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 607 00002000:00100000:0.0:1788619916.970241:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c71f0000 x1875503571462912/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/4536 e 0 to 0 dl 1788619927 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619916.970246:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c71f0000 with x1875503571462912 00000100:00100000:3.0:1788619916.974022:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619916.974217:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571463040 00000100:00100000:1.0:1788619916.974228:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571463040 00002000:00100000:1.0:1788619916.974280:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c71f0380 x1875503571463040/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/0 e 0 to 0 dl 1788619927 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619916.974291:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c71f0380 with x1875503571463040 00000100:00100000:1.0:1788619916.974416:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 608 00000100:00100000:1.0:1788619916.974427:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2355:x1875503571463040:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619916.974702:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2355:x1875503571463040:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 276us (689us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619916.974710:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 608 00002000:00100000:1.0:1788619916.974716:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c71f0380 x1875503571463040/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/4536 e 0 to 0 dl 1788619927 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619916.974724:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c71f0380 with x1875503571463040 00000100:00100000:3.0:1788619916.978173:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619916.978280:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571463168 00000100:00100000:0.0:1788619916.978291:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571463168 00002000:00100000:0.0:1788619916.978376:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c71f0e00 x1875503571463168/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/0 e 0 to 0 dl 1788619927 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619916.978409:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c71f0e00 with x1875503571463168 00000100:00100000:0.0:1788619916.978444:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 609 00000100:00100000:0.0:1788619916.978454:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2353:x1875503571463168:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619916.978751:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2353:x1875503571463168:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 298us (587us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619916.978758:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 609 00002000:00100000:0.0:1788619916.978765:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c71f0e00 x1875503571463168/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/4536 e 0 to 0 dl 1788619927 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619916.978772:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c71f0e00 with x1875503571463168 00000100:00100000:3.0:1788619916.982152:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619916.982364:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571463296 00000100:00100000:1.0:1788619916.982374:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571463296 00002000:00100000:1.0:1788619916.982424:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c71f1880 x1875503571463296/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/0 e 0 to 0 dl 1788619927 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619916.982435:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c71f1880 with x1875503571463296 00000100:00100000:1.0:1788619916.982475:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 610 00000100:00100000:1.0:1788619916.982485:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2355:x1875503571463296:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619916.982735:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2355:x1875503571463296:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 249us (592us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619916.982742:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 610 00002000:00100000:1.0:1788619916.982750:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c71f1880 x1875503571463296/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/4536 e 0 to 0 dl 1788619927 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619916.982758:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c71f1880 with x1875503571463296 00000100:00100000:3.0:1788619916.988237:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619916.989725:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571463424 00000100:00100000:1.0:1788619916.989738:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571463424 00002000:00100000:1.0:1788619916.989803:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c71f0700 x1875503571463424/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/0 e 0 to 0 dl 1788619927 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619916.989842:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c71f0700 with x1875503571463424 00000100:00100000:1.0:1788619916.989898:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 611 00000100:00100000:1.0:1788619916.989908:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2354:x1875503571463424:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619916.990192:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2354:x1875503571463424:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 284us (1963us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619916.990201:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 611 00002000:00100000:1.0:1788619916.990209:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c71f0700 x1875503571463424/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/4536 e 0 to 0 dl 1788619927 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619916.990216:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c71f0700 with x1875503571463424 00000100:00100000:3.0:1788619916.999918:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619917.000194:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571463552 00000100:00100000:0.0:1788619917.000205:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571463552 00002000:00100000:0.0:1788619917.000259:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c71f3480 x1875503571463552/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/0 e 0 to 0 dl 1788619927 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619917.000271:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c71f3480 with x1875503571463552 00000100:00100000:0.0:1788619917.000310:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 612 00000100:00100000:0.0:1788619917.000321:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2355:x1875503571463552:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619917.000655:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2355:x1875503571463552:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 334us (748us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619917.000663:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 612 00002000:00100000:0.0:1788619917.000670:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c71f3480 x1875503571463552/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:12/0 lens 488/4536 e 0 to 0 dl 1788619927 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619917.000678:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c71f3480 with x1875503571463552 00000100:00100000:3.0:1788619917.003760:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619917.008023:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571463680 00000100:00100000:2.0:1788619917.008040:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571463680 00002000:00100000:2.0:1788619917.008495:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c71f2680 x1875503571463680/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:13/0 lens 488/0 e 0 to 0 dl 1788619928 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619917.008509:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c71f2680 with x1875503571463680 00000100:00100000:2.0:1788619917.008560:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 613 00000100:00100000:2.0:1788619917.008571:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2355:x1875503571463680:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:1.0:1788619917.011510:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1032771 -> 1032771 00000004:00010000:1.0:1788619917.011522:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619916 -> 1788619916 00000004:00010000:1.0:1788619917.011524:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619914 -> 1788619914 00000004:00010000:1.0:1788619917.011527:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619916 -> 1788619916 00000004:00010000:1.0:1788619917.011529:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 902 -> 902 00000100:00100000:1.0:1788619917.011630:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2355:x1875503571463680:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 3055us (7873us total) trans 4294967676 rc 0/0 00000100:00100000:1.0:1788619917.011642:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 613 00002000:00100000:1.0:1788619917.011659:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c71f2680 x1875503571463680/t4294967676(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:13/0 lens 488/448 e 0 to 0 dl 1788619928 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619917.011670:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c71f2680 with x1875503571463680 00000100:00100000:3.0:1788619917.014894:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619917.015558:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571463808 00000100:00100000:1.0:1788619917.015568:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571463808 00000100:00100000:1.0:1788619917.015629:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 992 00000100:00100000:1.0:1788619917.015639:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5019c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2353:x1875503571463808:12345-192.168.201.5@tcp:16:fsx.0 00000001:00080000:0.0:1788619917.052520:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967675 is committed 00000001:00080000:0.0:1788619917.052554:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967664 is committed 00000001:00080000:0.0:1788619917.052577:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967676 is committed 00000001:00080000:0.0:1788619917.052596:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967667 is committed 00000100:00100000:3.0:1788619917.053016:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5019c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2353:x1875503571463808:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 37372us (38127us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619917.053029:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 992 00000100:00100000:3.0:1788619917.055755:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619917.055867:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571463936 00000100:00100000:1.0:1788619917.055876:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571463936 00000100:00100000:1.0:1788619917.055946:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 993 00000100:00100000:1.0:1788619917.055955:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+8:14908:x1875503571463936:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619917.055975:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619917.055997:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753000/0x1cd7fe5368734cbf lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c0a6 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.056209:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752600/0x1cd7fe5368734cc6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.056225:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752600/0x1cd7fe5368734cc6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.056236:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752600/0x1cd7fe5368734cc6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788619917.056266:0:11397:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1032771 00000004:00200000:1.0:1788619917.056270:0:11397:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788619917.056287:0:11397:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752600/0x1cd7fe5368734cc6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.056311:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753000/0x1cd7fe5368734cbf lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c0a6 expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.056367:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752600/0x1cd7fe5368734cc6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c0a6 expref: 9 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.056377:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619917.056394:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788619917.056398:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752600, rc 0) 00000100:00100000:1.0:1788619917.056466:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571463936:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 511us (719us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619917.056472:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 993 00000100:00100000:2.0:1788619917.060959:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619917.061096:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571464064 00000100:00100000:1.0:1788619917.061106:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571464064 00000100:00100000:1.0:1788619917.061169:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 994 00000100:00100000:1.0:1788619917.061180:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571464064:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619917.061201:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619917.061224:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752a00/0x1cd7fe5368734ccd lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c0ad expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.061265:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752c00/0x1cd7fe5368734c2c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c036 expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619917.061308:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501d500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589121280:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619917.061433:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619917.063911:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619917.064255:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571464192 00000100:00100000:0.0:1788619917.064264:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571464192 00000100:00100000:0.0:1788619917.064316:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 791 00000100:00100000:0.0:1788619917.064358:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5018e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571464192:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619917.064375:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619917.064393:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003c6ffade/0x1cd7fe5368734cd4 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c0b4 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619917.064452:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c3100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589121408:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619917.064532:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78ce7f3540 going to sleep for 11 seconds 00000004:00010000:1.0:1788619917.066063:0:5845:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619914 -> 1788619916 00000100:00100000:1.0:1788619917.066096:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501d500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589121280:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619917.066120:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1032771 00010000:00010000:1.0:1788619917.066139:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752a00/0x1cd7fe5368734ccd lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c0ad expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.066161:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619917.066165:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619917.066171:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752a00/0x1cd7fe5368734ccd lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c0ad expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.066207:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752a00, rc 0) 00000100:00100000:1.0:1788619917.066266:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571464064:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5087us (5328us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619917.066275:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 994 00000100:00100000:0.0:1788619917.069212:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c3100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589121408:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619917.069255:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003c6ffade/0x1cd7fe5368734cd4 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c0b4 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619917.069279:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003c6ffade/0x1cd7fe5368734cd4 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c0b4 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619917.069293:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767b400, rc 0) 00000100:00100000:0.0:1788619917.069387:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5018e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571464192:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5060us (5483us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619917.069397:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 791 00000100:00100000:3.0:1788619917.078485:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619917.078667:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571464320 00000100:00100000:1.0:1788619917.078677:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571464320 00000100:00100000:1.0:1788619917.078742:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 995 00000100:00100000:1.0:1788619917.078752:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571464320:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619917.078774:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619917.078803:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752a00/0x1cd7fe5368734cdb lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c0bb expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.078842:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752c00/0x1cd7fe5368734c2c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c036 expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619917.078881:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501d500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589121536:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619917.079086:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619917.079835:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619917.082129:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571464448 00000100:00100000:0.0:1788619917.082144:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571464448 00000100:00100000:0.0:1788619917.082299:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 792 00000100:00100000:0.0:1788619917.082310:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5018380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571464448:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619917.082350:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619917.082369:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003c6ffade/0x1cd7fe5368734ce2 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c0c2 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619917.082426:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c3100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589121664:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619917.082518:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78ce7f3540 going to sleep for 11 seconds 00000100:00100000:1.0:1788619917.084552:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501d500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589121536:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619917.084579:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1032771 00010000:00010000:1.0:1788619917.084598:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752a00/0x1cd7fe5368734cdb lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c0bb expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.084618:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619917.084621:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619917.084627:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752a00/0x1cd7fe5368734cdb lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c0bb expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.084657:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752a00, rc 0) 00000100:00100000:1.0:1788619917.084721:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571464320:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5970us (6243us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619917.084730:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 995 00000100:00100000:0.0:1788619917.089562:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c3100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589121664:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619917.089606:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003c6ffade/0x1cd7fe5368734ce2 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c0c2 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619917.089631:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003c6ffade/0x1cd7fe5368734ce2 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c0c2 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619917.089644:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767b400, rc 0) 00000100:00100000:0.0:1788619917.089774:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5018380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571464448:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7464us (9939us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619917.089792:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 792 00010000:00010000:1.0:1788619917.091559:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788619917.101971:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619917.102031:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619917.102245:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571464576 00000100:00100000:2.0:1788619917.102256:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571464576 00000100:00100000:2.0:1788619917.102365:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 996 00000100:00100000:2.0:1788619917.102376:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571464576:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619917.102398:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619917.102421:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779a000/0x1cd7fe5368734ce9 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c0c9 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.102481:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752c00/0x1cd7fe5368734c2c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c036 expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619917.102524:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78cdcc3480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589121792:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619917.102611:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc840 going to sleep for 11 seconds 00000100:00100000:1.0:1788619917.103372:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571464704 00000100:00100000:1.0:1788619917.103379:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571464704 00000100:00100000:1.0:1788619917.103415:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 793 00000100:00100000:1.0:1788619917.103423:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571464704:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619917.103434:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619917.103446:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c03a8843/0x1cd7fe5368734cf0 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c0d0 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619917.103491:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501d500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589121920:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619917.103548:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a6c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619917.106554:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78cdcc3480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589121792:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619917.106585:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1032771 00010000:00010000:2.0:1788619917.106618:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779a000/0x1cd7fe5368734ce9 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c0c9 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.106642:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619917.106645:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619917.106652:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779a000/0x1cd7fe5368734ce9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c0c9 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.106669:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c779a000, rc 0) 00000100:00100000:2.0:1788619917.106734:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571464576:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4361us (4777us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619917.106744:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 996 00000100:00100000:1.0:1788619917.107857:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501d500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589121920:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619917.107893:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c03a8843/0x1cd7fe5368734cf0 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c0d0 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.107916:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c03a8843/0x1cd7fe5368734cf0 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c0d0 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.107927:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752a00, rc 0) 00000100:00100000:1.0:1788619917.107995:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571464704:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4571us (5962us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619917.108016:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 793 00000100:00100000:0.0:1788619917.112646:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619917.112695:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619917.112764:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571464960 00000100:00100000:2.0:1788619917.112775:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571464960 00000100:00100000:2.0:1788619917.112854:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 794 00000100:00100000:2.0:1788619917.112866:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571464960:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619917.112888:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619917.112909:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076679377/0x1cd7fe5368734cf7 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c0de expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619917.113007:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78cdcc3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589122048:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619917.113119:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc840 going to sleep for 11 seconds 00000100:00100000:1.0:1788619917.113237:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571464832 00000100:00100000:1.0:1788619917.113243:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571464832 00000100:00100000:1.0:1788619917.113287:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 997 00000100:00100000:1.0:1788619917.113295:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571464832:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619917.113306:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619917.113323:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753000/0x1cd7fe5368734cfe lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c0d7 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.113391:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752c00/0x1cd7fe5368734c2c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c036 expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619917.113417:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501d500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589122176:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619917.113452:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a6c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619917.121025:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78cdcc3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589122048:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619917.121082:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000076679377/0x1cd7fe5368734cf7 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c0de expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.121111:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076679377/0x1cd7fe5368734cf7 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c0de expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.121125:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c779a400, rc 0) 00000100:00100000:2.0:1788619917.121199:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571464960:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8334us (8502us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619917.121208:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 794 00000100:00100000:1.0:1788619917.121657:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501d500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589122176:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619917.121673:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1032771 00010000:00010000:1.0:1788619917.121685:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753000/0x1cd7fe5368734cfe lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c0d7 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.121699:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619917.121702:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619917.121707:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753000/0x1cd7fe5368734cfe lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c0d7 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.121719:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5753000, rc 0) 00000100:00100000:1.0:1788619917.121753:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571464832:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8460us (9120us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619917.121779:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 997 00000100:00100000:3.0:1788619917.127137:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619917.127361:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571465216 00000100:00100000:1.0:1788619917.127373:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571465216 00000100:00100000:1.0:1788619917.127434:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 998 00000100:00100000:1.0:1788619917.127445:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:14908:x1875503571465216:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:1.0:1788619917.127750:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752a00/0x1cd7fe5368734d05 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.127769:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752600/0x1cd7fe5368734cc6 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c0a6 expref: 8 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.127779:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752a00/0x1cd7fe5368734d05 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.127828:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752600/0x1cd7fe5368734cc6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c0a6 expref: 8 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.127842:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752600/0x1cd7fe5368734cc6 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c0a6 expref: 8 pid: 11397 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619917.127856:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501d500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589122304:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619917.127944:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619917.131248:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619917.131387:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501d500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589122304:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619917.131413:0:5846:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752a00/0x1cd7fe5368734d05 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619917.131495:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571465344 00000100:00100000:2.0:1788619917.131504:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571465344 00010000:00010000:2.0:1788619917.131540:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752600/0x1cd7fe5368734cc6 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c0a6 expref: 9 pid: 11397 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619917.131567:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 332 00000100:00100000:2.0:1788619917.131575:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5018700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571465344:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619917.131590:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619917.131596:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752600/0x1cd7fe5368734cc6 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c0a6 expref: 9 pid: 11397 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.131640:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752600/0x1cd7fe5368734cc6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c0a6 expref: 9 pid: 11397 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.131663:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752600/0x1cd7fe5368734cc6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c0a6 expref: 9 pid: 11397 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.131677:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619917.131682:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752a00/0x1cd7fe5368734d05 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.131691:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752a00/0x1cd7fe5368734d05 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.131698:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752a00/0x1cd7fe5368734d05 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.131726:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619917.131813:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5018700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+8:2355:x1875503571465344:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 208us (544us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619917.131821:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 332 00010000:00010000:1.0:1788619917.131878:0:5846:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752a00/0x1cd7fe5368734d05 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.131897:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752a00/0x1cd7fe5368734d05 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788619917.132100:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967677 00000004:00010000:1.0:1788619917.132159:0:5846:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1032771 -> 1032771 00000004:00010000:1.0:1788619917.132163:0:5846:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619916 -> 1788619917 00000004:00010000:1.0:1788619917.132166:0:5846:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619916 -> 1788619914 00000004:00010000:1.0:1788619917.132168:0:5846:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619916 -> 1788619917 00000004:00010000:1.0:1788619917.132169:0:5846:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 902 -> 518 00000004:00010000:1.0:1788619917.132182:0:5846:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000e2caa7ca reply state 000000001434451e transno 4294967678 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752a00/0x1cd7fe5368734d05 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619917.132191:0:5846:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788619917.132266:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:14908:x1875503571465216:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 4822us (5145us total) trans 4294967678 rc 0/0 00000100:00100000:1.0:1788619917.132275:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 998 00000020:00080000:1.0:1788619917.132279:0:5846:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1788619917.136632:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619917.136711:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571465472 00000100:00100000:2.0:1788619917.136721:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571465472 00002000:00010000:2.0:1788619917.136762:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (2089886->18446744073709551615) 00002000:00100000:2.0:1788619917.136768:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c50c3b80 with x1875503571465472 ext(2089886->18446744073709551615) 00002000:00010000:2.0:1788619917.136791:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c50c3b80. 00000100:00100000:2.0:1788619917.136823:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 435 00000100:00100000:2.0:1788619917.136834:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2352:x1875503571465472:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:2.0:1788619917.137094:0:6703:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 0 -> 2089886 00002000:00010000:2.0:1788619917.137098:0:6703:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619916 -> 1788619917 00002000:00010000:2.0:1788619917.137100:0:6703:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619916 -> 1788619916 00002000:00010000:2.0:1788619917.137103:0:6703:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619916 -> 1788619917 00002000:00010000:2.0:1788619917.137105:0:6703:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 391 -> 391 00000100:00100000:2.0:1788619917.137174:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2352:x1875503571465472:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 342us (553us total) trans 4294967425 rc 0/0 00000100:00100000:2.0:1788619917.137182:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 435 00002000:00010000:2.0:1788619917.137186:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (2089886->18446744073709551615) 00002000:00100000:2.0:1788619917.137189:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c50c3b80 with x1875503571465472 ext(2089886->18446744073709551615) 00002000:00010000:2.0:1788619917.137194:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c50c3b80. 00010000:00010000:3.0:1788619917.137383:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752a00/0x1cd7fe5368734d05 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.137393:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752a00/0x1cd7fe5368734d05 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.137402:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752a00/0x1cd7fe5368734d05 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.137408:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c5752a00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:3.0:1788619917.137413:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752a00/0x1cd7fe5368734d05 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.137422:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752a00/0x1cd7fe5368734d05 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.137448:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619917.137451:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788619917.137456:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752a00/0x1cd7fe5368734d05 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.137463:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752a00/0x1cd7fe5368734d05 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619917.145472:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619917.145586:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571465600 00000100:00100000:2.0:1788619917.145597:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571465600 00002000:00100000:2.0:1788619917.145661:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1568768->2093055) req@ffff8e77c5019880 x1875503571465600/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:13/0 lens 488/0 e 0 to 0 dl 1788619928 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619917.145673:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5019880 with x1875503571465600 ext(1568768->2093055) 00002000:00010000:2.0:1788619917.145679:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5019880 00000100:00100000:2.0:1788619917.145721:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 436 00000100:00100000:2.0:1788619917.145732:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5019880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2352:x1875503571465600:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619917.160191:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5019880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2352:x1875503571465600:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 14449us (14718us total) trans 0 rc 524288/524288 00000100:00100000:2.0:1788619917.160206:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 436 00002000:00100000:2.0:1788619917.160219:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1568768->2093055) req@ffff8e77c5019880 x1875503571465600/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:13/0 lens 488/440 e 0 to 0 dl 1788619928 ref 1 fl Complete:/600/0 rc 524288/524288 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619917.160255:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5019880 with x1875503571465600 ext(1568768->2093055) 00002000:00010000:2.0:1788619917.160261:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5019880 00000100:00100000:0.0:1788619917.164958:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619917.165260:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571465728 00000100:00100000:1.0:1788619917.165272:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571465728 00002000:00100000:1.0:1788619917.165321:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (2056192->2089885) req@ffff8e77c50c0a80 x1875503571465728/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:13/0 lens 488/0 e 0 to 0 dl 1788619928 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619917.165369:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c50c0a80 with x1875503571465728 ext(2056192->2089885) 00002000:00010000:1.0:1788619917.165375:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c50c0a80 00000100:00100000:1.0:1788619917.165412:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 437 00000100:00100000:1.0:1788619917.165422:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2353:x1875503571465728:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:1.0:1788619917.172612:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+11:2353:x1875503571465728:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 7180us (7662us total) trans 4294967426 rc 0/0 00000100:00100000:1.0:1788619917.172635:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 437 00002000:00100000:1.0:1788619917.172648:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (2056192->2089885) req@ffff8e77c50c0a80 x1875503571465728/t4294967426(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:13/0 lens 488/448 e 0 to 0 dl 1788619928 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619917.172657:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c50c0a80 with x1875503571465728 ext(2056192->2089885) 00002000:00010000:1.0:1788619917.172663:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c50c0a80 00000100:00100000:0.0:1788619917.181771:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619917.181868:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571465856 00000100:00100000:3.0:1788619917.181879:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571465856 00000100:00100000:3.0:1788619917.181940:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 795 00000100:00100000:3.0:1788619917.181959:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+11:2352:x1875503571465856:12345-192.168.201.5@tcp:16:fsx.0 00000100:00100000:3.0:1788619917.225253:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+11:2352:x1875503571465856:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 43293us (43480us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619917.225267:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 795 00000001:00080000:3.0:1788619917.225308:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967426 is committed 00000100:00100000:1.0:1788619917.230695:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619917.231624:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571465984 00000100:00100000:3.0:1788619917.231638:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571465984 00000100:00100000:3.0:1788619917.231703:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 999 00000100:00100000:3.0:1788619917.231713:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+8:14908:x1875503571465984:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619917.231735:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619917.231756:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368734d0c lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c0e5 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.231868:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368734d13 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.231906:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368734d13 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.231919:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368734d13 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788619917.231951:0:9417:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 2089886 00000004:00200000:3.0:1788619917.231956:0:9417:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788619917.231975:0:9417:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368734d13 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.232003:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368734d0c lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c0e5 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.232023:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368734d13 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c0e5 expref: 9 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.232031:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619917.232033:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788619917.232037:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683ce00, rc 0) 00000100:00100000:3.0:1788619917.232252:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571465984:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 539us (1565us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619917.232259:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 999 00000100:00100000:2.0:1788619917.236879:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619917.237020:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571466112 00000100:00100000:1.0:1788619917.237030:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571466112 00000100:00100000:1.0:1788619917.237101:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1000 00000100:00100000:1.0:1788619917.237111:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571466112:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619917.237132:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619917.237156:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753000/0x1cd7fe5368734d1a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c0ec expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.237196:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752c00/0x1cd7fe5368734c2c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c036 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619917.237238:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501ca80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589122432:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619917.237614:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a900 going to sleep for 11 seconds 00000100:00100000:0.0:1788619917.241835:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000004:00010000:1.0:1788619917.242494:0:5845:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619914 -> 1788619917 00000100:00100000:1.0:1788619917.242519:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501ca80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589122432:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619917.242543:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1032771 00010000:00010000:1.0:1788619917.242562:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753000/0x1cd7fe5368734d1a lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c0ec expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.242601:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619917.242663:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619917.242672:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753000/0x1cd7fe5368734d1a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c0ec expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.242686:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5753000, rc 0) 00000100:00100000:1.0:1788619917.242811:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571466112:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5701us (5942us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619917.242821:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1000 00000100:00100000:3.0:1788619917.242956:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571466240 00000100:00100000:3.0:1788619917.242964:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571466240 00000100:00100000:3.0:1788619917.243005:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 796 00000100:00100000:3.0:1788619917.243014:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571466240:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619917.243028:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619917.243061:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe5368734d21 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c0f3 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619917.243118:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589122560:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619917.243172:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaab40 going to sleep for 11 seconds 00002000:00010000:3.0:1788619917.247125:0:6697:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619916 -> 1788619917 00000100:00100000:3.0:1788619917.247165:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589122560:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619917.247204:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe5368734d21 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c0f3 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.247229:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe5368734d21 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c0f3 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.247243:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c200, rc 0) 00000100:00100000:3.0:1788619917.247380:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571466240:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4366us (5557us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619917.247388:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 796 00000100:00100000:2.0:1788619917.251864:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619917.251948:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619917.252016:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571466368 00000100:00100000:3.0:1788619917.252027:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571466368 00000100:00100000:3.0:1788619917.252095:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1001 00000100:00100000:3.0:1788619917.252106:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa40000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571466368:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619917.252126:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619917.252145:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368734d28 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c0fa expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.252182:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752c00/0x1cd7fe5368734c2c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c036 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619917.252236:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589122688:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619917.252320:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaab40 going to sleep for 11 seconds 00000100:00100000:3.0:1788619917.255273:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589122688:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619917.255301:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1032771 00010000:00010000:3.0:1788619917.255321:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368734d28 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c0fa expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.255368:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619917.255371:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619917.255377:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368734d28 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c0fa expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.255392:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c000, rc 0) 00000100:00100000:3.0:1788619917.255455:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa40000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571466368:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3350us (3505us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619917.255463:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1001 00000100:00100000:1.0:1788619917.257777:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571466496 00000100:00100000:1.0:1788619917.257802:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571466496 00000100:00100000:1.0:1788619917.257869:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 797 00000100:00100000:1.0:1788619917.257880:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571466496:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619917.257899:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619917.257926:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a9ce7747/0x1cd7fe5368734d2f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c101 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619917.257998:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589122816:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619917.258097:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a900 going to sleep for 11 seconds 00000100:00100000:3.0:1788619917.261226:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589122816:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619917.261265:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a9ce7747/0x1cd7fe5368734d2f lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c101 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.261289:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a9ce7747/0x1cd7fe5368734d2f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c101 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.261302:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5753000, rc 0) 00000100:00100000:3.0:1788619917.261400:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571466496:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3522us (9544us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619917.261409:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 797 00000100:00100000:0.0:1788619917.264090:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619917.264193:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571466624 00000100:00100000:1.0:1788619917.264203:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571466624 00000100:00100000:1.0:1788619917.264272:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 438 00000100:00100000:1.0:1788619917.264282:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571466624:12345-192.168.201.5@tcp:22:fsx.0 00002000:00010000:1.0:1788619917.264638:0:6701:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 2089886 -> 2089886 00002000:00010000:1.0:1788619917.264643:0:6701:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619917 -> 1788619917 00002000:00010000:1.0:1788619917.264657:0:6701:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619917 -> 1788619916 00002000:00010000:1.0:1788619917.264659:0:6701:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619917 -> 1788619917 00002000:00010000:1.0:1788619917.264662:0:6701:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 391 -> 135 00000100:00100000:1.0:1788619917.264709:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571466624:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 428us (630us total) trans 4294967427 rc 0/0 00000100:00100000:1.0:1788619917.264717:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 438 00000100:00100000:0.0:1788619917.267589:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619917.267671:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571466752 00000100:00100000:3.0:1788619917.267680:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571466752 00000100:00100000:3.0:1788619917.267739:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1002 00000100:00100000:3.0:1788619917.267749:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571466752:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619917.267773:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619917.267804:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368734d36 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c108 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.267891:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368734d3d lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.267917:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368734d3d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.267930:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368734d3d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788619917.267957:0:9417:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 2089886 00000004:00200000:3.0:1788619917.267961:0:9417:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788619917.267980:0:9417:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368734d3d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.268005:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368734d36 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c108 expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.268023:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368734d3d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c108 expref: 15 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.268197:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619917.268201:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788619917.268205:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c600, rc 0) 00000100:00100000:3.0:1788619917.268265:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:14908:x1875503571466752:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 517us (685us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619917.268272:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1002 00000100:00100000:0.0:1788619917.271313:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619917.271389:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619917.271458:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571467008 00000100:00100000:2.0:1788619917.271468:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571467008 00000100:00100000:2.0:1788619917.271521:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 798 00000100:00100000:2.0:1788619917.271532:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571467008:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619917.271549:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619917.271567:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab493e4e/0x1cd7fe5368734d44 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c116 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619917.271635:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78dfa42a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589122944:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619917.271716:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccd80 going to sleep for 11 seconds 00000100:00100000:1.0:1788619917.272659:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571466880 00000100:00100000:1.0:1788619917.272736:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571466880 00000100:00100000:1.0:1788619917.272944:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1003 00000100:00100000:1.0:1788619917.273001:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571466880:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619917.273017:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619917.273033:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752000/0x1cd7fe5368734d4b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c10f expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.273068:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752c00/0x1cd7fe5368734c2c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c036 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619917.273097:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501e300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589123072:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619917.273163:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a900 going to sleep for 11 seconds 00002000:00010000:2.0:1788619917.274715:0:7654:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619916 -> 1788619917 00000100:00100000:2.0:1788619917.274742:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78dfa42a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589122944:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619917.274775:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ab493e4e/0x1cd7fe5368734d44 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c116 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.274809:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab493e4e/0x1cd7fe5368734d44 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c116 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.274822:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c779ae00, rc 0) 00000100:00100000:2.0:1788619917.274892:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571467008:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3360us (3501us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619917.274900:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 798 00000100:00100000:1.0:1788619917.275906:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501e300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589123072:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619917.275927:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1032771 00010000:00010000:1.0:1788619917.275943:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752000/0x1cd7fe5368734d4b lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c10f expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.275960:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619917.275964:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619917.275981:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752000/0x1cd7fe5368734d4b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c10f expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.275994:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752000, rc 0) 00000100:00100000:1.0:1788619917.276045:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571466880:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3046us (4741us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619917.276616:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1003 00000100:00100000:0.0:1788619917.280086:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619917.280224:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571467136 00000100:00100000:3.0:1788619917.280234:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571467136 00000100:00100000:3.0:1788619917.280353:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1004 00000100:00100000:3.0:1788619917.280363:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571467136:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619917.280384:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619917.280404:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368734d52 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c11d expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.280440:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752c00/0x1cd7fe5368734c2c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c036 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619917.280490:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589123200:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619917.280576:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaab40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619917.282664:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619917.283377:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571467264 00000100:00100000:3.0:1788619917.283386:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571467264 00000100:00100000:3.0:1788619917.283442:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 799 00000100:00100000:3.0:1788619917.283461:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe13bc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571467264:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619917.283479:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619917.284123:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad29727f/0x1cd7fe5368734d59 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c124 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619917.284200:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5019f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589123328:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619917.284282:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa840 going to sleep for 11 seconds 00000100:00100000:2.0:1788619917.287173:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5019f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589123328:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619917.287217:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ad29727f/0x1cd7fe5368734d59 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c124 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.287243:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad29727f/0x1cd7fe5368734d59 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c124 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.287256:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c000, rc 0) 00000100:00100000:3.0:1788619917.287278:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589123200:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619917.287290:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1032771 00010000:00010000:3.0:1788619917.287302:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368734d52 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c11d expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619917.287315:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe13bc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571467264:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3864us (4660us total) trans 0 rc 301/301 00010000:00010000:3.0:1788619917.287316:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619917.287320:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00000100:00100000:2.0:1788619917.287339:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 799 00010000:00010000:3.0:1788619917.288582:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368734d52 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c11d expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.288657:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d000, rc 0) 00000100:00100000:3.0:1788619917.288710:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571467136:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8347us (8631us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619917.288719:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1004 00000100:00100000:1.0:1788619917.292242:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619917.292284:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619917.292352:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571467392 00000100:00100000:2.0:1788619917.292362:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571467392 00000100:00100000:3.0:1788619917.292441:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571467520 00000100:00100000:3.0:1788619917.292446:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571467520 00000100:00100000:2.0:1788619917.292462:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1005 00000100:00100000:2.0:1788619917.292472:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571467392:12345-192.168.201.5@tcp:101:fsx.0 00000100:00100000:3.0:1788619917.292491:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 800 00010000:00010000:2.0:1788619917.292492:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1788619917.292497:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571467520:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619917.292504:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619917.292515:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe5368734d67 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c132 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.292520:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779ae00/0x1cd7fe5368734d60 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c12b expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.292549:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752c00/0x1cd7fe5368734c2c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c036 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619917.292570:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78dfa43b80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589123584:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619917.292571:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589123456:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619917.292654:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccd80 going to sleep for 11 seconds 00000100:00100000:3.0:1788619917.292789:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaab40 going to sleep for 11 seconds 00000100:00100000:3.0:1788619917.295893:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78dfa43b80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589123584:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619917.295927:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1032771 00010000:00010000:3.0:1788619917.295948:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779ae00/0x1cd7fe5368734d60 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c12b expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.295969:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619917.295972:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619917.295978:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779ae00/0x1cd7fe5368734d60 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c12b expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.295993:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c779ae00, rc 0) 00000100:00100000:3.0:1788619917.296056:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571467392:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3584us (3769us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619917.296064:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1005 00000100:00100000:3.0:1788619917.298388:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589123456:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619917.298436:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe5368734d67 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c132 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.298550:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe5368734d67 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c132 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.298564:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c200, rc 0) 00000100:00100000:3.0:1788619917.298626:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571467520:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6128us (6393us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619917.298635:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 800 00000100:00100000:0.0:1788619917.302984:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619917.303074:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571467648 00000100:00100000:1.0:1788619917.303084:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571467648 00000100:00100000:1.0:1788619917.303182:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1006 00000100:00100000:1.0:1788619917.303193:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571467648:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619917.303215:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619917.303240:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752000/0x1cd7fe5368734d6e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c139 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.303271:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752c00/0x1cd7fe5368734c2c lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c036 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.303282:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752000/0x1cd7fe5368734d6e lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c139 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.303317:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752c00/0x1cd7fe5368734c2c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c036 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.303363:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752c00/0x1cd7fe5368734c2c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c036 expref: 14 pid: 5846 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619917.303399:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501c700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589123712:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619917.303628:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a900 going to sleep for 11 seconds 00000100:00100000:1.0:1788619917.307553:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501c700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589123712:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619917.307592:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752000/0x1cd7fe5368734d6e lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c139 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.307610:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619917.307613:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619917.307618:0:5845:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752000/0x1cd7fe5368734d6e lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c139 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.307629:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752000, rc 0) 00000100:00100000:1.0:1788619917.307820:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:14908:x1875503571467648:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4627us (4845us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619917.307829:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1006 00000100:00100000:0.0:1788619917.308432:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619917.308492:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571467776 00000100:00100000:3.0:1788619917.308500:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571467776 00010000:00010000:3.0:1788619917.308542:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752c00/0x1cd7fe5368734c2c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c036 expref: 15 pid: 5846 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619917.308574:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 333 00000100:00100000:3.0:1788619917.308584:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571467776:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619917.308596:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619917.308622:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752c00/0x1cd7fe5368734c2c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c036 expref: 15 pid: 5846 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.308636:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752c00/0x1cd7fe5368734c2c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c036 expref: 15 pid: 5846 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.308656:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752c00/0x1cd7fe5368734c2c lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c036 expref: 15 pid: 5846 timeout: 509 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.308786:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619917.308794:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752000/0x1cd7fe5368734d6e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c139 expref: 9 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.308805:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752000/0x1cd7fe5368734d6e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c139 expref: 9 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.308815:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752000/0x1cd7fe5368734d6e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c139 expref: 9 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.308853:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752000/0x1cd7fe5368734d6e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c139 expref: 9 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619917.308891:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501ad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589123840:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619917.311207:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaab40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619917.318268:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619917.318393:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501ad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589123840:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619917.318417:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619917.318485:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571467776:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 9902us (10055us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619917.318497:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 333 00000100:00100000:2.0:1788619917.319410:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571467904 00000100:00100000:2.0:1788619917.319422:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571467904 00002000:00100000:2.0:1788619917.319484:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c50c2d80 x1875503571467904/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/0 e 0 to 0 dl 1788619928 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619917.319495:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c50c2d80 with x1875503571467904 00000100:00100000:2.0:1788619917.319530:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 614 00000100:00100000:2.0:1788619917.319540:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571467904:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619917.320218:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571467904:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 677us (1958us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619917.320226:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 614 00002000:00100000:2.0:1788619917.320234:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c50c2d80 x1875503571467904/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/4536 e 0 to 0 dl 1788619928 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619917.320242:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c50c2d80 with x1875503571467904 00000100:00100000:0.0:1788619917.338360:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619917.338461:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571468032 00000100:00100000:2.0:1788619917.338474:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571468032 00002000:00100000:2.0:1788619917.338536:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c50c3480 x1875503571468032/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/0 e 0 to 0 dl 1788619928 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619917.338547:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c50c3480 with x1875503571468032 00000100:00100000:2.0:1788619917.338586:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 615 00000100:00100000:2.0:1788619917.338597:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571468032:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619917.338912:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571468032:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 316us (586us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619917.338919:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 615 00002000:00100000:2.0:1788619917.338926:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c50c3480 x1875503571468032/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/4536 e 0 to 0 dl 1788619928 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619917.338949:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c50c3480 with x1875503571468032 00000100:00100000:0.0:1788619917.345854:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619917.345939:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571468160 00000100:00100000:2.0:1788619917.345949:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571468160 00002000:00100000:2.0:1788619917.346000:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c50c2a00 x1875503571468160/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/0 e 0 to 0 dl 1788619928 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619917.346011:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c50c2a00 with x1875503571468160 00000100:00100000:2.0:1788619917.346057:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 616 00000100:00100000:2.0:1788619917.346067:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571468160:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619917.346320:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571468160:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 253us (476us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619917.346352:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 616 00002000:00100000:2.0:1788619917.346359:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c50c2a00 x1875503571468160/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/4536 e 0 to 0 dl 1788619928 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619917.346366:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c50c2a00 with x1875503571468160 00000100:00100000:3.0:1788619917.351739:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619917.351816:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571468288 00000100:00100000:0.0:1788619917.351827:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571468288 00002000:00100000:0.0:1788619917.351880:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c501ad80 x1875503571468288/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/0 e 0 to 0 dl 1788619928 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619917.351891:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c501ad80 with x1875503571468288 00000100:00100000:0.0:1788619917.351929:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 617 00000100:00100000:0.0:1788619917.351940:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571468288:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619917.352210:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571468288:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 272us (482us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619917.352217:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 617 00002000:00100000:0.0:1788619917.352223:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c501ad80 x1875503571468288/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/4536 e 0 to 0 dl 1788619928 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619917.352230:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c501ad80 with x1875503571468288 00000100:00100000:0.0:1788619917.371657:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619917.371762:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571468416 00000100:00100000:3.0:1788619917.371775:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571468416 00002000:00100000:3.0:1788619917.371840:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c50c0e00 x1875503571468416/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/0 e 0 to 0 dl 1788619928 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619917.371950:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c50c0e00 with x1875503571468416 00000100:00100000:3.0:1788619917.371996:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 618 00000100:00100000:3.0:1788619917.372006:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571468416:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619917.372279:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571468416:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 273us (633us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619917.372287:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 618 00002000:00100000:3.0:1788619917.372294:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c50c0e00 x1875503571468416/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/4536 e 0 to 0 dl 1788619928 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619917.372302:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c50c0e00 with x1875503571468416 00000100:00100000:1.0:1788619917.376220:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619917.376309:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571468544 00000100:00100000:2.0:1788619917.376319:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571468544 00002000:00100000:2.0:1788619917.376397:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c501c700 x1875503571468544/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/0 e 0 to 0 dl 1788619928 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619917.376408:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c501c700 with x1875503571468544 00000100:00100000:2.0:1788619917.376436:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 619 00000100:00100000:2.0:1788619917.376444:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571468544:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619917.376979:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571468544:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 533us (766us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619917.376987:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 619 00002000:00100000:2.0:1788619917.377009:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c501c700 x1875503571468544/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/4536 e 0 to 0 dl 1788619928 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619917.377016:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c501c700 with x1875503571468544 00000100:00100000:0.0:1788619917.386840:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619917.386959:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571468672 00000100:00100000:3.0:1788619917.386971:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571468672 00002000:00100000:3.0:1788619917.387036:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c50c2300 x1875503571468672/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/0 e 0 to 0 dl 1788619928 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619917.387058:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c50c2300 with x1875503571468672 00000100:00100000:3.0:1788619917.387093:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 620 00000100:00100000:3.0:1788619917.387103:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571468672:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619917.387408:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571468672:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 305us (580us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619917.387416:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 620 00002000:00100000:3.0:1788619917.387423:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c50c2300 x1875503571468672/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/4536 e 0 to 0 dl 1788619928 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619917.387431:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c50c2300 with x1875503571468672 00000100:00100000:0.0:1788619917.396919:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619917.397039:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571468800 00000100:00100000:2.0:1788619917.397058:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571468800 00002000:00100000:2.0:1788619917.397109:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c50c1c00 x1875503571468800/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/0 e 0 to 0 dl 1788619928 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619917.397120:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c50c1c00 with x1875503571468800 00000100:00100000:2.0:1788619917.397158:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 621 00000100:00100000:2.0:1788619917.397168:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571468800:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619917.397462:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571468800:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 295us (563us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619917.397472:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 621 00002000:00100000:2.0:1788619917.397479:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c50c1c00 x1875503571468800/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/4536 e 0 to 0 dl 1788619928 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619917.397485:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c50c1c00 with x1875503571468800 00000100:00100000:1.0:1788619917.408825:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619917.410579:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571468928 00000100:00100000:0.0:1788619917.410595:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571468928 00002000:00100000:0.0:1788619917.410660:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c501fb80 x1875503571468928/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/0 e 0 to 0 dl 1788619928 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619917.410672:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c501fb80 with x1875503571468928 00000100:00100000:0.0:1788619917.410707:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 622 00000100:00100000:0.0:1788619917.410716:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571468928:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619917.410981:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571468928:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 265us (2168us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619917.410989:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 622 00002000:00100000:0.0:1788619917.410996:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c501fb80 x1875503571468928/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/4536 e 0 to 0 dl 1788619928 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619917.411004:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c501fb80 with x1875503571468928 00000100:00100000:0.0:1788619917.426427:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619917.426504:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571469056 00000100:00100000:3.0:1788619917.426514:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571469056 00002000:00100000:3.0:1788619917.426563:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c50c4380 x1875503571469056/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/0 e 0 to 0 dl 1788619928 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619917.426574:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c50c4380 with x1875503571469056 00000100:00100000:3.0:1788619917.426616:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 623 00000100:00100000:3.0:1788619917.426624:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571469056:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619917.426897:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571469056:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 272us (479us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619917.426905:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 623 00002000:00100000:3.0:1788619917.426923:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c50c4380 x1875503571469056/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/4536 e 0 to 0 dl 1788619928 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619917.426930:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c50c4380 with x1875503571469056 00000100:00100000:1.0:1788619917.440488:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619917.440752:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571469184 00000100:00100000:2.0:1788619917.440762:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571469184 00002000:00100000:2.0:1788619917.440837:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c501f100 x1875503571469184/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/0 e 0 to 0 dl 1788619928 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619917.440848:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c501f100 with x1875503571469184 00000100:00100000:2.0:1788619917.440885:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 624 00000100:00100000:2.0:1788619917.440895:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571469184:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619917.441177:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571469184:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 282us (698us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619917.441201:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 624 00002000:00100000:2.0:1788619917.441208:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c501f100 x1875503571469184/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/4536 e 0 to 0 dl 1788619928 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619917.441215:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c501f100 with x1875503571469184 00000100:00100000:1.0:1788619917.459105:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619917.459264:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571469312 00000100:00100000:0.0:1788619917.459277:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571469312 00002000:00100000:0.0:1788619917.459378:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c501ca80 x1875503571469312/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/0 e 0 to 0 dl 1788619928 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619917.459391:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c501ca80 with x1875503571469312 00000100:00100000:0.0:1788619917.459440:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 625 00000100:00100000:0.0:1788619917.459452:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571469312:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619917.459810:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571469312:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 359us (714us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619917.459836:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 625 00002000:00100000:0.0:1788619917.459844:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c501ca80 x1875503571469312/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/4536 e 0 to 0 dl 1788619928 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619917.459852:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c501ca80 with x1875503571469312 00000100:00100000:0.0:1788619917.473622:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619917.473716:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571469440 00000100:00100000:3.0:1788619917.473726:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571469440 00002000:00100000:3.0:1788619917.473791:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c50c6d80 x1875503571469440/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/0 e 0 to 0 dl 1788619928 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619917.473803:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c50c6d80 with x1875503571469440 00000100:00100000:3.0:1788619917.473841:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 626 00000100:00100000:3.0:1788619917.473851:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571469440:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619917.474173:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571469440:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 321us (561us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619917.474181:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 626 00002000:00100000:3.0:1788619917.474189:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c50c6d80 x1875503571469440/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/4536 e 0 to 0 dl 1788619928 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619917.474197:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c50c6d80 with x1875503571469440 00000100:00100000:1.0:1788619917.477816:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619917.477904:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571469568 00000100:00100000:3.0:1788619917.477923:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571469568 00000100:00100000:3.0:1788619917.477981:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 801 00000100:00100000:3.0:1788619917.477991:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e6479880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571469568:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619917.478008:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619917.478026:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe5368734d75 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c140 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619917.478089:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5018a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589123968:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619917.478194:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:3.0:1788619917.480443:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5018a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589123968:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619917.480481:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe5368734d75 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c140 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.480521:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe5368734d75 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c140 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.480535:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c200, rc 0) 00000100:00100000:3.0:1788619917.480588:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e6479880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571469568:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2598us (2781us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619917.480596:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 801 00000100:00100000:0.0:1788619917.486409:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619917.486492:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571469696 00000100:00100000:2.0:1788619917.486502:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571469696 00000100:00100000:2.0:1788619917.486557:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 802 00000100:00100000:2.0:1788619917.486568:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571469696:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619917.486587:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619917.486617:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008901cab6/0x1cd7fe5368734d7c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c147 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619917.486675:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78dfa42a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589124096:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619917.486750:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc780 going to sleep for 11 seconds 00000100:00100000:2.0:1788619917.489003:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78dfa42a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589124096:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619917.489058:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008901cab6/0x1cd7fe5368734d7c lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c147 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.489083:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008901cab6/0x1cd7fe5368734d7c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c147 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.489096:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c779a600, rc 0) 00000100:00100000:2.0:1788619917.489158:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571469696:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2591us (2759us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619917.489185:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 802 00000100:00100000:1.0:1788619917.493289:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619917.493410:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571469952 00000100:00100000:3.0:1788619917.493421:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571469952 00000100:00100000:3.0:1788619917.493476:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1007 00000100:00100000:3.0:1788619917.493487:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e647aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571469952:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:3.0:1788619917.493541:0:5846:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619917.493572:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368734d3d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c108 expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.493590:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619917.493595:0:5846:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788619917.493819:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368734d83 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.493835:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368734d13 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c0e5 expref: 9 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.493843:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368734d83 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.493893:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368734d13 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c0e5 expref: 9 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.494273:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368734d13 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c0e5 expref: 9 pid: 9417 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619917.494292:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5018a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589124224:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619917.494572:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:0.0:1788619917.502090:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619917.502183:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571470080 00000100:00100000:2.0:1788619917.502193:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571470080 00010000:00010000:2.0:1788619917.502235:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368734d13 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c0e5 expref: 10 pid: 9417 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619917.502277:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 334 00000100:00100000:2.0:1788619917.502287:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c855b850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571470080:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619917.502301:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619917.502308:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368734d13 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c0e5 expref: 10 pid: 9417 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.502322:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368734d13 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c0e5 expref: 10 pid: 9417 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.502378:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619917.502384:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368734d83 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.502395:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368734d83 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.502402:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368734d83 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.502422:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619917.502472:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c855b850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571470080:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 187us (392us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619917.502500:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 334 00010000:00010000:3.0:1788619917.503946:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368734d13 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c0e5 expref: 9 pid: 9417 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619917.503968:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5018a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589124224:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619917.503991:0:5846:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368734d83 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.504001:0:5846:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368734d83 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.504007:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368734d83 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788619917.504135:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967679 00000004:00010000:3.0:1788619917.504184:0:5846:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1032771 -> 1032771 00000004:00010000:3.0:1788619917.504188:0:5846:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619917 -> 1788619917 00000004:00010000:3.0:1788619917.504191:0:5846:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619917 -> 1788619914 00000004:00010000:3.0:1788619917.504193:0:5846:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619917 -> 1788619917 00000004:00010000:3.0:1788619917.504196:0:5846:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 518 -> 518 00000004:00010000:3.0:1788619917.504214:0:5846:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000020bd4b85 reply state 000000002271b2fa transno 4294967680 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368734d83 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619917.504222:0:5846:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788619917.504450:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e647aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:14908:x1875503571469952:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 10963us (11170us total) trans 4294967680 rc 0/0 00000100:00100000:3.0:1788619917.504458:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1007 00000020:00080000:3.0:1788619917.504463:0:5846:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:2.0:1788619917.508305:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368734d83 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.508323:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368734d83 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.508358:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368734d83 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.508365:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c683c200 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:2.0:1788619917.508371:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368734d83 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.508379:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368734d83 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.508403:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619917.508407:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788619917.508413:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368734d83 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.508420:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368734d83 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619917.509369:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619917.509596:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571470208 00000100:00100000:3.0:1788619917.509612:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571470208 00000100:00100000:3.0:1788619917.509668:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1008 00000100:00100000:3.0:1788619917.509678:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e647b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:14908:x1875503571470208:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619917.509702:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619917.509721:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368734d8a lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c14e expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.509751:0:9417:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752000/0x1cd7fe5368734d6e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c139 expref: 8 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.509762:0:9417:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368734d8a lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c14e expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.509806:0:9417:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752000/0x1cd7fe5368734d6e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c139 expref: 8 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.509821:0:9417:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752000/0x1cd7fe5368734d6e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c139 expref: 8 pid: 5845 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619917.509836:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5018a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589124352:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619917.509940:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:0.0:1788619917.513420:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619917.513534:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5018a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589124352:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619917.513569:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368734d8a lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c14e expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.513588:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619917.513591:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619917.513595:0:9417:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368734d8a lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c14e expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.513617:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c600, rc 0) 00000100:00100000:3.0:1788619917.513665:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e647b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:14908:x1875503571470208:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3988us (4303us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619917.513674:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1008 00000100:00100000:1.0:1788619917.514359:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571470336 00000100:00100000:1.0:1788619917.514368:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571470336 00010000:00010000:1.0:1788619917.514404:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752000/0x1cd7fe5368734d6e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c139 expref: 9 pid: 5845 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619917.514457:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 335 00000100:00100000:1.0:1788619917.514466:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571470336:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619917.514480:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619917.514506:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752000/0x1cd7fe5368734d6e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c139 expref: 9 pid: 5845 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.514519:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752000/0x1cd7fe5368734d6e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c139 expref: 9 pid: 5845 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.514541:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752000/0x1cd7fe5368734d6e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c139 expref: 9 pid: 5845 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.514556:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619917.514562:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368734d8a lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c14e expref: 15 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.514572:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368734d8a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c14e expref: 15 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.514582:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368734d8a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c14e expref: 15 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.514626:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368734d8a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c14e expref: 15 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619917.514652:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e6478e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589124480:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619917.514717:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a900 going to sleep for 11 seconds 00000100:00100000:1.0:1788619917.518061:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e6478e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589124480:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619917.518087:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619917.518141:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+8:2354:x1875503571470336:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3675us (4730us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619917.518152:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 335 00000100:00100000:0.0:1788619917.520820:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619917.520860:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619917.520941:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571470592 00000100:00100000:1.0:1788619917.520951:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571470592 00002000:00010000:1.0:1788619917.520993:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:1.0:1788619917.520998:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c50c5500 with x1875503571470592 ext(0->18446744073709551615) 00002000:00010000:1.0:1788619917.521010:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c50c5500. 00000100:00100000:1.0:1788619917.521058:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 439 00000100:00100000:1.0:1788619917.521068:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2352:x1875503571470592:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:1.0:1788619917.521387:0:6702:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 2089886 -> 0 00002000:00010000:1.0:1788619917.521391:0:6702:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619917 -> 1788619917 00002000:00010000:1.0:1788619917.521394:0:6702:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619917 -> 1788619916 00002000:00010000:1.0:1788619917.521396:0:6702:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619917 -> 1788619917 00002000:00010000:1.0:1788619917.521398:0:6702:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 135 -> 135 00000100:00100000:1.0:1788619917.521449:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2352:x1875503571470592:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 383us (588us total) trans 4294967428 rc 0/0 00000100:00100000:1.0:1788619917.521457:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 439 00002000:00010000:1.0:1788619917.521461:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:1.0:1788619917.521464:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c50c5500 with x1875503571470592 ext(0->18446744073709551615) 00002000:00010000:1.0:1788619917.521484:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c50c5500. 00000100:00100000:2.0:1788619917.521708:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571470464 00000100:00100000:2.0:1788619917.521714:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571470464 00002000:00010000:2.0:1788619917.521736:0:5859:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:2.0:1788619917.521740:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c50c7b80 with x1875503571470464 00000100:00100000:2.0:1788619917.521767:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 627 00000100:00100000:2.0:1788619917.521783:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2352:x1875503571470464:12345-192.168.201.5@tcp:10:fsx.0 00000004:00010000:2.0:1788619917.521967:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1032771 -> 937952 00000004:00010000:2.0:1788619917.521971:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619917 -> 1788619917 00000004:00010000:2.0:1788619917.521974:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619914 -> 1788619914 00000004:00010000:2.0:1788619917.521976:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619917 -> 1788619917 00000004:00010000:2.0:1788619917.521978:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 518 -> 518 00000100:00100000:2.0:1788619917.522012:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2352:x1875503571470464:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 238us (1201us total) trans 4294967681 rc 0/0 00000100:00100000:2.0:1788619917.522020:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 627 00002000:00010000:2.0:1788619917.522023:0:5859:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:2.0:1788619917.522026:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c50c7b80 with x1875503571470464 00000100:00100000:0.0:1788619917.525898:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619917.526095:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571470720 00000100:00100000:2.0:1788619917.526106:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571470720 00000100:00100000:2.0:1788619917.526168:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1009 00000100:00100000:2.0:1788619917.526178:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+17:14908:x1875503571470720:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619917.526202:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619917.526221:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779a600/0x1cd7fe5368734d91 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c155 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.526310:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779bc00/0x1cd7fe5368734d98 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.526360:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779bc00/0x1cd7fe5368734d98 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.526374:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779bc00/0x1cd7fe5368734d98 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788619917.526405:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 937952 00000004:00200000:2.0:1788619917.526427:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788619917.526450:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779bc00/0x1cd7fe5368734d98 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.526477:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779a600/0x1cd7fe5368734d91 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c155 expref: 19 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.526495:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779bc00/0x1cd7fe5368734d98 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c155 expref: 18 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.526505:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619917.526509:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788619917.526512:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c779bc00, rc 0) 00000100:00100000:2.0:1788619917.526560:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+18:14908:x1875503571470720:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 384us (671us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619917.526568:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1009 00000100:00100000:0.0:1788619917.533973:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619917.534061:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571470848 00000100:00100000:2.0:1788619917.534072:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571470848 00000100:00100000:2.0:1788619917.534131:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 803 00000100:00100000:2.0:1788619917.534141:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:14908:x1875503571470848:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619917.534160:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619917.534176:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008901cab6/0x1cd7fe5368734d9f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c15c expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.534210:0:6696:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000072940acc/0x1cd7fe5368734c02 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1077248->1134591) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c01a expref: 7 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.534219:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=1s v=(3 12) 00010000:00010000:2.0:1788619917.534237:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(4 12) 00010000:00010000:2.0:1788619917.534243:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008901cab6/0x1cd7fe5368734d9f lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [2023424->2047999] (req 2023424->2047999) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c15c expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.534278:0:6696:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000072940acc/0x1cd7fe5368734c02 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1077248->1134591) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c01a expref: 7 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.534293:0:6696:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000072940acc/0x1cd7fe5368734c02 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1077248->1134591) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c01a expref: 7 pid: 6698 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619917.534307:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78dfa42a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589124608:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619917.534408:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc000 going to sleep for 11 seconds 00000100:00100000:0.0:1788619917.537674:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619917.537788:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571470976 00000100:00100000:1.0:1788619917.537798:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571470976 00010000:00010000:1.0:1788619917.537837:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000072940acc/0x1cd7fe5368734c02 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1077248->1134591) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c01a expref: 8 pid: 6698 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619917.537879:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 336 00000100:00100000:1.0:1788619917.537889:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571470976:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619917.538006:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619917.538035:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000072940acc/0x1cd7fe5368734c02 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1077248->1134591) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c01a expref: 8 pid: 6698 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.538059:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000072940acc/0x1cd7fe5368734c02 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1077248->1134591) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c01a expref: 8 pid: 6698 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.538085:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(4 12) 00010000:00010000:1.0:1788619917.538090:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(4 12) 00010000:00010000:1.0:1788619917.538100:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000008901cab6/0x1cd7fe5368734d9f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2023424->2047999) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c15c expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.538111:0:5831:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008901cab6/0x1cd7fe5368734d9f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2023424->2047999) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c15c expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.538154:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000008901cab6/0x1cd7fe5368734d9f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2023424->2047999) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c15c expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619917.538193:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e6478e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5831:1875503589124736:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619917.538288:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a900 going to sleep for 11 seconds 00010000:00010000:2.0:1788619917.538752:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000072940acc/0x1cd7fe5368734c02 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1077248->1134591) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c01a expref: 8 pid: 6698 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619917.538768:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78dfa42a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589124608:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619917.538801:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008901cab6/0x1cd7fe5368734d9f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2023424->2047999) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c15c expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.538813:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c779a600, rc 0) 00000100:00100000:2.0:1788619917.538859:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:14908:x1875503571470848:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4718us (4895us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619917.538866:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 803 00000100:00100000:1.0:1788619917.541300:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e6478e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5831:1875503589124736:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619917.541362:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619917.541403:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571470976:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3515us (3744us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619917.541415:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 336 00000100:00100000:0.0:1788619917.545161:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619917.545237:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571471104 00000100:00100000:3.0:1788619917.545247:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571471104 00000100:00100000:3.0:1788619917.545306:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1010 00000100:00100000:3.0:1788619917.545317:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+8:14908:x1875503571471104:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619917.545372:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619917.545393:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368734da6 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c163 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.545487:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368734dad lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.545503:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368734dad lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.545536:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368734dad lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788619917.545569:0:5846:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 937952 00000004:00200000:3.0:1788619917.545573:0:5846:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788619917.545592:0:5846:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368734dad lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.545631:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368734da6 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c163 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.545649:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368734dad lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c163 expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.545657:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619917.545660:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788619917.545663:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d000, rc 0) 00000100:00100000:3.0:1788619917.545717:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571471104:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 402us (566us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619917.545724:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1010 00000100:00100000:0.0:1788619917.548851:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619917.548933:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571471232 00000100:00100000:3.0:1788619917.548942:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571471232 00000100:00100000:3.0:1788619917.549003:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1011 00000100:00100000:3.0:1788619917.549012:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571471232:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619917.549032:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619917.549061:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368734db4 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c16a expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.549118:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368734d8a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c14e expref: 17 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619917.549155:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5018a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589124864:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619917.549237:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000004:00010000:3.0:1788619917.552241:0:9417:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619914 -> 1788619916 00000100:00100000:3.0:1788619917.552273:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5018a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589124864:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619917.552294:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 937952 00010000:00010000:3.0:1788619917.552312:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368734db4 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c16a expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.552362:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619917.552366:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619917.552372:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368734db4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c16a expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.552388:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683cc00, rc 0) 00000100:00100000:3.0:1788619917.552436:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571471232:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3425us (3594us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619917.552446:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1011 00000100:00100000:0.0:1788619917.555217:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619917.555289:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571471360 00000100:00100000:3.0:1788619917.555300:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571471360 00000100:00100000:3.0:1788619917.555383:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1012 00000100:00100000:3.0:1788619917.555394:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571471360:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619917.555414:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619917.555450:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368734dbb lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c171 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.555488:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368734d8a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c14e expref: 17 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619917.555539:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5018a80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589124992:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619917.555623:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:3.0:1788619917.558411:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5018a80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589124992:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619917.558436:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 937952 00010000:00010000:3.0:1788619917.558456:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368734dbb lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c171 expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.558477:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619917.558481:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619917.558487:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368734dbb lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c171 expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.558501:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683ce00, rc 0) 00000100:00100000:3.0:1788619917.558597:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571471360:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3204us (3387us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619917.558614:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1012 00000100:00100000:0.0:1788619917.562667:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619917.562786:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571471488 00000100:00100000:1.0:1788619917.562798:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571471488 00000100:00100000:1.0:1788619917.562859:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1013 00000100:00100000:1.0:1788619917.562870:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571471488:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619917.562891:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619917.562933:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752c00/0x1cd7fe5368734dc2 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c178 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.562972:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368734d8a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c14e expref: 17 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619917.563013:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e6478e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589125120:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619917.563201:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a900 going to sleep for 11 seconds 00000100:00100000:1.0:1788619917.566112:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e6478e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589125120:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619917.566138:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 937952 00010000:00010000:1.0:1788619917.566160:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752c00/0x1cd7fe5368734dc2 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c178 expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.566205:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619917.566208:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619917.566215:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752c00/0x1cd7fe5368734dc2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c178 expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.566231:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752c00, rc 0) 00000100:00100000:1.0:1788619917.566304:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571471488:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3434us (3646us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619917.566313:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1013 00000100:00100000:0.0:1788619917.569870:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619917.570238:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571471616 00000100:00100000:3.0:1788619917.570249:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571471616 00002000:00100000:3.0:1788619917.570304:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1585152->1589247) req@ffff8e77c50c5880 x1875503571471616/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/0 e 0 to 0 dl 1788619928 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619917.570316:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c50c5880 with x1875503571471616 ext(1585152->1589247) 00002000:00010000:3.0:1788619917.570377:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c50c5880 00000100:00100000:3.0:1788619917.570404:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 440 00000100:00100000:3.0:1788619917.570414:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571471616:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619917.570662:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571471616:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 248us (803us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619917.570670:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 440 00002000:00100000:3.0:1788619917.570678:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1585152->1589247) req@ffff8e77c50c5880 x1875503571471616/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/440 e 0 to 0 dl 1788619928 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619917.570686:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c50c5880 with x1875503571471616 ext(1585152->1589247) 00002000:00010000:3.0:1788619917.570689:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c50c5880 00000100:00100000:2.0:1788619917.574518:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619917.574679:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571471744 00000100:00100000:3.0:1788619917.574691:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571471744 00002000:00100000:3.0:1788619917.574745:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1589248->1593343) req@ffff8e78dfa42a00 x1875503571471744/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/0 e 0 to 0 dl 1788619928 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619917.574757:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa42a00 with x1875503571471744 ext(1589248->1593343) 00002000:00010000:3.0:1788619917.574763:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa42a00 00000100:00100000:3.0:1788619917.574805:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 441 00000100:00100000:3.0:1788619917.574815:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571471744:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619917.575113:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571471744:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 298us (605us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619917.575121:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 441 00002000:00100000:3.0:1788619917.575128:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1589248->1593343) req@ffff8e78dfa42a00 x1875503571471744/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/440 e 0 to 0 dl 1788619928 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619917.575137:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa42a00 with x1875503571471744 ext(1589248->1593343) 00002000:00010000:3.0:1788619917.575154:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa42a00 00000100:00100000:0.0:1788619917.578497:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619917.578606:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571471872 00000100:00100000:1.0:1788619917.578617:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571471872 00002000:00100000:1.0:1788619917.578662:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1593344->1597439) req@ffff8e78dfa14e00 x1875503571471872/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/0 e 0 to 0 dl 1788619928 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619917.578673:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa14e00 with x1875503571471872 ext(1593344->1597439) 00002000:00010000:1.0:1788619917.578678:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa14e00 00000100:00100000:1.0:1788619917.578713:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 442 00000100:00100000:1.0:1788619917.578723:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa14e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571471872:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619917.579007:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa14e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571471872:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 284us (519us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619917.579015:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 442 00002000:00100000:1.0:1788619917.579022:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1593344->1597439) req@ffff8e78dfa14e00 x1875503571471872/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/440 e 0 to 0 dl 1788619928 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619917.579030:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa14e00 with x1875503571471872 ext(1593344->1597439) 00002000:00010000:1.0:1788619917.579049:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa14e00 00000100:00100000:0.0:1788619917.582258:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619917.582363:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571472000 00000100:00100000:2.0:1788619917.582373:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571472000 00002000:00100000:2.0:1788619917.582438:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1597440->1601535) req@ffff8e78dfa15c00 x1875503571472000/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/0 e 0 to 0 dl 1788619928 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619917.582450:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa15c00 with x1875503571472000 ext(1597440->1601535) 00002000:00010000:2.0:1788619917.582455:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa15c00 00000100:00100000:2.0:1788619917.582490:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 443 00000100:00100000:2.0:1788619917.582499:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa15c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571472000:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619917.582761:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa15c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571472000:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 262us (515us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619917.582770:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 443 00002000:00100000:2.0:1788619917.582786:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1597440->1601535) req@ffff8e78dfa15c00 x1875503571472000/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/440 e 0 to 0 dl 1788619928 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619917.582794:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa15c00 with x1875503571472000 ext(1597440->1601535) 00002000:00010000:2.0:1788619917.582798:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa15c00 00000100:00100000:2.0:1788619917.586019:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619917.586167:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571472128 00000100:00100000:3.0:1788619917.586177:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571472128 00002000:00100000:3.0:1788619917.586224:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1601536->1605631) req@ffff8e78dfa41180 x1875503571472128/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/0 e 0 to 0 dl 1788619928 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619917.586236:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa41180 with x1875503571472128 ext(1601536->1605631) 00002000:00010000:3.0:1788619917.586241:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa41180 00000100:00100000:3.0:1788619917.586273:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 444 00000100:00100000:3.0:1788619917.586283:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa41180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571472128:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619917.586571:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa41180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571472128:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 288us (561us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619917.586579:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 444 00002000:00100000:3.0:1788619917.586586:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1601536->1605631) req@ffff8e78dfa41180 x1875503571472128/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/440 e 0 to 0 dl 1788619928 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619917.586594:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa41180 with x1875503571472128 ext(1601536->1605631) 00002000:00010000:3.0:1788619917.586598:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa41180 00000100:00100000:0.0:1788619917.589469:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619917.589572:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571472256 00000100:00100000:1.0:1788619917.589583:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571472256 00002000:00100000:1.0:1788619917.589644:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1605632->1609727) req@ffff8e78dfa14700 x1875503571472256/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/0 e 0 to 0 dl 1788619928 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619917.589656:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa14700 with x1875503571472256 ext(1605632->1609727) 00002000:00010000:1.0:1788619917.589662:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa14700 00000100:00100000:1.0:1788619917.589695:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 445 00000100:00100000:1.0:1788619917.589705:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa14700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571472256:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619917.589976:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa14700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571472256:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 270us (516us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619917.589984:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 445 00002000:00100000:1.0:1788619917.589990:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1605632->1609727) req@ffff8e78dfa14700 x1875503571472256/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/440 e 0 to 0 dl 1788619928 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619917.589998:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa14700 with x1875503571472256 ext(1605632->1609727) 00002000:00010000:1.0:1788619917.590002:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa14700 00000100:00100000:0.0:1788619917.593129:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619917.593273:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571472384 00000100:00100000:2.0:1788619917.593285:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571472384 00002000:00100000:2.0:1788619917.593376:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1609728->1613823) req@ffff8e78dfa16680 x1875503571472384/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/0 e 0 to 0 dl 1788619928 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619917.593388:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa16680 with x1875503571472384 ext(1609728->1613823) 00002000:00010000:2.0:1788619917.593395:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa16680 00000100:00100000:2.0:1788619917.593429:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 446 00000100:00100000:2.0:1788619917.593439:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa16680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571472384:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619917.593679:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa16680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571472384:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 240us (558us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619917.593704:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 446 00002000:00100000:2.0:1788619917.593712:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1609728->1613823) req@ffff8e78dfa16680 x1875503571472384/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/440 e 0 to 0 dl 1788619928 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619917.593721:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa16680 with x1875503571472384 ext(1609728->1613823) 00002000:00010000:2.0:1788619917.593725:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa16680 00000100:00100000:2.0:1788619917.596899:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619917.597057:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571472512 00000100:00100000:3.0:1788619917.597067:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571472512 00002000:00100000:3.0:1788619917.597114:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1613824->1617919) req@ffff8e78dfa40a80 x1875503571472512/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/0 e 0 to 0 dl 1788619928 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619917.597125:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa40a80 with x1875503571472512 ext(1613824->1617919) 00002000:00010000:3.0:1788619917.597131:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa40a80 00000100:00100000:3.0:1788619917.597163:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 447 00000100:00100000:3.0:1788619917.597173:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa40a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571472512:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619917.597463:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa40a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571472512:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 290us (572us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619917.597471:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 447 00002000:00100000:3.0:1788619917.597479:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1613824->1617919) req@ffff8e78dfa40a80 x1875503571472512/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/440 e 0 to 0 dl 1788619928 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619917.597488:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa40a80 with x1875503571472512 ext(1613824->1617919) 00002000:00010000:3.0:1788619917.597492:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa40a80 00000100:00100000:0.0:1788619917.600900:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619917.601071:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571472640 00000100:00100000:1.0:1788619917.601081:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571472640 00002000:00100000:1.0:1788619917.601130:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1617920->1622015) req@ffff8e78dfa15500 x1875503571472640/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/0 e 0 to 0 dl 1788619928 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619917.601141:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa15500 with x1875503571472640 ext(1617920->1622015) 00002000:00010000:1.0:1788619917.601147:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa15500 00000100:00100000:1.0:1788619917.601183:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 448 00000100:00100000:1.0:1788619917.601193:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa15500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571472640:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619917.601498:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa15500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571472640:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 305us (607us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619917.601505:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 448 00002000:00100000:1.0:1788619917.601512:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1617920->1622015) req@ffff8e78dfa15500 x1875503571472640/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/440 e 0 to 0 dl 1788619928 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619917.601520:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa15500 with x1875503571472640 ext(1617920->1622015) 00002000:00010000:1.0:1788619917.601523:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa15500 00000100:00100000:0.0:1788619917.604915:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619917.605077:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571472768 00000100:00100000:2.0:1788619917.605087:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571472768 00002000:00100000:2.0:1788619917.605145:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1622016->1626111) req@ffff8e78dfa14a80 x1875503571472768/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/0 e 0 to 0 dl 1788619928 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619917.605157:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa14a80 with x1875503571472768 ext(1622016->1626111) 00002000:00010000:2.0:1788619917.605164:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa14a80 00000100:00100000:2.0:1788619917.605201:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 449 00000100:00100000:2.0:1788619917.605211:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa14a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571472768:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619917.605486:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa14a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571472768:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 275us (591us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619917.605494:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 449 00002000:00100000:2.0:1788619917.605501:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1622016->1626111) req@ffff8e78dfa14a80 x1875503571472768/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/440 e 0 to 0 dl 1788619928 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619917.605510:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa14a80 with x1875503571472768 ext(1622016->1626111) 00002000:00010000:2.0:1788619917.605514:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa14a80 00000100:00100000:2.0:1788619917.609419:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619917.609499:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571472896 00000100:00100000:3.0:1788619917.609508:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571472896 00002000:00100000:3.0:1788619917.609557:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1626112->1630207) req@ffff8e78dfa41f80 x1875503571472896/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/0 e 0 to 0 dl 1788619928 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619917.609568:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa41f80 with x1875503571472896 ext(1626112->1630207) 00002000:00010000:3.0:1788619917.609574:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa41f80 00000100:00100000:3.0:1788619917.609606:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 450 00000100:00100000:3.0:1788619917.609615:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa41f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571472896:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619917.609859:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa41f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571472896:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 244us (449us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619917.609866:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 450 00002000:00100000:3.0:1788619917.609873:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1626112->1630207) req@ffff8e78dfa41f80 x1875503571472896/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/440 e 0 to 0 dl 1788619928 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619917.609881:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa41f80 with x1875503571472896 ext(1626112->1630207) 00002000:00010000:3.0:1788619917.609885:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa41f80 00000100:00100000:0.0:1788619917.612696:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619917.612803:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571473024 00000100:00100000:1.0:1788619917.612812:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571473024 00002000:00100000:1.0:1788619917.612863:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1630208->1634303) req@ffff8e78dfa17b80 x1875503571473024/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/0 e 0 to 0 dl 1788619928 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619917.612875:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa17b80 with x1875503571473024 ext(1630208->1634303) 00002000:00010000:1.0:1788619917.612880:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa17b80 00000100:00100000:1.0:1788619917.612957:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 451 00000100:00100000:1.0:1788619917.612968:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa17b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571473024:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619917.613369:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa17b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571473024:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 402us (680us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619917.613377:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 451 00002000:00100000:1.0:1788619917.613384:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1630208->1634303) req@ffff8e78dfa17b80 x1875503571473024/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/440 e 0 to 0 dl 1788619928 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619917.613393:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa17b80 with x1875503571473024 ext(1630208->1634303) 00002000:00010000:1.0:1788619917.613397:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa17b80 00000100:00100000:0.0:1788619917.618297:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619917.618530:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571473152 00000100:00100000:3.0:1788619917.618540:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571473152 00000100:00100000:3.0:1788619917.618618:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1014 00000100:00100000:3.0:1788619917.618628:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa16300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571473152:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619917.618652:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619917.618673:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368734dc9 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c17f expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.618711:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368734d8a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c14e expref: 17 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619917.618750:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5018a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589125248:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619917.618890:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:3.0:1788619917.621862:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5018a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589125248:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619917.621887:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 937952 00010000:00010000:3.0:1788619917.621972:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368734dc9 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c17f expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.621998:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619917.622001:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619917.622008:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368734dc9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c17f expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.622023:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683ce00, rc 0) 00000100:00100000:3.0:1788619917.622196:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa16300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571473152:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3568us (3909us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619917.622206:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1014 00000100:00100000:0.0:1788619917.625305:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619917.625417:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571473280 00000100:00100000:3.0:1788619917.625426:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571473280 00000100:00100000:3.0:1788619917.625556:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1015 00000100:00100000:3.0:1788619917.625567:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa15180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571473280:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619917.625587:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619917.625618:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368734dd0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c186 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.625657:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368734d8a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c14e expref: 17 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619917.625700:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5018a80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589125376:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619917.625793:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:3.0:1788619917.628182:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5018a80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589125376:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619917.628206:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 937952 00010000:00010000:3.0:1788619917.628222:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368734dd0 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c186 expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.628240:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619917.628244:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619917.628249:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368734dd0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c186 expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.628263:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683cc00, rc 0) 00000100:00100000:3.0:1788619917.628318:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa15180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571473280:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2753us (3022us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619917.628358:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1015 00000100:00100000:0.0:1788619917.633612:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619917.633790:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571473536 00000100:00100000:1.0:1788619917.633801:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571473536 00000100:00100000:1.0:1788619917.633871:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1016 00000100:00100000:1.0:1788619917.633881:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa17100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+18:14908:x1875503571473536:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:1.0:1788619917.633945:0:5844:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619917.633972:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779bc00/0x1cd7fe5368734d98 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c155 expref: 18 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.633991:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619917.633996:0:5844:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788619917.634371:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752c00/0x1cd7fe5368734dd7 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.634388:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368734dad lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c163 expref: 8 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.634398:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752c00/0x1cd7fe5368734dd7 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.634429:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368734dad lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c163 expref: 8 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.634444:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368734dad lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c163 expref: 8 pid: 5846 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619917.634458:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e6478e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589125504:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619917.634531:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a900 going to sleep for 11 seconds 00000100:00100000:0.0:1788619917.637726:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619917.637804:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571473664 00000100:00100000:3.0:1788619917.637814:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571473664 00010000:00010000:3.0:1788619917.637871:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368734dad lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c163 expref: 9 pid: 5846 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619917.637926:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 337 00000100:00100000:3.0:1788619917.637936:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa15880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571473664:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619917.637948:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619917.637954:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368734dad lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c163 expref: 9 pid: 5846 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.637968:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368734dad lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c163 expref: 9 pid: 5846 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.637990:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619917.637995:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752c00/0x1cd7fe5368734dd7 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.638005:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752c00/0x1cd7fe5368734dd7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.638012:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752c00/0x1cd7fe5368734dd7 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.638030:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619917.638093:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa15880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571473664:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 158us (375us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619917.638100:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 337 00010000:00010000:1.0:1788619917.638276:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368734dad lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c163 expref: 8 pid: 5846 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619917.638305:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e6478e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589125504:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619917.638352:0:5844:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752c00/0x1cd7fe5368734dd7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.638363:0:5844:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752c00/0x1cd7fe5368734dd7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.638369:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752c00/0x1cd7fe5368734dd7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788619917.638478:0:5844:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967682 00000004:00010000:1.0:1788619917.638529:0:5844:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 937952 -> 937952 00000004:00010000:1.0:1788619917.638533:0:5844:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619917 -> 1788619917 00000004:00010000:1.0:1788619917.638535:0:5844:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619916 -> 1788619914 00000004:00010000:1.0:1788619917.638538:0:5844:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619917 -> 1788619917 00000004:00010000:1.0:1788619917.638540:0:5844:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 518 -> 518 00000004:00010000:1.0:1788619917.638547:0:5844:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000003bca7579 reply state 000000005c941f8f transno 4294967683 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752c00/0x1cd7fe5368734dd7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619917.638556:0:5844:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788619917.638622:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa17100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+20:14908:x1875503571473536:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 4743us (5030us total) trans 4294967683 rc 0/0 00000100:00100000:1.0:1788619917.638629:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1016 00000020:00080000:1.0:1788619917.638633:0:5844:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1788619917.643524:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619917.643626:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571473792 00000100:00100000:1.0:1788619917.643635:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571473792 00000100:00100000:1.0:1788619917.643689:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 804 00000100:00100000:1.0:1788619917.643698:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa14000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:14908:x1875503571473792:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619917.643728:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619917.643749:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000022bdbaff/0x1cd7fe5368734dde lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c18d expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.643808:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000008901cab6/0x1cd7fe5368734d9f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2023424->2047999) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c15c expref: 5 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.643822:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(4 12) 00010000:00010000:1.0:1788619917.643826:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(5 12) 00010000:00010000:1.0:1788619917.643832:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000022bdbaff/0x1cd7fe5368734dde lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c18d expref: 8 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.643866:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000008901cab6/0x1cd7fe5368734d9f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2023424->2047999) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c15c expref: 5 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.643879:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000008901cab6/0x1cd7fe5368734d9f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2023424->2047999) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c15c expref: 5 pid: 6696 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619917.643892:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e6478e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589125632:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619917.644058:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752c00/0x1cd7fe5368734dd7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.644068:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752c00/0x1cd7fe5368734dd7 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.644082:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752c00/0x1cd7fe5368734dd7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.644087:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c5752c00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:3.0:1788619917.644091:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752c00/0x1cd7fe5368734dd7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.644099:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752c00/0x1cd7fe5368734dd7 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.644115:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619917.644118:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788619917.644123:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752c00/0x1cd7fe5368734dd7 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.644129:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752c00/0x1cd7fe5368734dd7 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619917.644234:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a900 going to sleep for 11 seconds 00000100:00100000:1.0:1788619917.650497:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e6478e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589125632:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619917.650533:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000022bdbaff/0x1cd7fe5368734dde lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c18d expref: 8 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.650555:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5753a00, rc 0) 00000100:00100000:1.0:1788619917.650640:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa14000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:14908:x1875503571473792:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6942us (7125us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619917.650649:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 804 00000100:00100000:2.0:1788619917.650839:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619917.651217:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571473920 00000100:00100000:3.0:1788619917.651224:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571473920 00002000:00100000:3.0:1788619917.651265:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (2023424->2047935) req@ffff8e78dfa41880 x1875503571473920/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/0 e 0 to 0 dl 1788619928 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619917.651274:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa41880 with x1875503571473920 ext(2023424->2047935) 00010000:00010000:3.0:1788619917.651292:0:6703:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000008901cab6/0x1cd7fe5368734d9f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2023424->2047999) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c15c expref: 6 pid: 6696 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.651306:0:6703:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000008901cab6/0x1cd7fe5368734d9f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2023424->2047999) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c15c expref: 6 pid: 6696 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788619917.651354:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff8e78dfa41880 00000100:00100000:3.0:1788619917.651376:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 15 00000100:00100000:3.0:1788619917.651386:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa41880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571473920:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:3.0:1788619917.654452:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa41880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+8:2354:x1875503571473920:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 3052us (3598us total) trans 4294967429 rc 0/0 00000100:00100000:3.0:1788619917.654467:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 15 00002000:00100000:3.0:1788619917.654481:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (2023424->2047935) req@ffff8e78dfa41880 x1875503571473920/t4294967429(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/448 e 0 to 0 dl 1788619928 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619917.654492:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa41880 with x1875503571473920 ext(2023424->2047935) 00010000:00010000:3.0:1788619917.654504:0:6703:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000008901cab6/0x1cd7fe5368734d9f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2023424->2047999) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c15c expref: 8 pid: 6696 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.654522:0:6703:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000008901cab6/0x1cd7fe5368734d9f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2023424->2047999) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c15c expref: 8 pid: 6696 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788619917.654530:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff8e78dfa41880 00000100:00100000:2.0:1788619917.659600:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619917.659785:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571474048 00000100:00100000:0.0:1788619917.659795:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571474048 00010000:00010000:0.0:1788619917.659834:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000008901cab6/0x1cd7fe5368734d9f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2023424->2047999) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c15c expref: 8 pid: 6696 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619917.659878:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 338 00000100:00100000:0.0:1788619917.659887:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa42680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+8:2354:x1875503571474048:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619917.659912:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:0.0:1788619917.659933:0:13330:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 0 -> 2047936 00010000:00010000:0.0:1788619917.659940:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000008901cab6/0x1cd7fe5368734d9f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2023424->2047999) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c15c expref: 8 pid: 6696 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619917.659955:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000008901cab6/0x1cd7fe5368734d9f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2023424->2047999) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c15c expref: 8 pid: 6696 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619917.659979:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008901cab6/0x1cd7fe5368734d9f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2023424->2047999) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c15c expref: 8 pid: 6696 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619917.659996:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(5 12) 00010000:00010000:0.0:1788619917.660001:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(5 12) 00010000:00010000:0.0:1788619917.660009:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000022bdbaff/0x1cd7fe5368734dde lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c18d expref: 7 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619917.660032:0:13330:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000022bdbaff/0x1cd7fe5368734dde lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c18d expref: 7 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619917.660093:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000022bdbaff/0x1cd7fe5368734dde lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c18d expref: 7 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619917.660107:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78dfa17480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589125760:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619917.660190:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c2c5ea80 going to sleep for 11 seconds 00000100:00100000:0.0:1788619917.666481:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78dfa17480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589125760:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619917.666511:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619917.666573:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa42680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571474048:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 6685us (6982us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619917.666585:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 338 00000100:00100000:3.0:1788619917.668451:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619917.668557:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619917.671960:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571474176 00000100:00100000:0.0:1788619917.671979:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571474176 00002000:00010000:0.0:1788619917.672034:0:5857:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:0.0:1788619917.672040:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5018000 with x1875503571474176 00000100:00100000:0.0:1788619917.672093:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 628 00000100:00100000:0.0:1788619917.672103:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5018000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2355:x1875503571474176:12345-192.168.201.5@tcp:10:fsx.0 00000004:00010000:0.0:1788619917.672388:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 937952 -> 231095 00000004:00010000:0.0:1788619917.672392:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619917 -> 1788619917 00000004:00010000:0.0:1788619917.672394:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619914 -> 1788619914 00000004:00010000:0.0:1788619917.672397:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619917 -> 1788619917 00000004:00010000:0.0:1788619917.672411:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 518 -> 518 00000100:00100000:0.0:1788619917.672485:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5018000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2355:x1875503571474176:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 384us (3928us total) trans 4294967684 rc 0/0 00000100:00100000:0.0:1788619917.672493:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 628 00002000:00010000:0.0:1788619917.672497:0:5857:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:0.0:1788619917.672500:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5018000 with x1875503571474176 00000100:00100000:1.0:1788619917.673060:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571474304 00000100:00100000:1.0:1788619917.673067:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571474304 00002000:00010000:1.0:1788619917.673090:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:1.0:1788619917.673095:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5018a80 with x1875503571474304 ext(0->18446744073709551615) 00002000:00010000:1.0:1788619917.673103:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5018a80. 00000100:00100000:1.0:1788619917.673176:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 452 00000100:00100000:1.0:1788619917.673184:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5018a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571474304:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:1.0:1788619917.673435:0:6702:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 2047936 -> 0 00002000:00010000:1.0:1788619917.673439:0:6702:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619917 -> 1788619917 00002000:00010000:1.0:1788619917.673441:0:6702:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619916 -> 1788619916 00002000:00010000:1.0:1788619917.673444:0:6702:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619917 -> 1788619917 00002000:00010000:1.0:1788619917.673446:0:6702:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 135 -> 135 00000100:00100000:1.0:1788619917.673482:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5018a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2355:x1875503571474304:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 298us (5043us total) trans 4294967430 rc 0/0 00000100:00100000:1.0:1788619917.673488:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 452 00002000:00010000:1.0:1788619917.673491:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:1.0:1788619917.673494:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5018a80 with x1875503571474304 ext(0->18446744073709551615) 00002000:00010000:1.0:1788619917.673498:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5018a80. 00000100:00100000:3.0:1788619917.680425:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619917.680540:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571474432 00000100:00100000:2.0:1788619917.680549:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571474432 00000100:00100000:2.0:1788619917.680617:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1017 00000100:00100000:2.0:1788619917.680627:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+8:14908:x1875503571474432:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619917.680647:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619917.680666:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779aa00/0x1cd7fe5368734de5 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c194 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.680750:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779b800/0x1cd7fe5368734dec lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.680766:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779b800/0x1cd7fe5368734dec lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.680833:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779b800/0x1cd7fe5368734dec lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788619917.680860:0:9417:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 231095 00000004:00200000:2.0:1788619917.680864:0:9417:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788619917.680882:0:9417:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779b800/0x1cd7fe5368734dec lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.681705:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779aa00/0x1cd7fe5368734de5 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c194 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.681728:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779b800/0x1cd7fe5368734dec lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c194 expref: 9 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.681737:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619917.681740:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788619917.681761:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c779b800, rc 0) 00000100:00100000:2.0:1788619917.681840:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571474432:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 1213us (1424us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619917.681848:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1017 00000100:00100000:2.0:1788619917.689661:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619917.689745:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619917.689852:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571474560 00000100:00100000:1.0:1788619917.689862:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571474560 00000100:00100000:1.0:1788619917.689972:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1018 00000100:00100000:1.0:1788619917.689984:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa43100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571474560:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619917.690005:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619917.690051:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753000/0x1cd7fe5368734df3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c19b expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.690098:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368734d8a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c14e expref: 19 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619917.690139:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e6478e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589125888:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619917.690222:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a900 going to sleep for 11 seconds 00000100:00100000:3.0:1788619917.690644:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571474688 00000100:00100000:3.0:1788619917.690649:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571474688 00000100:00100000:3.0:1788619917.690698:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 805 00000100:00100000:3.0:1788619917.690706:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa43800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571474688:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619917.690726:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619917.690737:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000485aa315/0x1cd7fe5368734dfa lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c1a2 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619917.690790:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5019500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589126016:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619917.690884:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:3.0:1788619917.693419:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5019500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589126016:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619917.693456:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000485aa315/0x1cd7fe5368734dfa lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1a2 expref: 8 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.693481:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000485aa315/0x1cd7fe5368734dfa lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1a2 expref: 8 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.693492:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683cc00, rc 0) 00000100:00100000:3.0:1788619917.693583:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa43800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571474688:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2877us (3994us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619917.693592:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 805 00000004:00010000:1.0:1788619917.693753:0:11397:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619914 -> 1788619916 00000100:00100000:1.0:1788619917.693783:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e6478e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589125888:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619917.693798:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 231095 00010000:00010000:1.0:1788619917.693813:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753000/0x1cd7fe5368734df3 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c19b expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.693826:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619917.693829:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619917.693833:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753000/0x1cd7fe5368734df3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c19b expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.693843:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5753000, rc 0) 00000100:00100000:1.0:1788619917.693876:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa43100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571474560:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3895us (4130us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619917.693882:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1018 00000100:00100000:1.0:1788619917.697250:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619917.697393:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619917.697460:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571474944 00000100:00100000:2.0:1788619917.697470:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571474944 00000100:00100000:2.0:1788619917.697535:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 806 00000100:00100000:2.0:1788619917.697546:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e6479f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+7:2352:x1875503571474944:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619917.697564:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619917.697587:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000528c47b1/0x1cd7fe5368734e01 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c1b0 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619917.697653:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78dfa40380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589126144:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619917.697736:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571474816 00000100:00100000:0.0:1788619917.697741:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571474816 00000100:00100000:2.0:1788619917.697747:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc000 going to sleep for 11 seconds 00000100:00100000:0.0:1788619917.698611:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1019 00000100:00100000:0.0:1788619917.698630:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e6478e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571474816:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619917.698643:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619917.698657:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767b400/0x1cd7fe5368734e08 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c1a9 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619917.698688:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368734d8a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c14e expref: 19 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619917.698715:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78dfa17480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589126272:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619917.698816:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c2c5ea80 going to sleep for 11 seconds 00000100:00100000:3.0:1788619917.701537:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78dfa40380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589126144:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619917.701575:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000528c47b1/0x1cd7fe5368734e01 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1b0 expref: 8 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.701614:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000528c47b1/0x1cd7fe5368734e01 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1b0 expref: 8 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.701648:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c779aa00, rc 0) 00000100:00100000:3.0:1788619917.701696:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e6479f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+7:2352:x1875503571474944:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4152us (4302us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619917.701705:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 806 00000100:00100000:0.0:1788619917.705667:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78dfa17480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589126272:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619917.705699:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 231095 00010000:00010000:0.0:1788619917.705718:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767b400/0x1cd7fe5368734e08 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1a9 expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619917.705739:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619917.705742:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619917.705747:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767b400/0x1cd7fe5368734e08 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1a9 expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619917.705784:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767b400, rc 0) 00000100:00100000:0.0:1788619917.705851:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e6478e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571474816:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7227us (8610us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619917.705860:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1019 00000100:00100000:0.0:1788619917.710281:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619917.710359:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619917.710425:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571475200 00000100:00100000:1.0:1788619917.710436:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571475200 00000100:00100000:2.0:1788619917.710466:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571475072 00000100:00100000:2.0:1788619917.710470:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571475072 00000100:00100000:1.0:1788619917.710491:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 807 00000100:00100000:2.0:1788619917.710507:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1020 00000100:00100000:2.0:1788619917.710516:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa17480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571475072:12345-192.168.201.5@tcp:101:fsx.0 00000100:00100000:1.0:1788619917.710521:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa17800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2353:x1875503571475200:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619917.710535:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619917.710535:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619917.710553:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a9ce7747/0x1cd7fe5368734e16 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c1be expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.710554:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779bc00/0x1cd7fe5368734e0f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c1b7 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.710587:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368734d8a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c14e expref: 19 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619917.710611:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e6479c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589126400:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619917.710621:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78dfa41c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589126528:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619917.710731:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccf00 going to sleep for 11 seconds 00000100:00100000:1.0:1788619917.713358:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a900 going to sleep for 11 seconds 00000100:00100000:1.0:1788619917.715947:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e6479c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589126400:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619917.715989:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a9ce7747/0x1cd7fe5368734e16 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1be expref: 8 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.716015:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a9ce7747/0x1cd7fe5368734e16 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1be expref: 8 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.716027:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5753000, rc 0) 00000100:00100000:1.0:1788619917.716096:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa17800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2353:x1875503571475200:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5575us (5769us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619917.716106:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 807 00000100:00100000:2.0:1788619917.717714:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78dfa41c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589126528:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619917.717740:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 231095 00010000:00010000:2.0:1788619917.717757:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779bc00/0x1cd7fe5368734e0f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1b7 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.717786:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619917.717790:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619917.717795:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779bc00/0x1cd7fe5368734e0f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1b7 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.717808:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c779bc00, rc 0) 00000100:00100000:2.0:1788619917.717897:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa17480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571475072:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7380us (7624us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619917.717920:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1020 00000100:00100000:0.0:1788619917.720628:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619917.720751:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571475328 00000100:00100000:1.0:1788619917.720760:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571475328 00000100:00100000:1.0:1788619917.720861:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1021 00000100:00100000:1.0:1788619917.720880:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa16d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571475328:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619917.720898:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619917.720932:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752400/0x1cd7fe5368734e1d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c1c5 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.720982:0:11397:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368734d8a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c14e expref: 19 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.720992:0:11397:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752400/0x1cd7fe5368734e1d lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1c5 expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.721025:0:11397:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368734d8a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c14e expref: 19 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.721047:0:11397:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368734d8a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c14e expref: 19 pid: 9417 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619917.721063:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e6479c00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589126656:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619917.721138:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a900 going to sleep for 11 seconds 00000100:00100000:3.0:1788619917.724426:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e6479c00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589126656:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619917.724462:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752400/0x1cd7fe5368734e1d lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1c5 expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.724481:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619917.724483:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619917.724488:0:11397:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752400/0x1cd7fe5368734e1d lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1c5 expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.724497:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752400, rc 0) 00000100:00100000:3.0:1788619917.724554:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa16d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+10:14908:x1875503571475328:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3684us (3934us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619917.724564:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1021 00000100:00100000:2.0:1788619917.724758:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619917.724880:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571475456 00000100:00100000:1.0:1788619917.724887:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571475456 00010000:00010000:1.0:1788619917.724932:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368734d8a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c14e expref: 20 pid: 9417 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619917.724966:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 339 00000100:00100000:1.0:1788619917.724975:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa41c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2354:x1875503571475456:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619917.724988:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619917.725012:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368734d8a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c14e expref: 20 pid: 9417 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.725049:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368734d8a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c14e expref: 20 pid: 9417 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.725071:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368734d8a lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c14e expref: 20 pid: 9417 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.725086:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619917.725090:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752400/0x1cd7fe5368734e1d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1c5 expref: 9 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.725099:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752400/0x1cd7fe5368734e1d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1c5 expref: 9 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.725107:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752400/0x1cd7fe5368734e1d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1c5 expref: 9 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.725137:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752400/0x1cd7fe5368734e1d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1c5 expref: 9 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619917.725150:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e6479c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589126784:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619917.725217:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a6c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619917.729462:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e6479c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589126784:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619917.729493:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619917.729566:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa41c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2354:x1875503571475456:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 4591us (4808us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619917.729610:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 339 00000100:00100000:0.0:1788619917.733125:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619917.733251:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571475584 00000100:00100000:3.0:1788619917.733262:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571475584 00002000:00100000:3.0:1788619917.733311:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78dfa16a00 x1875503571475584/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/0 e 0 to 0 dl 1788619928 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619917.733322:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78dfa16a00 with x1875503571475584 00000100:00100000:3.0:1788619917.733381:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 629 00000100:00100000:3.0:1788619917.733390:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa16a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571475584:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619917.733622:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa16a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571475584:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 232us (506us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619917.733630:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 629 00002000:00100000:3.0:1788619917.733636:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78dfa16a00 x1875503571475584/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/4536 e 0 to 0 dl 1788619928 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619917.733644:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78dfa16a00 with x1875503571475584 00000100:00100000:2.0:1788619917.739198:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619917.739308:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571475712 00000100:00100000:1.0:1788619917.739319:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571475712 00002000:00100000:1.0:1788619917.739403:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78dfa42300 x1875503571475712/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/0 e 0 to 0 dl 1788619928 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619917.739414:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78dfa42300 with x1875503571475712 00000100:00100000:1.0:1788619917.739452:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 630 00000100:00100000:1.0:1788619917.739460:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa42300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571475712:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619917.739731:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa42300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571475712:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 271us (544us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619917.739740:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 630 00002000:00100000:1.0:1788619917.739747:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78dfa42300 x1875503571475712/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/4536 e 0 to 0 dl 1788619928 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619917.739754:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78dfa42300 with x1875503571475712 00000100:00100000:2.0:1788619917.744032:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619917.744247:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571475840 00000100:00100000:0.0:1788619917.744258:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571475840 00002000:00100000:0.0:1788619917.744312:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78dfa40380 x1875503571475840/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/0 e 0 to 0 dl 1788619928 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619917.744370:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78dfa40380 with x1875503571475840 00000100:00100000:0.0:1788619917.744398:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 631 00000100:00100000:0.0:1788619917.744408:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa40380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571475840:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619917.744658:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa40380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571475840:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 251us (635us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619917.744666:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 631 00002000:00100000:0.0:1788619917.744673:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78dfa40380 x1875503571475840/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/4536 e 0 to 0 dl 1788619928 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619917.744680:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78dfa40380 with x1875503571475840 00000100:00100000:0.0:1788619917.748882:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619917.749129:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571475968 00000100:00100000:3.0:1788619917.749140:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571475968 00002000:00100000:3.0:1788619917.749198:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df385880 x1875503571475968/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/0 e 0 to 0 dl 1788619928 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619917.749209:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df385880 with x1875503571475968 00000100:00100000:3.0:1788619917.749253:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 632 00000100:00100000:3.0:1788619917.749263:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df385880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571475968:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619917.749557:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df385880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571475968:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 294us (684us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619917.749566:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 632 00002000:00100000:3.0:1788619917.749573:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df385880 x1875503571475968/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/4536 e 0 to 0 dl 1788619928 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619917.749580:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df385880 with x1875503571475968 00000100:00100000:2.0:1788619917.754120:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619917.754224:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571476096 00000100:00100000:1.0:1788619917.754234:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571476096 00002000:00100000:1.0:1788619917.754287:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78dfa43b80 x1875503571476096/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/0 e 0 to 0 dl 1788619928 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619917.754299:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78dfa43b80 with x1875503571476096 00000100:00100000:1.0:1788619917.754375:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 633 00000100:00100000:1.0:1788619917.754386:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa43b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571476096:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619917.754621:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa43b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571476096:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 235us (510us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619917.754628:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 633 00002000:00100000:1.0:1788619917.754653:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78dfa43b80 x1875503571476096/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/4536 e 0 to 0 dl 1788619928 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619917.754661:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78dfa43b80 with x1875503571476096 00000100:00100000:2.0:1788619917.758805:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619917.759064:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571476224 00000100:00100000:0.0:1788619917.759076:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571476224 00002000:00100000:0.0:1788619917.759154:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c50c1f80 x1875503571476224/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/0 e 0 to 0 dl 1788619928 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619917.759167:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c50c1f80 with x1875503571476224 00000100:00100000:0.0:1788619917.759210:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 634 00000100:00100000:0.0:1788619917.759220:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571476224:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619917.759540:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571476224:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 320us (749us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619917.759549:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 634 00002000:00100000:0.0:1788619917.759557:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c50c1f80 x1875503571476224/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/4536 e 0 to 0 dl 1788619928 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619917.759565:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c50c1f80 with x1875503571476224 00000100:00100000:0.0:1788619917.764253:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619917.764369:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571476352 00000100:00100000:3.0:1788619917.764378:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571476352 00002000:00100000:3.0:1788619917.764429:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df387b80 x1875503571476352/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/0 e 0 to 0 dl 1788619928 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619917.764440:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df387b80 with x1875503571476352 00000100:00100000:3.0:1788619917.764483:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 635 00000100:00100000:3.0:1788619917.764492:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df387b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571476352:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619917.764763:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df387b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571476352:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 271us (519us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619917.764782:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 635 00002000:00100000:3.0:1788619917.764789:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df387b80 x1875503571476352/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/4536 e 0 to 0 dl 1788619928 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619917.764796:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df387b80 with x1875503571476352 00000100:00100000:2.0:1788619917.768672:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619917.768876:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571476480 00000100:00100000:1.0:1788619917.768886:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571476480 00002000:00100000:1.0:1788619917.769072:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6d02d80 x1875503571476480/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/0 e 0 to 0 dl 1788619928 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619917.769083:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6d02d80 with x1875503571476480 00000100:00100000:1.0:1788619917.769118:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 636 00000100:00100000:1.0:1788619917.769129:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d02d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571476480:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619917.769623:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d02d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571476480:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 494us (959us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619917.769632:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 636 00002000:00100000:1.0:1788619917.769640:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6d02d80 x1875503571476480/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/4536 e 0 to 0 dl 1788619928 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619917.769648:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6d02d80 with x1875503571476480 00000100:00100000:2.0:1788619917.772959:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619917.773190:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571476608 00000100:00100000:0.0:1788619917.773201:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571476608 00002000:00100000:0.0:1788619917.773255:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6d01880 x1875503571476608/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/0 e 0 to 0 dl 1788619928 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619917.773265:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6d01880 with x1875503571476608 00000100:00100000:0.0:1788619917.773300:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 637 00000100:00100000:0.0:1788619917.773310:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d01880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571476608:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619917.773615:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d01880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571476608:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 306us (664us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619917.773623:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 637 00002000:00100000:0.0:1788619917.773630:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6d01880 x1875503571476608/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/4536 e 0 to 0 dl 1788619928 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619917.773658:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6d01880 with x1875503571476608 00000100:00100000:0.0:1788619917.778236:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619917.778368:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571476736 00000100:00100000:1.0:1788619917.778378:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571476736 00002000:00100000:1.0:1788619917.778431:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df384a80 x1875503571476736/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/0 e 0 to 0 dl 1788619928 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619917.778441:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df384a80 with x1875503571476736 00000100:00100000:1.0:1788619917.778467:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 638 00000100:00100000:1.0:1788619917.778477:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df384a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571476736:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619917.778783:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df384a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571476736:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 306us (556us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619917.778808:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 638 00002000:00100000:1.0:1788619917.778816:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df384a80 x1875503571476736/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/4536 e 0 to 0 dl 1788619928 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619917.778824:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df384a80 with x1875503571476736 00000100:00100000:2.0:1788619917.782422:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619917.782679:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571476864 00000100:00100000:3.0:1788619917.782690:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571476864 00002000:00100000:3.0:1788619917.782738:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5c08a80 x1875503571476864/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/0 e 0 to 0 dl 1788619928 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619917.782749:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5c08a80 with x1875503571476864 00000100:00100000:3.0:1788619917.782791:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 639 00000100:00100000:3.0:1788619917.782801:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c08a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571476864:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619917.783201:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c08a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571476864:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 399us (789us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619917.783209:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 639 00002000:00100000:3.0:1788619917.783216:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5c08a80 x1875503571476864/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/4536 e 0 to 0 dl 1788619928 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619917.783224:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5c08a80 with x1875503571476864 00000100:00100000:2.0:1788619917.788135:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619917.788242:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571476992 00000100:00100000:0.0:1788619917.788252:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571476992 00002000:00100000:0.0:1788619917.788300:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5c09180 x1875503571476992/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/0 e 0 to 0 dl 1788619928 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619917.788311:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5c09180 with x1875503571476992 00000100:00100000:0.0:1788619917.788404:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 640 00000100:00100000:0.0:1788619917.788416:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c09180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571476992:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619917.788730:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c09180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571476992:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 316us (604us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619917.788738:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 640 00002000:00100000:0.0:1788619917.788745:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5c09180 x1875503571476992/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:13/0 lens 488/4536 e 0 to 0 dl 1788619928 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619917.788752:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5c09180 with x1875503571476992 00000100:00100000:3.0:1788619917.791814:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619917.793836:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571477120 00000100:00100000:1.0:1788619917.793861:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571477120 00000100:00100000:1.0:1788619917.793942:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1022 00000100:00100000:1.0:1788619917.793953:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5019500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+19:14908:x1875503571477120:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619917.793977:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619917.793997:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734e24 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c1cc expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.794087:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753000/0x1cd7fe5368734e2b lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.794104:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753000/0x1cd7fe5368734e2b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.794115:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753000/0x1cd7fe5368734e2b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788619917.794144:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 231095 00000004:00200000:1.0:1788619917.794148:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788619917.794169:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753000/0x1cd7fe5368734e2b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.794196:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752e00/0x1cd7fe5368734e24 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1cc expref: 21 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.794233:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753000/0x1cd7fe5368734e2b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1cc expref: 20 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.794241:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619917.794244:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788619917.794247:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5753000, rc 0) 00000100:00100000:1.0:1788619917.794315:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5019500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+20:14908:x1875503571477120:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 364us (2511us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619917.794322:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1022 00000100:00100000:2.0:1788619917.801409:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619917.801584:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571477248 00000100:00100000:0.0:1788619917.801595:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571477248 00000100:00100000:0.0:1788619917.801727:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1023 00000100:00100000:0.0:1788619917.801736:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2355:x1875503571477248:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619917.801759:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619917.801789:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767b000/0x1cd7fe5368734e32 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c1d3 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619917.801833:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767b000/0x1cd7fe5368734e32 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1d3 expref: 21 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788619917.801861:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 231095 00010000:00010000:0.0:1788619917.801871:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767b000/0x1cd7fe5368734e32 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1d3 expref: 21 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619917.801886:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619917.801890:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619917.801894:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767b000, rc 0) 00000100:00100000:0.0:1788619917.801959:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2355:x1875503571477248:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 223us (561us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619917.801966:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1023 00000100:00100000:2.0:1788619917.805975:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619917.806072:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571477376 00000100:00100000:3.0:1788619917.806082:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571477376 00000100:00100000:3.0:1788619917.806147:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1024 00000100:00100000:3.0:1788619917.806171:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+21:14908:x1875503571477376:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619917.806189:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619917.806192:0:11397:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1788619917.806233:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767b000/0x1cd7fe5368734e32 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1d3 expref: 21 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.806254:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619917.806259:0:11397:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788619917.806272:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368734e39 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c1da expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.806297:0:11397:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752400/0x1cd7fe5368734e1d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1c5 expref: 9 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.806307:0:11397:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368734e39 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1da expref: 21 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.806401:0:11397:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752400/0x1cd7fe5368734e1d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1c5 expref: 9 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.806416:0:11397:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752400/0x1cd7fe5368734e1d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1c5 expref: 9 pid: 11397 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619917.806429:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501b800 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589126912:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619917.809929:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa240 going to sleep for 11 seconds 00000100:00100000:0.0:1788619917.810411:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619917.810495:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571477504 00000100:00100000:1.0:1788619917.810515:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571477504 00010000:00010000:1.0:1788619917.810558:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752400/0x1cd7fe5368734e1d lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1c5 expref: 10 pid: 11397 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619917.810602:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 340 00000100:00100000:1.0:1788619917.810612:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df386a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571477504:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619917.810628:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619917.810663:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752400/0x1cd7fe5368734e1d lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1c5 expref: 10 pid: 11397 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.810677:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752400/0x1cd7fe5368734e1d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1c5 expref: 10 pid: 11397 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.810699:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619917.810705:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368734e39 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1da expref: 21 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.810732:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368734e39 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1da expref: 21 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.810742:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368734e39 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1da expref: 21 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.810797:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368734e39 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1da expref: 21 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619917.810810:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e6479c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589127040:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619917.810888:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a240 going to sleep for 11 seconds 00010000:00010000:3.0:1788619917.814442:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752400/0x1cd7fe5368734e1d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1c5 expref: 10 pid: 11397 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619917.814486:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501b800 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589126912:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619917.814529:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368734e39 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1da expref: 21 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.814545:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619917.814549:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619917.814553:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683cc00, rc 0) 00000100:00100000:2.0:1788619917.814902:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+21:14908:x1875503571477376:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8724us (8927us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619917.814913:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1024 00000100:00100000:1.0:1788619917.816319:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e6479c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589127040:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619917.816376:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619917.816430:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df386a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571477504:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 5819us (6022us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619917.816441:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 340 00000100:00100000:3.0:1788619917.819914:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619917.820008:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571477632 00000100:00100000:2.0:1788619917.820017:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571477632 00000100:00100000:2.0:1788619917.820097:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 641 00000100:00100000:2.0:1788619917.820108:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2354:x1875503571477632:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:2.0:1788619917.820410:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 231095 -> 231095 00000004:00010000:2.0:1788619917.820414:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619917 -> 1788619917 00000004:00010000:2.0:1788619917.820418:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619916 -> 1788619916 00000004:00010000:2.0:1788619917.820420:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619917 -> 1788619917 00000004:00010000:2.0:1788619917.820423:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 518 -> 518 00000100:00100000:2.0:1788619917.820559:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2354:x1875503571477632:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 452us (655us total) trans 4294967685 rc 0/0 00000100:00100000:2.0:1788619917.820567:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 641 00000100:00100000:2.0:1788619917.826151:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619917.827510:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571477888 00000100:00100000:0.0:1788619917.827524:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571477888 00000100:00100000:0.0:1788619917.827591:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1025 00000100:00100000:0.0:1788619917.827603:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+22:14908:x1875503571477888:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:0.0:1788619917.827660:0:5846:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619917.827686:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753000/0x1cd7fe5368734e2b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1cc expref: 22 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619917.827709:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619917.827714:0:5846:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788619917.827936:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a400/0x1cd7fe5368734e40 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619917.827954:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779b800/0x1cd7fe5368734dec lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c194 expref: 8 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619917.827989:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a400/0x1cd7fe5368734e40 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619917.828025:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779b800/0x1cd7fe5368734dec lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c194 expref: 8 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619917.828039:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779b800/0x1cd7fe5368734dec lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c194 expref: 8 pid: 9417 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619917.828053:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df384000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589127168:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619917.828142:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c2c5e300 going to sleep for 11 seconds 00000100:00100000:0.0:1788619917.831546:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619917.831673:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df384000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589127168:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619917.831702:0:5846:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a400/0x1cd7fe5368734e40 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619917.831717:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571478016 00000100:00100000:3.0:1788619917.831724:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571478016 00010000:00010000:3.0:1788619917.831756:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779b800/0x1cd7fe5368734dec lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c194 expref: 9 pid: 9417 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619917.831843:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 341 00000100:00100000:3.0:1788619917.831853:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8559050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571478016:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619917.831867:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619917.831873:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779b800/0x1cd7fe5368734dec lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c194 expref: 9 pid: 9417 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.831884:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779b800/0x1cd7fe5368734dec lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c194 expref: 9 pid: 9417 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.831916:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779b800/0x1cd7fe5368734dec lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c194 expref: 9 pid: 9417 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.831950:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619917.831956:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a400/0x1cd7fe5368734e40 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.831966:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a400/0x1cd7fe5368734e40 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.831974:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a400/0x1cd7fe5368734e40 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.832004:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619917.832066:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8559050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+8:2355:x1875503571478016:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 214us (528us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619917.832074:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 341 00010000:00010000:2.0:1788619917.832211:0:5846:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a400/0x1cd7fe5368734e40 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.832219:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a400/0x1cd7fe5368734e40 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788619917.832343:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967686 00000004:00010000:2.0:1788619917.832399:0:5846:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 231095 -> 231095 00000004:00010000:2.0:1788619917.832403:0:5846:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619917 -> 1788619917 00000004:00010000:2.0:1788619917.832406:0:5846:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619916 -> 1788619916 00000004:00010000:2.0:1788619917.832408:0:5846:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619917 -> 1788619917 00000004:00010000:2.0:1788619917.832411:0:5846:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 518 -> 518 00000004:00010000:2.0:1788619917.832418:0:5846:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000249ca07d reply state 0000000090cd437a transno 4294967687 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a400/0x1cd7fe5368734e40 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619917.832426:0:5846:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788619917.832518:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+24:14908:x1875503571477888:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 4918us (6378us total) trans 4294967687 rc 0/0 00000100:00100000:2.0:1788619917.832526:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1025 00000020:00080000:2.0:1788619917.832530:0:5846:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1788619917.836216:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00010000:00010000:2.0:1788619917.836725:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a400/0x1cd7fe5368734e40 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.836739:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a400/0x1cd7fe5368734e40 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.836747:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a400/0x1cd7fe5368734e40 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.836753:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c767a400 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:2.0:1788619917.836760:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a400/0x1cd7fe5368734e40 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.836780:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a400/0x1cd7fe5368734e40 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.836798:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619917.836802:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788619917.836822:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a400/0x1cd7fe5368734e40 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.836830:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a400/0x1cd7fe5368734e40 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619917.836959:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571478144 00000100:00100000:3.0:1788619917.836970:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571478144 00002000:00010000:3.0:1788619917.837007:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1808661->18446744073709551615) 00002000:00100000:3.0:1788619917.837012:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df384380 with x1875503571478144 ext(1808661->18446744073709551615) 00002000:00010000:3.0:1788619917.837023:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df384380. 00000100:00100000:3.0:1788619917.837061:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 453 00000100:00100000:3.0:1788619917.837072:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df384380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571478144:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:3.0:1788619917.837295:0:6703:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 0 -> 1808661 00002000:00010000:3.0:1788619917.837299:0:6703:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619917 -> 1788619917 00002000:00010000:3.0:1788619917.837302:0:6703:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619916 -> 1788619916 00002000:00010000:3.0:1788619917.837304:0:6703:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619917 -> 1788619917 00002000:00010000:3.0:1788619917.837306:0:6703:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 135 -> 135 00000100:00100000:3.0:1788619917.837406:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df384380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+10:2354:x1875503571478144:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 334us (1199us total) trans 4294967431 rc 0/0 00000100:00100000:3.0:1788619917.837414:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 453 00002000:00010000:3.0:1788619917.837417:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1808661->18446744073709551615) 00002000:00100000:3.0:1788619917.837420:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df384380 with x1875503571478144 ext(1808661->18446744073709551615) 00002000:00010000:3.0:1788619917.837425:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df384380. 00000100:00100000:0.0:1788619917.840561:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619917.840734:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571478272 00000100:00100000:3.0:1788619917.840744:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571478272 00000100:00100000:3.0:1788619917.840803:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1026 00000100:00100000:3.0:1788619917.840814:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df386680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+23:14908:x1875503571478272:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619917.840838:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619917.840859:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368734e47 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c1e1 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.840959:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368734e4e lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.840974:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368734e4e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.840986:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368734e4e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788619917.841013:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1808661 00000004:00200000:3.0:1788619917.841017:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788619917.841047:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368734e4e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.841074:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368734e47 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1e1 expref: 25 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.841115:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368734e4e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1e1 expref: 24 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.841125:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619917.841128:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788619917.841131:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d600, rc 0) 00000100:00100000:3.0:1788619917.841197:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df386680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+24:14908:x1875503571478272:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 386us (646us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619917.841204:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1026 00000100:00100000:1.0:1788619917.844864:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619917.844940:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571478400 00000100:00100000:3.0:1788619917.844950:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571478400 00000100:00100000:3.0:1788619917.845005:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 808 00000100:00100000:3.0:1788619917.845016:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e6479c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:14908:x1875503571478400:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619917.845045:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619917.845065:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007736e663/0x1cd7fe5368734e55 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c1e8 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.845100:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000022bdbaff/0x1cd7fe5368734dde lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c18d expref: 9 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.845110:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(5 12) 00010000:00010000:3.0:1788619917.845114:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(6 12) 00010000:00010000:3.0:1788619917.845119:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007736e663/0x1cd7fe5368734e55 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1150976->1196031] (req 1150976->1196031) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1e8 expref: 8 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.845175:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000022bdbaff/0x1cd7fe5368734dde lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c18d expref: 9 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.845190:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000022bdbaff/0x1cd7fe5368734dde lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c18d expref: 9 pid: 6697 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619917.845204:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5019180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589127296:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619917.845296:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaacc0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619917.850951:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619917.851172:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571478528 00000100:00100000:0.0:1788619917.851184:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571478528 00010000:00010000:0.0:1788619917.851223:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000022bdbaff/0x1cd7fe5368734dde lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c18d expref: 10 pid: 6697 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619917.851263:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 342 00000100:00100000:0.0:1788619917.851272:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010ff800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+10:2353:x1875503571478528:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619917.851286:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619917.851309:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000022bdbaff/0x1cd7fe5368734dde lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c18d expref: 10 pid: 6697 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619917.851359:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000022bdbaff/0x1cd7fe5368734dde lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c18d expref: 10 pid: 6697 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619917.851386:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(6 12) 00010000:00010000:0.0:1788619917.851390:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(6 12) 00010000:00010000:0.0:1788619917.851399:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000007736e663/0x1cd7fe5368734e55 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1150976->1196031) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1e8 expref: 8 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619917.851411:0:13330:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007736e663/0x1cd7fe5368734e55 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1150976->1196031) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1e8 expref: 8 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619917.851450:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000007736e663/0x1cd7fe5368734e55 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1150976->1196031) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1e8 expref: 8 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619917.851463:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df384700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589127424:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619917.851549:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c2c5ef00 going to sleep for 11 seconds 00010000:00010000:3.0:1788619917.854543:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000022bdbaff/0x1cd7fe5368734dde lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c18d expref: 10 pid: 6697 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619917.854569:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5019180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589127296:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619917.854613:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007736e663/0x1cd7fe5368734e55 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1150976->1196031) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1e8 expref: 8 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.854630:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d400, rc 0) 00000100:00100000:2.0:1788619917.854859:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e6479c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+8:14908:x1875503571478400:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 9843us (10005us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619917.854867:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 808 00000100:00100000:0.0:1788619917.855232:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df384700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589127424:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619917.855247:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619917.855281:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010ff800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2353:x1875503571478528:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 4009us (4341us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619917.855290:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 342 00000100:00100000:3.0:1788619917.857417:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619917.857482:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571478656 00000100:00100000:1.0:1788619917.857493:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571478656 00000100:00100000:1.0:1788619917.857570:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 454 00000100:00100000:1.0:1788619917.857581:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5019180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+8:2354:x1875503571478656:12345-192.168.201.5@tcp:22:fsx.0 00002000:00010000:1.0:1788619917.857860:0:6702:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1808661 -> 1808661 00002000:00010000:1.0:1788619917.857864:0:6702:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619917 -> 1788619917 00002000:00010000:1.0:1788619917.857867:0:6702:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619916 -> 1788619916 00002000:00010000:1.0:1788619917.857883:0:6702:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619917 -> 1788619917 00002000:00010000:1.0:1788619917.857886:0:6702:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 135 -> 135 00000100:00100000:1.0:1788619917.858028:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5019180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571478656:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 448us (700us total) trans 4294967432 rc 0/0 00000100:00100000:1.0:1788619917.858045:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 454 00000100:00100000:1.0:1788619917.860231:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619917.860316:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571478784 00000100:00100000:2.0:1788619917.860338:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571478784 00000100:00100000:2.0:1788619917.860396:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1027 00000100:00100000:2.0:1788619917.860407:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010ff480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+8:14908:x1875503571478784:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619917.860426:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619917.860446:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779ae00/0x1cd7fe5368734e5c lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c1ef expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.860547:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779a800/0x1cd7fe5368734e63 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.860564:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779a800/0x1cd7fe5368734e63 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.860578:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779a800/0x1cd7fe5368734e63 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788619917.860616:0:11397:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1808661 00000004:00200000:2.0:1788619917.860621:0:11397:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788619917.860639:0:11397:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779a800/0x1cd7fe5368734e63 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.860685:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779ae00/0x1cd7fe5368734e5c lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1ef expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.860702:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779a800/0x1cd7fe5368734e63 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1ef expref: 9 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.860711:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619917.860713:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788619917.860717:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c779a800, rc 0) 00000100:00100000:2.0:1788619917.860786:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010ff480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571478784:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 381us (565us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619917.860793:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1027 00000100:00100000:0.0:1788619917.863367:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619917.863698:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571478912 00000100:00100000:2.0:1788619917.863709:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571478912 00000100:00100000:2.0:1788619917.863761:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1028 00000100:00100000:2.0:1788619917.863780:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df384700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571478912:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619917.863798:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619917.863818:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779be00/0x1cd7fe5368734e6a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c1f6 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.863853:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368734e39 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1da expref: 23 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619917.863889:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be1880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589127552:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619917.863966:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccf00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619917.868375:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be1880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589127552:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619917.868435:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 231095 00010000:00010000:2.0:1788619917.868458:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779be00/0x1cd7fe5368734e6a lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1f6 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.868482:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619917.868486:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619917.868491:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779be00/0x1cd7fe5368734e6a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1f6 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.868517:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c779be00, rc 0) 00000100:00100000:2.0:1788619917.868573:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df384700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571478912:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4803us (5245us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619917.868582:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1028 00000100:00100000:1.0:1788619917.872001:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619917.872365:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571479040 00000100:00100000:0.0:1788619917.872375:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571479040 00000100:00100000:0.0:1788619917.872443:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1029 00000100:00100000:0.0:1788619917.872452:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010fe680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571479040:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619917.872473:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619917.872491:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a400/0x1cd7fe5368734e71 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c1fd expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619917.872536:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368734e39 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1da expref: 23 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619917.872576:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df385c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589127680:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619917.872650:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c2c5ef00 going to sleep for 11 seconds 00000100:00100000:1.0:1788619917.875497:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df385c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589127680:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619917.875535:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 231095 00010000:00010000:1.0:1788619917.875552:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a400/0x1cd7fe5368734e71 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1fd expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.875572:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619917.875575:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619917.875581:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a400/0x1cd7fe5368734e71 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1fd expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.875604:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767a400, rc 0) 00000100:00100000:1.0:1788619917.875659:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010fe680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571479040:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3207us (3667us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619917.875667:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1029 00000100:00100000:1.0:1788619917.878177:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619917.878260:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571479296 00000100:00100000:0.0:1788619917.878268:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571479296 00000100:00100000:0.0:1788619917.878358:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1030 00000100:00100000:0.0:1788619917.878369:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010fdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571479296:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:0.0:1788619917.878427:0:9417:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619917.878454:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779a800/0x1cd7fe5368734e63 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1ef expref: 9 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619917.878471:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619917.878476:0:9417:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788619917.879197:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767be00/0x1cd7fe5368734e78 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619917.879215:0:9417:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368734e4e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1e1 expref: 23 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619917.879224:0:9417:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767be00/0x1cd7fe5368734e78 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619917.879273:0:9417:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368734e4e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1e1 expref: 23 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619917.879286:0:9417:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368734e4e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1e1 expref: 23 pid: 5845 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619917.879302:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df385500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589127808:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619917.879395:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c2c5ef00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619917.882059:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619917.882561:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df385500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589127808:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619917.882587:0:9417:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767be00/0x1cd7fe5368734e78 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619917.882757:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571479424 00000100:00100000:1.0:1788619917.882764:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571479424 00010000:00010000:1.0:1788619917.882809:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368734e4e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1e1 expref: 24 pid: 5845 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619917.882836:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 343 00000100:00100000:1.0:1788619917.882857:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be1880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2354:x1875503571479424:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619917.882870:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619917.882876:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368734e4e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1e1 expref: 24 pid: 5845 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.882887:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368734e4e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1e1 expref: 24 pid: 5845 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.882917:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368734e4e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1e1 expref: 24 pid: 5845 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.882933:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619917.882938:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767be00/0x1cd7fe5368734e78 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.882949:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767be00/0x1cd7fe5368734e78 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.882958:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767be00/0x1cd7fe5368734e78 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.882986:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619917.883048:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be1880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+23:2354:x1875503571479424:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 179us (985us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619917.883055:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 343 00010000:00010000:0.0:1788619917.884009:0:9417:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767be00/0x1cd7fe5368734e78 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619917.884019:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767be00/0x1cd7fe5368734e78 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788619917.884144:0:9417:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967688 00000004:00010000:0.0:1788619917.884189:0:9417:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 231095 -> 231095 00000004:00010000:0.0:1788619917.884193:0:9417:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619917 -> 1788619917 00000004:00010000:0.0:1788619917.884195:0:9417:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619916 -> 1788619916 00000004:00010000:0.0:1788619917.884197:0:9417:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619917 -> 1788619917 00000004:00010000:0.0:1788619917.884200:0:9417:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 518 -> 518 00000004:00010000:0.0:1788619917.884207:0:9417:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000375fb512 reply state 000000001c318258 transno 4294967689 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767be00/0x1cd7fe5368734e78 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619917.884231:0:9417:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788619917.884323:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010fdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571479296:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 5955us (6152us total) trans 4294967689 rc 0/0 00000100:00100000:0.0:1788619917.884344:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1030 00000020:00080000:0.0:1788619917.884348:0:9417:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1788619917.887606:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00010000:00010000:3.0:1788619917.888257:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767be00/0x1cd7fe5368734e78 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.888269:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767be00/0x1cd7fe5368734e78 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.888277:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767be00/0x1cd7fe5368734e78 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.888294:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c767be00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:3.0:1788619917.888299:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767be00/0x1cd7fe5368734e78 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.888308:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767be00/0x1cd7fe5368734e78 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.888468:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619917.888472:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788619917.888477:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767be00/0x1cd7fe5368734e78 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.888484:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767be00/0x1cd7fe5368734e78 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619917.888702:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571479552 00000100:00100000:0.0:1788619917.888712:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571479552 00000100:00100000:0.0:1788619917.888812:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1031 00000100:00100000:0.0:1788619917.888822:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:14908:x1875503571479552:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619917.888843:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619917.888861:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a000/0x1cd7fe5368734e7f lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c204 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619917.888886:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368734e39 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1da expref: 22 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619917.888895:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a000/0x1cd7fe5368734e7f lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c204 expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619917.888943:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368734e39 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1da expref: 22 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619917.888957:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368734e39 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1da expref: 22 pid: 11397 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619917.888970:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df385500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589127936:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619917.889075:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c2c5ef00 going to sleep for 11 seconds 00000100:00100000:0.0:1788619917.892042:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df385500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589127936:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619917.892081:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a000/0x1cd7fe5368734e7f lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c204 expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619917.892100:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619917.892103:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619917.892107:0:5846:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a000/0x1cd7fe5368734e7f lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c204 expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619917.892118:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767a000, rc 0) 00000100:00100000:0.0:1788619917.892178:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571479552:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3357us (4590us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619917.892187:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1031 00000100:00100000:1.0:1788619917.894750:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619917.894844:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571479680 00000100:00100000:2.0:1788619917.894853:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571479680 00010000:00010000:2.0:1788619917.894895:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368734e39 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1da expref: 23 pid: 11397 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619917.894945:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 344 00000100:00100000:2.0:1788619917.894957:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010fed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+23:2354:x1875503571479680:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619917.894975:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619917.895002:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368734e39 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1da expref: 23 pid: 11397 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.895018:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368734e39 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1da expref: 23 pid: 11397 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.895233:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368734e39 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1da expref: 23 pid: 11397 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.895250:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619917.895256:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a000/0x1cd7fe5368734e7f lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c204 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.895267:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a000/0x1cd7fe5368734e7f lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c204 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.895277:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a000/0x1cd7fe5368734e7f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c204 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.895317:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a000/0x1cd7fe5368734e7f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c204 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619917.895370:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be3b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589128064:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619917.895451:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccf00 going to sleep for 11 seconds 00000100:00100000:1.0:1788619917.899442:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619917.899476:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619917.899639:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be3b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589128064:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619917.899669:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619917.899723:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010fed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2354:x1875503571479680:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 4768us (4982us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619917.899735:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 344 00000100:00100000:3.0:1788619917.899870:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571479808 00000100:00100000:3.0:1788619917.899878:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571479808 00002000:00010000:3.0:1788619917.899914:0:5857:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:3.0:1788619917.899919:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79010fc000 with x1875503571479808 00000100:00100000:3.0:1788619917.899952:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 642 00000100:00100000:3.0:1788619917.899962:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010fc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571479808:12345-192.168.201.5@tcp:10:fsx.0 00000004:00010000:3.0:1788619917.900177:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 231095 -> 994373 00000004:00010000:3.0:1788619917.900181:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619917 -> 1788619917 00000004:00010000:3.0:1788619917.900184:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619916 -> 1788619916 00000004:00010000:3.0:1788619917.900187:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619917 -> 1788619917 00000004:00010000:3.0:1788619917.900190:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 518 -> 518 00000100:00100000:3.0:1788619917.900226:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010fc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571479808:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 266us (748us total) trans 4294967690 rc 0/0 00000100:00100000:3.0:1788619917.900253:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 642 00002000:00010000:3.0:1788619917.900257:0:5857:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:3.0:1788619917.900260:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79010fc000 with x1875503571479808 00000100:00100000:0.0:1788619917.900522:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571479936 00000100:00100000:0.0:1788619917.900528:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571479936 00002000:00010000:0.0:1788619917.900549:0:6701:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:0.0:1788619917.900553:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e79010fca80 with x1875503571479936 ext(0->18446744073709551615) 00002000:00010000:0.0:1788619917.900562:0:6701:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e79010fca80. 00000100:00100000:0.0:1788619917.900587:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 455 00000100:00100000:0.0:1788619917.900602:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010fca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571479936:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:0.0:1788619917.900799:0:6701:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1808661 -> 0 00002000:00010000:0.0:1788619917.900802:0:6701:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619917 -> 1788619917 00002000:00010000:0.0:1788619917.900805:0:6701:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619916 -> 1788619916 00002000:00010000:0.0:1788619917.900808:0:6701:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619917 -> 1788619917 00002000:00010000:0.0:1788619917.900810:0:6701:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 135 -> 135 00000100:00100000:0.0:1788619917.900844:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010fca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571479936:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 243us (1413us total) trans 4294967433 rc 0/0 00000100:00100000:0.0:1788619917.900852:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 455 00002000:00010000:0.0:1788619917.900855:0:6701:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:0.0:1788619917.900858:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e79010fca80 with x1875503571479936 ext(0->18446744073709551615) 00002000:00010000:0.0:1788619917.900862:0:6701:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e79010fca80. 00000100:00100000:1.0:1788619917.905219:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619917.905349:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571480064 00000100:00100000:0.0:1788619917.905359:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571480064 00000100:00100000:0.0:1788619917.905416:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1032 00000100:00100000:0.0:1788619917.905426:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010ff100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+22:14908:x1875503571480064:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619917.905447:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619917.905471:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a800/0x1cd7fe5368734e86 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c20b expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619917.905786:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767ac00/0x1cd7fe5368734e8d lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619917.905803:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767ac00/0x1cd7fe5368734e8d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619917.905816:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767ac00/0x1cd7fe5368734e8d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619917.905844:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 994373 00000004:00200000:0.0:1788619917.905848:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619917.905867:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767ac00/0x1cd7fe5368734e8d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619917.905894:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a800/0x1cd7fe5368734e86 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c20b expref: 24 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619917.905923:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767ac00/0x1cd7fe5368734e8d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c20b expref: 23 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619917.905932:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619917.905934:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619917.905938:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767ac00, rc 0) 00000100:00100000:0.0:1788619917.906002:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010ff100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+23:14908:x1875503571480064:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 577us (792us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619917.906009:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1032 00000100:00100000:1.0:1788619917.909934:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619917.909976:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619917.910060:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571480192 00000100:00100000:0.0:1788619917.910069:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571480192 00000100:00100000:0.0:1788619917.910128:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1033 00000100:00100000:0.0:1788619917.910139:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010fe300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+23:2354:x1875503571480192:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619917.910161:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619917.910181:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767b000/0x1cd7fe5368734e94 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c212 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619917.910218:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a000/0x1cd7fe5368734e7f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c204 expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619917.910260:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df385500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589128192:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619917.910365:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c2c5ef00 going to sleep for 11 seconds 00000100:00100000:3.0:1788619917.911004:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571480320 00000100:00100000:3.0:1788619917.911011:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571480320 00000100:00100000:3.0:1788619917.911063:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 809 00000100:00100000:3.0:1788619917.911073:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010fd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2355:x1875503571480320:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619917.911083:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619917.911094:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c4fc38e/0x1cd7fe5368734e9b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c219 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619917.911137:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5018700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589128320:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619917.911188:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaacc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619917.914390:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df385500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589128192:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619917.914421:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:0.0:1788619917.914442:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767b000/0x1cd7fe5368734e94 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c212 expref: 24 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619917.914480:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619917.914483:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619917.914489:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767b000/0x1cd7fe5368734e94 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c212 expref: 24 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619917.914515:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767b000, rc 0) 00000100:00100000:0.0:1788619917.914567:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010fe300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+23:2354:x1875503571480192:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4431us (4644us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619917.914577:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1033 00000100:00100000:3.0:1788619917.917443:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5018700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589128320:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619917.917482:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004c4fc38e/0x1cd7fe5368734e9b lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c219 expref: 10 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.917574:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c4fc38e/0x1cd7fe5368734e9b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c219 expref: 10 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.917588:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683ce00, rc 0) 00000100:00100000:2.0:1788619917.917833:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010fd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2355:x1875503571480320:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6759us (7853us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619917.917859:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 809 00000100:00100000:0.0:1788619917.921200:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619917.921280:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619917.921351:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571480576 00000100:00100000:3.0:1788619917.921360:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571480576 00000100:00100000:3.0:1788619917.921404:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 810 00000100:00100000:3.0:1788619917.921412:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df386300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2353:x1875503571480576:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619917.921440:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619917.921457:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000485aa315/0x1cd7fe5368734ea2 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c227 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619917.921610:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5018700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589128448:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619917.921688:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaacc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619917.923952:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5018700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589128448:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619917.923991:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000485aa315/0x1cd7fe5368734ea2 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c227 expref: 10 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.924019:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000485aa315/0x1cd7fe5368734ea2 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c227 expref: 10 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.924044:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683cc00, rc 0) 00000100:00100000:3.0:1788619917.924132:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df386300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2353:x1875503571480576:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2719us (2850us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619917.924140:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 810 00000100:00100000:2.0:1788619917.924356:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571480448 00000100:00100000:2.0:1788619917.924363:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571480448 00000100:00100000:2.0:1788619917.924407:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1034 00000100:00100000:2.0:1788619917.924415:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df385500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+23:2353:x1875503571480448:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619917.924432:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619917.924449:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779a200/0x1cd7fe5368734ea9 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c220 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.924481:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a000/0x1cd7fe5368734e7f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c204 expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619917.924729:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be3b80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589128576:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619917.924789:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccf00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619917.927734:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be3b80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589128576:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619917.927765:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:2.0:1788619917.927797:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779a200/0x1cd7fe5368734ea9 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c220 expref: 24 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.927817:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619917.927819:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619917.927825:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c779a200/0x1cd7fe5368734ea9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c220 expref: 24 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619917.927840:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c779a200, rc 0) 00000100:00100000:2.0:1788619917.928016:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df385500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+23:2353:x1875503571480448:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3600us (6822us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619917.928101:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1034 00000100:00100000:0.0:1788619917.931193:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619917.931267:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571480704 00000100:00100000:1.0:1788619917.931277:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571480704 00000100:00100000:1.0:1788619917.931387:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1035 00000100:00100000:1.0:1788619917.931398:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df385f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+23:14908:x1875503571480704:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619917.931421:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619917.931442:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753a00/0x1cd7fe5368734eb0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c22e expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.931470:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a000/0x1cd7fe5368734e7f lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c204 expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.931481:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753a00/0x1cd7fe5368734eb0 lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c22e expref: 24 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.931621:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a000/0x1cd7fe5368734e7f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c204 expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.931633:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a000/0x1cd7fe5368734e7f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c204 expref: 11 pid: 5846 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619917.931646:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed63480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589128704:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619917.931727:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a240 going to sleep for 11 seconds 00000100:00100000:1.0:1788619917.937822:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed63480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589128704:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619917.937863:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753a00/0x1cd7fe5368734eb0 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c22e expref: 24 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619917.938018:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619917.938129:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571480832 00000100:00100000:3.0:1788619917.938137:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571480832 00010000:00010000:3.0:1788619917.938171:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a000/0x1cd7fe5368734e7f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c204 expref: 12 pid: 5846 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619917.938209:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 345 00000100:00100000:3.0:1788619917.938218:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be3b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571480832:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619917.938233:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619917.938350:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619917.938354:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619917.938360:0:5844:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753a00/0x1cd7fe5368734eb0 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c22e expref: 24 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.938374:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5753a00, rc 0) 00000100:00100000:1.0:1788619917.938437:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df385f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+24:14908:x1875503571480704:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7041us (7253us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619917.938445:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1035 00010000:00010000:3.0:1788619917.939870:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a000/0x1cd7fe5368734e7f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c204 expref: 12 pid: 5846 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.939887:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a000/0x1cd7fe5368734e7f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c204 expref: 12 pid: 5846 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.939916:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767a000/0x1cd7fe5368734e7f lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c204 expref: 12 pid: 5846 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.939932:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619917.939939:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753a00/0x1cd7fe5368734eb0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c22e expref: 23 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.939949:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753a00/0x1cd7fe5368734eb0 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c22e expref: 23 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.939959:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753a00/0x1cd7fe5368734eb0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c22e expref: 23 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619917.940003:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753a00/0x1cd7fe5368734eb0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c22e expref: 23 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619917.940017:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5018700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589128832:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619917.940091:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaacc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619917.946871:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619917.947008:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571480960 00000100:00100000:1.0:1788619917.947018:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571480960 00002000:00100000:1.0:1788619917.947075:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be2680 x1875503571480960/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:13/0 lens 488/0 e 0 to 0 dl 1788619928 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619917.947087:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be2680 with x1875503571480960 00000100:00100000:1.0:1788619917.947127:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 643 00000100:00100000:1.0:1788619917.947138:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2354:x1875503571480960:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619917.948656:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5018700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589128832:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619917.948680:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619917.948966:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be3b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571480832:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 10746us (10947us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619917.948984:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 345 00000100:00100000:1.0:1788619917.978436:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2354:x1875503571480960:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 31285us (31561us total) trans 0 rc 995328/995328 00000100:00100000:1.0:1788619917.978454:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 643 00002000:00100000:1.0:1788619917.978470:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be2680 x1875503571480960/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:13/0 lens 488/440 e 0 to 0 dl 1788619928 ref 1 fl Complete:/600/0 rc 995328/995328 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619917.978483:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be2680 with x1875503571480960 00000100:00100000:0.0:1788619917.982825:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619917.982965:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571481088 00000100:00100000:1.0:1788619917.982975:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571481088 00000100:00100000:1.0:1788619917.983058:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 811 00000100:00100000:1.0:1788619917.983069:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df387100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571481088:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619917.983090:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619917.983108:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d40dc9c9/0x1cd7fe5368734eb7 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c235 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619917.983168:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed63480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589128960:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619917.983251:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a240 going to sleep for 11 seconds 00000100:00100000:1.0:1788619917.985900:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed63480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589128960:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619917.985944:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d40dc9c9/0x1cd7fe5368734eb7 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c235 expref: 10 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.985971:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d40dc9c9/0x1cd7fe5368734eb7 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c235 expref: 10 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.985986:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752c00, rc 0) 00000100:00100000:1.0:1788619917.986056:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df387100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571481088:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2988us (3239us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619917.986066:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 811 00000100:00100000:3.0:1788619917.989994:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619917.990158:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571481216 00000100:00100000:1.0:1788619917.990168:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571481216 00000100:00100000:1.0:1788619917.990227:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 812 00000100:00100000:1.0:1788619917.990238:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5018700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2355:x1875503571481216:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619917.990255:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619917.990273:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001786a1ee/0x1cd7fe5368734ebe lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c23c expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619917.990387:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed63480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589129088:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619917.990470:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a240 going to sleep for 11 seconds 00000100:00100000:1.0:1788619917.993371:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed63480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589129088:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619917.993424:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001786a1ee/0x1cd7fe5368734ebe lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c23c expref: 10 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.993453:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001786a1ee/0x1cd7fe5368734ebe lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c23c expref: 10 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619917.993468:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5753200, rc 0) 00000100:00100000:1.0:1788619917.993548:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5018700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2355:x1875503571481216:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3310us (3562us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619917.993558:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 812 00000100:00100000:2.0:1788619917.998394:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619917.998569:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571481344 00000100:00100000:0.0:1788619917.998581:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571481344 00000100:00100000:0.0:1788619917.998632:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1036 00000100:00100000:0.0:1788619917.998642:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571481344:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619917.998664:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619917.998683:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767aa00/0x1cd7fe5368734ec5 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c243 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619917.998717:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767aa00/0x1cd7fe5368734ec5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c243 expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788619917.998730:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:0.0:1788619917.998737:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767aa00/0x1cd7fe5368734ec5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c243 expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619917.998775:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619917.998778:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619917.998782:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767aa00, rc 0) 00000100:00100000:0.0:1788619917.998849:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571481344:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 208us (467us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619917.998855:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1036 00000100:00100000:3.0:1788619918.003157:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619918.003289:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571481472 00000100:00100000:2.0:1788619918.003299:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571481472 00002000:00100000:2.0:1788619918.003395:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5018e00 x1875503571481472/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:14/0 lens 488/0 e 0 to 0 dl 1788619929 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619918.003405:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5018e00 with x1875503571481472 00000100:00100000:2.0:1788619918.003434:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 644 00000100:00100000:2.0:1788619918.003442:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5018e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2353:x1875503571481472:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619918.003725:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5018e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2353:x1875503571481472:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 283us (577us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619918.003733:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 644 00002000:00100000:2.0:1788619918.003739:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5018e00 x1875503571481472/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:14/0 lens 488/4536 e 0 to 0 dl 1788619929 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619918.003747:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5018e00 with x1875503571481472 00000100:00080000:0.0:1788619918.004737:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1788619918.007466:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619918.008513:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571481600 00000100:00100000:3.0:1788619918.008526:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571481600 00002000:00100000:3.0:1788619918.008578:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fed63480 x1875503571481600/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:14/0 lens 488/0 e 0 to 0 dl 1788619929 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619918.008599:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fed63480 with x1875503571481600 00000100:00100000:3.0:1788619918.008639:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 645 00000100:00100000:3.0:1788619918.008650:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed63480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2353:x1875503571481600:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619918.008940:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed63480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2353:x1875503571481600:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 291us (1482us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619918.008949:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 645 00002000:00100000:3.0:1788619918.008956:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fed63480 x1875503571481600/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:14/0 lens 488/4536 e 0 to 0 dl 1788619929 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619918.008964:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fed63480 with x1875503571481600 00000100:00100000:3.0:1788619918.012929:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619918.013022:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571481728 00000100:00100000:1.0:1788619918.013031:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571481728 00002000:00100000:1.0:1788619918.013089:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5018380 x1875503571481728/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:14/0 lens 488/0 e 0 to 0 dl 1788619929 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619918.013101:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5018380 with x1875503571481728 00000100:00100000:1.0:1788619918.013137:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 646 00000100:00100000:1.0:1788619918.013148:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5018380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2353:x1875503571481728:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619918.013446:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5018380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2353:x1875503571481728:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 299us (526us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619918.013473:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 646 00002000:00100000:1.0:1788619918.013480:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5018380 x1875503571481728/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:14/0 lens 488/4536 e 0 to 0 dl 1788619929 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619918.013488:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5018380 with x1875503571481728 00000100:00100000:0.0:1788619918.017258:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619918.017558:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571481856 00000100:00100000:2.0:1788619918.017571:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571481856 00000100:00100000:2.0:1788619918.017658:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 813 00000100:00100000:2.0:1788619918.017670:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df384e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2352:x1875503571481856:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619918.017693:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619918.017716:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dfdc9508/0x1cd7fe5368734ecc lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c24a expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619918.017820:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589129216:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619918.017919:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc600 going to sleep for 11 seconds 00000100:00080000:0.0:1788619918.018057:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:2.0:1788619918.020312:0:6698:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619916 -> 1788619917 00000100:00100000:2.0:1788619918.020359:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589129216:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619918.020394:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000dfdc9508/0x1cd7fe5368734ecc lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c24a expref: 10 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.020419:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dfdc9508/0x1cd7fe5368734ecc lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c24a expref: 10 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.020433:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8078200, rc 0) 00000100:00100000:2.0:1788619918.020488:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df384e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2352:x1875503571481856:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2820us (3242us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619918.020639:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 813 00000100:00100000:3.0:1788619918.023748:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619918.023932:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571481984 00000100:00100000:0.0:1788619918.023943:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571481984 00000100:00100000:0.0:1788619918.024027:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 814 00000100:00100000:0.0:1788619918.024049:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5019f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2353:x1875503571481984:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619918.024072:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619918.024092:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f4734b2f/0x1cd7fe5368734ed3 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c251 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619918.024165:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df385c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589129344:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619918.024254:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c2c5e300 going to sleep for 11 seconds 00000100:00100000:3.0:1788619918.026866:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df385c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589129344:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619918.026912:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f4734b2f/0x1cd7fe5368734ed3 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c251 expref: 10 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619918.026936:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f4734b2f/0x1cd7fe5368734ed3 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c251 expref: 10 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619918.026970:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767a600, rc 0) 00000100:00100000:3.0:1788619918.027027:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5019f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2353:x1875503571481984:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2990us (3290us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619918.027042:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 814 00000100:00100000:2.0:1788619918.031876:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619918.032206:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571482112 00000100:00100000:1.0:1788619918.032217:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571482112 00002000:00100000:1.0:1788619918.032265:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be1500 x1875503571482112/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:14/0 lens 488/0 e 0 to 0 dl 1788619929 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619918.032277:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be1500 with x1875503571482112 00000100:00100000:1.0:1788619918.032315:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 647 00000100:00100000:1.0:1788619918.032361:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2352:x1875503571482112:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619918.032656:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2352:x1875503571482112:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 332us (791us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619918.032664:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 647 00002000:00100000:1.0:1788619918.032671:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be1500 x1875503571482112/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:14/0 lens 488/4536 e 0 to 0 dl 1788619929 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619918.032679:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be1500 with x1875503571482112 00000100:00100000:2.0:1788619918.036469:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619918.036613:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571482240 00000100:00100000:1.0:1788619918.036623:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571482240 00002000:00100000:1.0:1788619918.036669:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be3100 x1875503571482240/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:14/0 lens 488/0 e 0 to 0 dl 1788619929 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619918.036681:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be3100 with x1875503571482240 00000100:00100000:1.0:1788619918.036715:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 648 00000100:00100000:1.0:1788619918.036723:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2353:x1875503571482240:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619918.037012:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2353:x1875503571482240:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 289us (552us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619918.037020:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 648 00002000:00100000:1.0:1788619918.037027:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be3100 x1875503571482240/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:14/0 lens 488/4536 e 0 to 0 dl 1788619929 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619918.037043:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be3100 with x1875503571482240 00000100:00100000:0.0:1788619918.040203:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619918.040297:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571482368 00000100:00100000:3.0:1788619918.040307:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571482368 00002000:00100000:3.0:1788619918.040398:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df386a00 x1875503571482368/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:14/0 lens 488/0 e 0 to 0 dl 1788619929 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619918.040409:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df386a00 with x1875503571482368 00000100:00100000:3.0:1788619918.040447:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 649 00000100:00100000:3.0:1788619918.040457:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df386a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2352:x1875503571482368:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619918.040782:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df386a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2352:x1875503571482368:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 325us (591us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619918.040789:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 649 00002000:00100000:3.0:1788619918.040796:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df386a00 x1875503571482368/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:14/0 lens 488/4536 e 0 to 0 dl 1788619929 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619918.040804:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df386a00 with x1875503571482368 00000100:00100000:0.0:1788619918.044880:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619918.044986:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571482496 00000100:00100000:2.0:1788619918.044996:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571482496 00002000:00100000:2.0:1788619918.045058:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df384000 x1875503571482496/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:14/0 lens 488/0 e 0 to 0 dl 1788619929 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619918.045070:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df384000 with x1875503571482496 00000100:00100000:2.0:1788619918.045100:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 650 00000100:00100000:2.0:1788619918.045110:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df384000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571482496:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619918.045453:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df384000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571482496:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 343us (582us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619918.045460:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 650 00002000:00100000:2.0:1788619918.045468:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df384000 x1875503571482496/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:14/0 lens 488/4536 e 0 to 0 dl 1788619929 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619918.045477:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df384000 with x1875503571482496 00000100:00100000:2.0:1788619918.048622:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619918.048791:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571482624 00000100:00100000:1.0:1788619918.048800:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571482624 00002000:00100000:1.0:1788619918.048849:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be2d80 x1875503571482624/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:14/0 lens 488/0 e 0 to 0 dl 1788619929 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619918.048860:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be2d80 with x1875503571482624 00000100:00100000:1.0:1788619918.048896:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 651 00000100:00100000:1.0:1788619918.048906:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571482624:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619918.049194:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571482624:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 288us (585us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619918.049202:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 651 00002000:00100000:1.0:1788619918.049210:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be2d80 x1875503571482624/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:14/0 lens 488/4536 e 0 to 0 dl 1788619929 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619918.049218:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be2d80 with x1875503571482624 00000100:00100000:0.0:1788619918.052885:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619918.052977:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571482752 00000100:00100000:3.0:1788619918.052989:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571482752 00002000:00100000:3.0:1788619918.053046:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df385c00 x1875503571482752/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:14/0 lens 488/0 e 0 to 0 dl 1788619929 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619918.053057:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df385c00 with x1875503571482752 00000100:00100000:3.0:1788619918.053098:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 652 00000100:00100000:3.0:1788619918.053117:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df385c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571482752:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619918.054270:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df385c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571482752:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 1148us (1389us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619918.054280:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 652 00002000:00100000:2.0:1788619918.054290:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df385c00 x1875503571482752/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:14/0 lens 488/4536 e 0 to 0 dl 1788619929 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619918.054299:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df385c00 with x1875503571482752 00000100:00100000:3.0:1788619918.057385:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619918.057550:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571482880 00000100:00100000:2.0:1788619918.057560:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571482880 00002000:00100000:2.0:1788619918.057618:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c50c7100 x1875503571482880/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:14/0 lens 488/0 e 0 to 0 dl 1788619929 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619918.057630:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c50c7100 with x1875503571482880 00000100:00100000:2.0:1788619918.057657:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 653 00000100:00100000:2.0:1788619918.057665:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571482880:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619918.057922:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571482880:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 256us (555us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619918.057930:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 653 00002000:00100000:2.0:1788619918.057937:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c50c7100 x1875503571482880/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:14/0 lens 488/4536 e 0 to 0 dl 1788619929 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619918.057959:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c50c7100 with x1875503571482880 00000100:00100000:3.0:1788619918.061690:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619918.061802:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571483008 00000100:00100000:2.0:1788619918.061812:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571483008 00002000:00100000:2.0:1788619918.061861:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c50c7800 x1875503571483008/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:14/0 lens 488/0 e 0 to 0 dl 1788619929 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619918.061873:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c50c7800 with x1875503571483008 00000100:00100000:2.0:1788619918.061912:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 654 00000100:00100000:2.0:1788619918.061921:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571483008:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619918.062202:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571483008:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 280us (707us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619918.062209:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 654 00002000:00100000:2.0:1788619918.062215:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c50c7800 x1875503571483008/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:14/0 lens 488/4536 e 0 to 0 dl 1788619929 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619918.062222:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c50c7800 with x1875503571483008 00000100:00100000:0.0:1788619918.066522:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619918.066665:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571483136 00000100:00100000:1.0:1788619918.066674:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571483136 00002000:00100000:1.0:1788619918.066719:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6003800 x1875503571483136/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:14/0 lens 488/0 e 0 to 0 dl 1788619929 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619918.066729:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6003800 with x1875503571483136 00000100:00100000:1.0:1788619918.066792:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 655 00000100:00100000:1.0:1788619918.066802:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6003800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571483136:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619918.067314:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6003800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571483136:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 510us (799us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619918.067352:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 655 00002000:00100000:1.0:1788619918.067361:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6003800 x1875503571483136/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:14/0 lens 488/4536 e 0 to 0 dl 1788619929 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619918.067370:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6003800 with x1875503571483136 00000100:00100000:0.0:1788619918.071412:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619918.071546:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571483264 00000100:00100000:1.0:1788619918.071557:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571483264 00002000:00100000:1.0:1788619918.071650:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6000700 x1875503571483264/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:14/0 lens 488/0 e 0 to 0 dl 1788619929 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619918.071661:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6000700 with x1875503571483264 00000100:00100000:1.0:1788619918.071731:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 656 00000100:00100000:1.0:1788619918.071741:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6000700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571483264:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619918.072086:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6000700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571483264:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 345us (682us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619918.072094:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 656 00002000:00100000:1.0:1788619918.072101:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6000700 x1875503571483264/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:14/0 lens 488/4536 e 0 to 0 dl 1788619929 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619918.072109:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6000700 with x1875503571483264 00000100:00100000:0.0:1788619918.075224:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619918.075323:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571483392 00000100:00100000:1.0:1788619918.075387:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571483392 00002000:00100000:1.0:1788619918.075436:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6001c00 x1875503571483392/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:14/0 lens 488/0 e 0 to 0 dl 1788619929 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619918.075466:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6001c00 with x1875503571483392 00000100:00100000:1.0:1788619918.075595:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 657 00000100:00100000:1.0:1788619918.075606:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6001c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571483392:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619918.075889:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6001c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571483392:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 283us (675us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619918.075897:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 657 00002000:00100000:1.0:1788619918.075904:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6001c00 x1875503571483392/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:14/0 lens 488/4536 e 0 to 0 dl 1788619929 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619918.075912:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6001c00 with x1875503571483392 00000100:00100000:2.0:1788619918.079377:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619918.080278:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571483520 00000100:00100000:0.0:1788619918.080304:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571483520 00002000:00100000:0.0:1788619918.080371:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be0a80 x1875503571483520/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:14/0 lens 488/0 e 0 to 0 dl 1788619929 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619918.080382:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be0a80 with x1875503571483520 00000100:00100000:0.0:1788619918.080408:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 658 00000100:00100000:0.0:1788619918.080417:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571483520:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619918.080826:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571483520:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 409us (1458us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619918.080835:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 658 00002000:00100000:0.0:1788619918.080842:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be0a80 x1875503571483520/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:14/0 lens 488/4536 e 0 to 0 dl 1788619929 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619918.080851:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be0a80 with x1875503571483520 00000100:00100000:0.0:1788619918.083968:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619918.084107:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571483648 00000100:00100000:2.0:1788619918.084117:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571483648 00002000:00100000:2.0:1788619918.084163:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6002d80 x1875503571483648/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:14/0 lens 488/0 e 0 to 0 dl 1788619929 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619918.084175:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6002d80 with x1875503571483648 00000100:00100000:2.0:1788619918.084210:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 659 00000100:00100000:2.0:1788619918.084221:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6002d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571483648:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619918.084669:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6002d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571483648:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 448us (709us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619918.084689:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 659 00002000:00100000:2.0:1788619918.084697:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6002d80 x1875503571483648/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:14/0 lens 488/4536 e 0 to 0 dl 1788619929 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619918.084707:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6002d80 with x1875503571483648 00000100:00100000:2.0:1788619918.088299:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619918.088438:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571483776 00000100:00100000:0.0:1788619918.088448:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571483776 00000100:00100000:0.0:1788619918.088519:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1037 00000100:00100000:0.0:1788619918.088529:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:14908:x1875503571483776:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619918.088554:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619918.088578:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767ae00/0x1cd7fe5368734eda lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c258 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.088670:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767ba00/0x1cd7fe5368734ee1 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.088685:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767ba00/0x1cd7fe5368734ee1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.088716:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767ba00/0x1cd7fe5368734ee1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619918.088745:0:5846:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 994373 00000004:00200000:0.0:1788619918.088750:0:5846:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619918.088781:0:5846:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767ba00/0x1cd7fe5368734ee1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.088808:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767ae00/0x1cd7fe5368734eda lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c258 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.088828:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767ba00/0x1cd7fe5368734ee1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c258 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.088837:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619918.088840:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619918.088844:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767ba00, rc 0) 00000100:00100000:0.0:1788619918.088902:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+13:14908:x1875503571483776:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 375us (616us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619918.088908:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1037 00000100:00100000:0.0:1788619918.093862:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619918.094006:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571483904 00000100:00100000:1.0:1788619918.094017:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571483904 00000100:00100000:1.0:1788619918.094085:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1038 00000100:00100000:1.0:1788619918.094095:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6002300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+13:14908:x1875503571483904:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619918.094118:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619918.094121:0:5844:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:1.0:1788619918.094170:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767aa00/0x1cd7fe5368734ec5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c243 expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619918.094187:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619918.094193:0:5844:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788619918.094205:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753200/0x1cd7fe5368734ee8 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c25f expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619918.094230:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753a00/0x1cd7fe5368734eb0 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c22e expref: 23 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619918.094259:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753200/0x1cd7fe5368734ee8 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c25f expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619918.094293:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753a00/0x1cd7fe5368734eb0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c22e expref: 23 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619918.094307:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753a00/0x1cd7fe5368734eb0 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c22e expref: 23 pid: 5844 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619918.094321:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed63800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589129472:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619918.094437:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619918.101198:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619918.101307:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed63800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589129472:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619918.101379:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753200/0x1cd7fe5368734ee8 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c25f expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619918.101399:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619918.101403:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619918.101407:0:5844:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753200/0x1cd7fe5368734ee8 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c25f expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619918.101420:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5753200, rc 0) 00000100:00100000:1.0:1788619918.101471:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6002300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+13:14908:x1875503571483904:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7377us (7617us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619918.101481:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1038 00000100:00100000:0.0:1788619918.101710:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571484032 00000100:00100000:0.0:1788619918.101716:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571484032 00010000:00010000:0.0:1788619918.101776:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753a00/0x1cd7fe5368734eb0 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c22e expref: 24 pid: 5844 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619918.101816:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 346 00000100:00100000:0.0:1788619918.101826:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa15880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2354:x1875503571484032:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619918.101856:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619918.101884:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753a00/0x1cd7fe5368734eb0 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c22e expref: 24 pid: 5844 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.101898:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753a00/0x1cd7fe5368734eb0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c22e expref: 24 pid: 5844 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.101921:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753a00/0x1cd7fe5368734eb0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c22e expref: 24 pid: 5844 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.101937:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619918.101942:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753200/0x1cd7fe5368734ee8 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c25f expref: 12 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.101953:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753200/0x1cd7fe5368734ee8 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c25f expref: 12 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.101964:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753200/0x1cd7fe5368734ee8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c25f expref: 12 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.102002:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753200/0x1cd7fe5368734ee8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c25f expref: 12 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619918.102016:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6003b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589129600:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619918.102084:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c2c5ea80 going to sleep for 11 seconds 00000100:00080000:1.0:1788619918.102686:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1788619918.105284:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6003b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589129600:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619918.105366:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619918.105430:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa15880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+23:2354:x1875503571484032:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3604us (4240us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619918.105442:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 346 00000100:00100000:2.0:1788619918.108847:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619918.108987:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571484160 00000100:00100000:3.0:1788619918.109006:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571484160 00002000:00100000:3.0:1788619918.109069:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be3480 x1875503571484160/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:14/0 lens 488/0 e 0 to 0 dl 1788619929 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619918.109080:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be3480 with x1875503571484160 00000100:00100000:3.0:1788619918.109110:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 660 00000100:00100000:3.0:1788619918.109120:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+13:2355:x1875503571484160:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619918.109432:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+13:2355:x1875503571484160:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 313us (595us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619918.109439:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 660 00002000:00100000:3.0:1788619918.109446:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be3480 x1875503571484160/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:14/0 lens 488/4536 e 0 to 0 dl 1788619929 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619918.109454:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be3480 with x1875503571484160 00000100:00100000:3.0:1788619918.112873:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619918.112978:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571484288 00000100:00100000:1.0:1788619918.112988:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571484288 00002000:00100000:1.0:1788619918.113041:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78dfa42680 x1875503571484288/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:14/0 lens 488/0 e 0 to 0 dl 1788619929 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619918.113052:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78dfa42680 with x1875503571484288 00000100:00100000:1.0:1788619918.113089:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 661 00000100:00100000:1.0:1788619918.113099:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa42680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+13:2353:x1875503571484288:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619918.113393:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa42680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+13:2353:x1875503571484288:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 295us (530us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619918.113401:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 661 00002000:00100000:1.0:1788619918.113408:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78dfa42680 x1875503571484288/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:14/0 lens 488/4536 e 0 to 0 dl 1788619929 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619918.113416:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78dfa42680 with x1875503571484288 00010000:00010000:1.0:1788619918.115472:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788619918.118639:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619918.118746:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571484416 00000100:00100000:2.0:1788619918.118758:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571484416 00000100:00100000:2.0:1788619918.118832:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1039 00000100:00100000:2.0:1788619918.118842:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6003b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+23:2355:x1875503571484416:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619918.118862:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619918.118881:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368734eef lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c266 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.118917:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753200/0x1cd7fe5368734ee8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c25f expref: 12 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619918.118958:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be1c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589129728:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619918.119051:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc300 going to sleep for 11 seconds 00000100:00100000:2.0:1788619918.121983:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000004:00010000:3.0:1788619918.122175:0:5845:0:(mdt_lvb.c:183:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime: 1788619917 -> 1788619918 00000004:00010000:3.0:1788619918.122179:0:5845:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619916 -> 1788619917 00000004:00010000:3.0:1788619918.122182:0:5845:0:(mdt_lvb.c:195:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime: 1788619917 -> 1788619918 00000100:00100000:3.0:1788619918.122205:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be1c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589129728:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619918.122228:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:3.0:1788619918.122246:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368734eef lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c266 expref: 24 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619918.122286:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619918.122289:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619918.122295:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368734eef lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c266 expref: 24 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619918.122310:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079400, rc 0) 00000100:00100000:3.0:1788619918.122400:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6003b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+23:2355:x1875503571484416:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3560us (3771us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619918.122407:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1039 00000100:00100000:1.0:1788619918.122949:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571484544 00000100:00100000:1.0:1788619918.122957:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571484544 00000100:00100000:1.0:1788619918.123009:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 815 00000100:00100000:1.0:1788619918.123019:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2355:x1875503571484544:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619918.123042:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619918.123059:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c03a8843/0x1cd7fe5368734ef6 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c26d expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619918.123117:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed63800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589129856:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619918.123172:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a6c0 going to sleep for 11 seconds 00002000:00010000:1.0:1788619918.125508:0:6696:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime: 1788619917 -> 1788619918 00002000:00010000:1.0:1788619918.125516:0:6696:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime: 1788619917 -> 1788619918 00000100:00100000:1.0:1788619918.125547:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed63800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589129856:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619918.125580:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c03a8843/0x1cd7fe5368734ef6 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c26d expref: 10 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619918.125682:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c03a8843/0x1cd7fe5368734ef6 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c26d expref: 10 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619918.125692:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752a00, rc 0) 00000100:00100000:1.0:1788619918.125740:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2355:x1875503571484544:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2723us (3768us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619918.125746:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 815 00000100:00100000:3.0:1788619918.127623:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619918.127652:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619918.127773:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571484672 00000100:00100000:2.0:1788619918.127781:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571484672 00000100:00100000:2.0:1788619918.127832:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1040 00000100:00100000:1.0:1788619918.127836:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571484800 00000100:00100000:1.0:1788619918.127840:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571484800 00000100:00100000:2.0:1788619918.127842:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa41c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+23:2355:x1875503571484672:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619918.127859:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1788619918.127871:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 816 00000100:00100000:1.0:1788619918.127877:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b16300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571484800:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619918.127884:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619918.127890:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe5368734efd lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c274 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619918.127893:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd2e9c29/0x1cd7fe5368734f04 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c27b expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.127932:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753200/0x1cd7fe5368734ee8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c25f expref: 12 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619918.127971:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be0e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589129984:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619918.128055:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc840 going to sleep for 11 seconds 00000100:00100000:1.0:1788619918.128404:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed63800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589130112:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619918.128450:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a6c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619918.130843:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be0e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589129984:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619918.130868:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:2.0:1788619918.130884:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe5368734efd lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c274 expref: 24 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.130912:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619918.130915:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619918.130921:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe5368734efd lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c274 expref: 24 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.130936:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079c00, rc 0) 00000100:00100000:2.0:1788619918.130987:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa41c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+23:2355:x1875503571484672:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3147us (3370us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619918.130995:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1040 00000100:00100000:1.0:1788619918.133849:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed63800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589130112:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619918.133889:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bd2e9c29/0x1cd7fe5368734f04 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c27b expref: 10 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619918.133948:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd2e9c29/0x1cd7fe5368734f04 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c27b expref: 10 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619918.133963:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752600, rc 0) 00000100:00100000:1.0:1788619918.134020:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b16300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571484800:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6142us (6365us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619918.134029:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 816 00000100:00100000:0.0:1788619918.136554:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619918.136800:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571484928 00000100:00100000:1.0:1788619918.136809:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571484928 00000100:00100000:1.0:1788619918.136870:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1041 00000100:00100000:1.0:1788619918.136880:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6003480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+23:2355:x1875503571484928:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619918.136899:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619918.136928:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752a00/0x1cd7fe5368734f0b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c282 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619918.136966:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753200/0x1cd7fe5368734ee8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c25f expref: 12 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619918.137008:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed63800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589130240:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619918.137090:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619918.145014:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619918.146176:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571485056 00000100:00100000:2.0:1788619918.146189:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571485056 00000100:00100000:1.0:1788619918.146368:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed63800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589130240:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619918.146397:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:1.0:1788619918.146417:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752a00/0x1cd7fe5368734f0b lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c282 expref: 24 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619918.146459:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619918.146463:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619918.146469:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752a00/0x1cd7fe5368734f0b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c282 expref: 24 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619918.146483:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752a00, rc 0) 00000100:00100000:1.0:1788619918.146545:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6003480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+23:2355:x1875503571484928:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 9666us (9999us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619918.146554:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1041 00000100:00100000:2.0:1788619918.147738:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 817 00000100:00100000:2.0:1788619918.147754:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b14a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2355:x1875503571485056:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619918.147780:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619918.147805:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000855993/0x1cd7fe5368734f12 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c289 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619918.147860:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589130368:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619918.148170:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc840 going to sleep for 11 seconds 00000100:00100000:2.0:1788619918.150308:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589130368:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619918.150418:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000000855993/0x1cd7fe5368734f12 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c289 expref: 10 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.150442:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000855993/0x1cd7fe5368734f12 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c289 expref: 10 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.150453:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079c00, rc 0) 00000100:00100000:2.0:1788619918.150508:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b14a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2355:x1875503571485056:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2759us (5504us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619918.150517:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 817 00000100:00100000:0.0:1788619918.152771:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619918.152845:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619918.152905:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571485312 00000100:00100000:3.0:1788619918.152913:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571485312 00000100:00100000:2.0:1788619918.153004:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571485184 00000100:00100000:2.0:1788619918.153009:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571485184 00000100:00100000:2.0:1788619918.153049:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1042 00000100:00100000:2.0:1788619918.153060:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6001180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+23:2354:x1875503571485184:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619918.153079:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619918.153097:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe5368734f19 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c290 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.153130:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753200/0x1cd7fe5368734ee8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c25f expref: 12 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619918.153194:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be0e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589130496:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619918.153262:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc840 going to sleep for 11 seconds 00000100:00100000:3.0:1788619918.153303:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 818 00000100:00100000:3.0:1788619918.153311:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6000a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571485312:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619918.153319:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619918.153360:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000485aa315/0x1cd7fe5368734f20 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c297 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619918.153392:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b17100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589130624:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619918.153417:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:0.0:1788619918.157534:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b17100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589130624:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619918.157575:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000485aa315/0x1cd7fe5368734f20 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c297 expref: 10 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.157601:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000485aa315/0x1cd7fe5368734f20 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c297 expref: 10 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.157615:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683cc00, rc 0) 00000100:00100000:0.0:1788619918.157682:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6000a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571485312:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4371us (4929us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619918.157691:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 818 00000100:00100000:2.0:1788619918.157832:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be0e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589130496:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619918.157846:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:2.0:1788619918.157858:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe5368734f19 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c290 expref: 24 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.157870:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619918.157873:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619918.157877:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe5368734f19 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c290 expref: 24 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.157888:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079000, rc 0) 00000100:00100000:2.0:1788619918.157910:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6001180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+23:2354:x1875503571485184:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4853us (5065us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619918.158179:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1042 00000100:00100000:3.0:1788619918.162258:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619918.162361:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619918.162663:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571485440 00000100:00100000:2.0:1788619918.162673:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571485440 00000100:00100000:2.0:1788619918.162730:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1043 00000100:00100000:2.0:1788619918.162739:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b15500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+23:2353:x1875503571485440:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619918.162759:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619918.162789:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe5368734f27 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c29e expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.162824:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753200/0x1cd7fe5368734ee8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c25f expref: 12 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619918.162876:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be0e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589130752:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619918.162958:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc840 going to sleep for 11 seconds 00000100:00100000:0.0:1788619918.163020:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571485568 00000100:00100000:0.0:1788619918.163025:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571485568 00000100:00100000:0.0:1788619918.163069:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 819 00000100:00100000:0.0:1788619918.163077:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b16680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2353:x1875503571485568:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619918.163085:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619918.163097:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006989f8b0/0x1cd7fe5368734f2e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c2a5 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619918.163140:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6000380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589130880:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619918.163165:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c2c5ea80 going to sleep for 11 seconds 00000100:00100000:2.0:1788619918.166846:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be0e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589130752:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619918.166872:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:2.0:1788619918.166890:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe5368734f27 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c29e expref: 24 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.166909:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619918.166912:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619918.166927:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe5368734f27 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c29e expref: 24 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.166942:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079c00, rc 0) 00000100:00100000:2.0:1788619918.166994:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b15500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+23:2353:x1875503571485440:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4255us (4745us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619918.167003:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1043 00000100:00100000:0.0:1788619918.167139:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6000380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589130880:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619918.167159:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000006989f8b0/0x1cd7fe5368734f2e lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2a5 expref: 10 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.167175:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006989f8b0/0x1cd7fe5368734f2e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2a5 expref: 10 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.167184:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767bc00, rc 0) 00000100:00100000:0.0:1788619918.167205:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b16680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2353:x1875503571485568:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4130us (4844us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619918.167212:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 819 00000100:00100000:1.0:1788619918.170743:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619918.170828:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619918.170894:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571485824 00000100:00100000:2.0:1788619918.170904:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571485824 00000100:00100000:2.0:1788619918.170971:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 820 00000100:00100000:2.0:1788619918.170982:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed63b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2352:x1875503571485824:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619918.171001:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619918.171021:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008447b954/0x1cd7fe5368734f35 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c2b3 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619918.171092:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589131008:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619918.171192:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc840 going to sleep for 11 seconds 00000100:00100000:0.0:1788619918.172179:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571485696 00000100:00100000:0.0:1788619918.172188:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571485696 00000100:00100000:0.0:1788619918.172275:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1044 00000100:00100000:0.0:1788619918.172284:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed63800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+23:2352:x1875503571485696:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619918.172299:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619918.172315:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767b000/0x1cd7fe5368734f3c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c2ac expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.172364:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753200/0x1cd7fe5368734ee8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c25f expref: 12 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619918.172413:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6000380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589131136:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619918.172479:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c2c5ea80 going to sleep for 11 seconds 00000100:00100000:3.0:1788619918.173484:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589131008:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619918.173533:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008447b954/0x1cd7fe5368734f35 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2b3 expref: 10 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619918.173555:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008447b954/0x1cd7fe5368734f35 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2b3 expref: 10 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619918.173570:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079000, rc 0) 00000100:00100000:3.0:1788619918.173618:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed63b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2352:x1875503571485824:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2638us (2789us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619918.173625:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 820 00000100:00100000:0.0:1788619918.178201:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6000380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589131136:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619918.178228:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:0.0:1788619918.178247:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767b000/0x1cd7fe5368734f3c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2ac expref: 24 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.178266:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619918.178269:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619918.178275:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767b000/0x1cd7fe5368734f3c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2ac expref: 24 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.178289:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c767b000, rc 0) 00000100:00100000:0.0:1788619918.178669:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed63800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+23:2352:x1875503571485696:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6384us (7940us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619918.178679:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1044 00000100:00100000:1.0:1788619918.183058:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619918.183163:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571486080 00000100:00100000:2.0:1788619918.183173:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571486080 00000100:00100000:2.0:1788619918.183242:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1045 00000100:00100000:2.0:1788619918.183253:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed60380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+13:14908:x1875503571486080:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:2.0:1788619918.183308:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619918.183363:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767ba00/0x1cd7fe5368734ee1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c258 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.183382:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619918.183386:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788619918.183590:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe5368734f43 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.183605:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767ac00/0x1cd7fe5368734e8d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c20b expref: 22 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.183614:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe5368734f43 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.183647:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767ac00/0x1cd7fe5368734e8d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c20b expref: 22 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.183662:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767ac00/0x1cd7fe5368734e8d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c20b expref: 22 pid: 5844 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619918.183674:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be0e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589131264:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619918.183757:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccd80 going to sleep for 11 seconds 00000100:00100000:1.0:1788619918.187693:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619918.187783:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571486208 00000100:00100000:0.0:1788619918.187791:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571486208 00010000:00010000:0.0:1788619918.187832:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767ac00/0x1cd7fe5368734e8d lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c20b expref: 23 pid: 5844 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619918.187870:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 347 00000100:00100000:0.0:1788619918.187879:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed60000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+23:2355:x1875503571486208:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619918.187893:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619918.187900:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767ac00/0x1cd7fe5368734e8d lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c20b expref: 23 pid: 5844 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.188183:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767ac00/0x1cd7fe5368734e8d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c20b expref: 23 pid: 5844 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.188207:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619918.188213:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe5368734f43 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.188223:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe5368734f43 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.188231:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe5368734f43 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.188250:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619918.188370:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed60000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+23:2355:x1875503571486208:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 491us (684us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619918.188377:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 347 00010000:00010000:2.0:1788619918.188563:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c767ac00/0x1cd7fe5368734e8d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c20b expref: 22 pid: 5844 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619918.188579:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be0e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589131264:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619918.188600:0:5845:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe5368734f43 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.188609:0:5845:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe5368734f43 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.188616:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe5368734f43 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788619918.188729:0:5845:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967691 00000004:00010000:2.0:1788619918.188791:0:5845:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 994373 -> 994373 00000004:00010000:2.0:1788619918.188794:0:5845:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619918 -> 1788619918 00000004:00010000:2.0:1788619918.188797:0:5845:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619917 -> 1788619916 00000004:00010000:2.0:1788619918.188799:0:5845:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619918 -> 1788619918 00000004:00010000:2.0:1788619918.188801:0:5845:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 518 -> 518 00000004:00010000:2.0:1788619918.188808:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000009cceb84d reply state 00000000f0dca011 transno 4294967692 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe5368734f43 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619918.188817:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788619918.188881:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed60380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+15:14908:x1875503571486080:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 5630us (5833us total) trans 4294967692 rc 0/0 00000100:00100000:2.0:1788619918.188888:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1045 00000020:00080000:2.0:1788619918.188892:0:5845:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:2.0:1788619918.190953:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe5368734f43 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.190987:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe5368734f43 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.190995:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe5368734f43 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.191001:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78c8079c00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:2.0:1788619918.191006:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe5368734f43 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.191014:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe5368734f43 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.191029:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619918.191032:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788619918.191036:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe5368734f43 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.191043:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe5368734f43 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619918.192148:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619918.192251:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571486336 00000100:00100000:0.0:1788619918.192261:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571486336 00002000:00010000:0.0:1788619918.192372:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1646471->18446744073709551615) 00002000:00100000:0.0:1788619918.192378:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fed61c00 with x1875503571486336 ext(1646471->18446744073709551615) 00002000:00010000:0.0:1788619918.192389:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fed61c00. 00000100:00100000:0.0:1788619918.192418:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 456 00000100:00100000:0.0:1788619918.192427:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed61c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571486336:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:0.0:1788619918.192671:0:6702:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 0 -> 1646471 00002000:00010000:0.0:1788619918.192675:0:6702:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619918 -> 1788619918 00002000:00010000:0.0:1788619918.192677:0:6702:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619917 -> 1788619916 00002000:00010000:0.0:1788619918.192690:0:6702:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619918 -> 1788619918 00002000:00010000:0.0:1788619918.192692:0:6702:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 135 -> 135 00000100:00100000:0.0:1788619918.192752:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed61c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2352:x1875503571486336:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 326us (610us total) trans 4294967434 rc 0/0 00000100:00100000:0.0:1788619918.192760:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 456 00002000:00010000:0.0:1788619918.192775:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1646471->18446744073709551615) 00002000:00100000:0.0:1788619918.192778:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fed61c00 with x1875503571486336 ext(1646471->18446744073709551615) 00002000:00010000:0.0:1788619918.192783:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fed61c00. 00000100:00080000:1.0:1788619918.193759:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1788619918.196249:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619918.196354:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571486464 00000100:00100000:2.0:1788619918.196364:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571486464 00000100:00100000:2.0:1788619918.196462:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1046 00000100:00100000:2.0:1788619918.196475:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6000380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+14:14908:x1875503571486464:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619918.196508:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619918.196528:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079600/0x1cd7fe5368734f4a lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c2ba expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.196624:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368734f51 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.196654:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368734f51 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.196667:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368734f51 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788619918.196698:0:11397:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1646471 00000004:00200000:2.0:1788619918.196702:0:11397:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788619918.196719:0:11397:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368734f51 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.196746:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079600/0x1cd7fe5368734f4a lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2ba expref: 16 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.196775:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368734f51 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2ba expref: 15 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.196784:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619918.196787:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788619918.196790:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079400, rc 0) 00000100:00100000:2.0:1788619918.196846:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6000380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+15:14908:x1875503571486464:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 373us (608us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619918.196853:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1046 00000100:00100000:3.0:1788619918.204910:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619918.206631:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571486592 00000100:00100000:1.0:1788619918.206647:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571486592 00000100:00100000:1.0:1788619918.206757:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1047 00000100:00100000:1.0:1788619918.206779:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b14e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+22:14908:x1875503571486592:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619918.206802:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619918.206826:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753000/0x1cd7fe5368734f58 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c2c1 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619918.206855:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753200/0x1cd7fe5368734ee8 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c25f expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619918.206885:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753000/0x1cd7fe5368734f58 lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2c1 expref: 23 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619918.206936:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753200/0x1cd7fe5368734ee8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c25f expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619918.206950:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753200/0x1cd7fe5368734ee8 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c25f expref: 14 pid: 5844 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619918.206965:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed61180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589131392:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619918.207064:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a6c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619918.214939:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619918.215610:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571486720 00000100:00100000:0.0:1788619918.215620:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571486720 00002000:00100000:0.0:1788619918.215692:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fed61f80 x1875503571486720/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:14/0 lens 488/0 e 0 to 0 dl 1788619929 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619918.215703:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fed61f80 with x1875503571486720 00010000:00010000:0.0:1788619918.215713:0:5857:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753200/0x1cd7fe5368734ee8 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c25f expref: 15 pid: 5844 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.215729:0:5857:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753200/0x1cd7fe5368734ee8 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c25f expref: 15 pid: 5844 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788619918.215738:0:5857:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e78fed61f80 00000100:00100000:0.0:1788619918.215782:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 23 00000100:00100000:0.0:1788619918.215793:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed61f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+15:2355:x1875503571486720:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:3.0:1788619918.218671:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 994373 -> 994373 00000004:00010000:3.0:1788619918.218683:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619918 -> 1788619918 00000004:00010000:3.0:1788619918.218686:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619916 -> 1788619916 00000004:00010000:3.0:1788619918.218689:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619918 -> 1788619918 00000004:00010000:3.0:1788619918.218691:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 518 -> 518 00000100:00100000:3.0:1788619918.218789:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed61f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+17:2355:x1875503571486720:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 2992us (3869us total) trans 4294967693 rc 0/0 00000100:00100000:3.0:1788619918.218800:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 23 00002000:00100000:3.0:1788619918.218816:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fed61f80 x1875503571486720/t4294967693(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:14/0 lens 488/448 e 0 to 0 dl 1788619929 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619918.218828:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fed61f80 with x1875503571486720 00010000:00010000:3.0:1788619918.218841:0:5857:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753200/0x1cd7fe5368734ee8 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c25f expref: 17 pid: 5844 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619918.218878:0:5857:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753200/0x1cd7fe5368734ee8 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c25f expref: 17 pid: 5844 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788619918.218885:0:5857:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e78fed61f80 00000100:00100000:2.0:1788619918.220315:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed61180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589131392:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619918.220378:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753000/0x1cd7fe5368734f58 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2c1 expref: 23 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.220400:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619918.220419:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619918.220424:0:5844:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753000/0x1cd7fe5368734f58 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2c1 expref: 23 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.220436:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5753000, rc 0) 00000100:00100000:2.0:1788619918.220609:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b14e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+23:14908:x1875503571486592:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 13831us (15708us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619918.220618:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1047 00000100:00100000:0.0:1788619918.221892:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619918.222112:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571486848 00000100:00100000:3.0:1788619918.222122:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571486848 00010000:00010000:3.0:1788619918.222166:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753200/0x1cd7fe5368734ee8 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c25f expref: 17 pid: 5844 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619918.222212:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 348 00000100:00100000:3.0:1788619918.222224:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c855ac50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+17:2354:x1875503571486848:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619918.222241:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619918.222274:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753200/0x1cd7fe5368734ee8 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c25f expref: 17 pid: 5844 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619918.222289:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753200/0x1cd7fe5368734ee8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c25f expref: 17 pid: 5844 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619918.222315:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753200/0x1cd7fe5368734ee8 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c25f expref: 17 pid: 5844 timeout: 510 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619918.222366:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619918.222372:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753000/0x1cd7fe5368734f58 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2c1 expref: 22 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619918.222382:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753000/0x1cd7fe5368734f58 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2c1 expref: 22 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619918.222404:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753000/0x1cd7fe5368734f58 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2c1 expref: 22 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619918.222439:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753000/0x1cd7fe5368734f58 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2c1 expref: 22 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619918.222452:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b15180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589131520:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619918.222530:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619918.225712:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b15180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589131520:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619918.225738:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619918.225805:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c855ac50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+16:2354:x1875503571486848:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3583us (3918us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619918.225816:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 348 00000100:00100000:1.0:1788619918.236483:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619918.236621:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571486976 00000100:00100000:3.0:1788619918.236631:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571486976 00002000:00100000:3.0:1788619918.236680:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fed60a80 x1875503571486976/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:14/0 lens 488/0 e 0 to 0 dl 1788619929 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619918.236691:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fed60a80 with x1875503571486976 00000100:00100000:3.0:1788619918.236730:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 662 00000100:00100000:3.0:1788619918.236739:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed60a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+23:2353:x1875503571486976:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619918.317169:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed60a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+23:2353:x1875503571486976:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 80418us (80683us total) trans 0 rc 995328/995328 00000100:00100000:3.0:1788619918.317184:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 662 00002000:00100000:3.0:1788619918.317197:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fed60a80 x1875503571486976/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:14/0 lens 488/440 e 0 to 0 dl 1788619929 ref 1 fl Complete:/600/0 rc 995328/995328 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619918.317221:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fed60a80 with x1875503571486976 00000100:00100000:0.0:1788619918.321514:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619918.324723:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571487104 00000100:00100000:2.0:1788619918.324739:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571487104 00000100:00100000:2.0:1788619918.324831:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1048 00000100:00100000:2.0:1788619918.324841:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6001880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+23:14908:x1875503571487104:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619918.324860:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619918.324882:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe5368734f5f lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c2c8 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.327926:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079800/0x1cd7fe5368734f66 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.327961:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079800/0x1cd7fe5368734f66 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.327974:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079800/0x1cd7fe5368734f66 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788619918.328012:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1646471 00000004:00200000:2.0:1788619918.328017:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788619918.328048:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079800/0x1cd7fe5368734f66 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.328115:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe5368734f5f lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2c8 expref: 25 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.328137:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079800/0x1cd7fe5368734f66 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2c8 expref: 24 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.328147:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619918.328150:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788619918.328155:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079800, rc 0) 00000100:00100000:2.0:1788619918.328243:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6001880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+24:14908:x1875503571487104:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3402us (6738us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619918.328251:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1048 00000100:00100000:2.0:1788619918.331260:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619918.331368:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571487232 00000100:00100000:1.0:1788619918.331379:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571487232 00000100:00100000:1.0:1788619918.331425:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 821 00000100:00100000:1.0:1788619918.331435:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571487232:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619918.331452:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619918.331467:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e4c897/0x1cd7fe5368734f6d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c2cf expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619918.331525:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed62a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589131648:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619918.331709:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a240 going to sleep for 11 seconds 00000100:00100000:1.0:1788619918.344538:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed62a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589131648:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619918.344589:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000029e4c897/0x1cd7fe5368734f6d lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2cf expref: 10 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619918.344631:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029e4c897/0x1cd7fe5368734f6d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2cf expref: 10 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619918.344645:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5753800, rc 0) 00000100:00100000:1.0:1788619918.344709:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571487232:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 13274us (13457us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619918.344719:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 821 00000100:00100000:2.0:1788619918.347399:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619918.347479:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571487360 00000100:00100000:3.0:1788619918.347489:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571487360 00000100:00100000:3.0:1788619918.347545:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 822 00000100:00100000:3.0:1788619918.347555:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571487360:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619918.347572:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619918.347591:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c4fc38e/0x1cd7fe5368734f74 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c2d6 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619918.347648:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b15180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589131776:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619918.347719:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:3.0:1788619918.370268:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b15180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589131776:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619918.370310:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004c4fc38e/0x1cd7fe5368734f74 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2d6 expref: 10 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619918.370377:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c4fc38e/0x1cd7fe5368734f74 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2d6 expref: 10 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619918.370391:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683ce00, rc 0) 00000100:00100000:3.0:1788619918.370452:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571487360:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 22897us (23061us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619918.370468:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 822 00000100:00100000:0.0:1788619918.386470:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619918.387055:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571487488 00000100:00100000:2.0:1788619918.387067:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571487488 00000100:00100000:2.0:1788619918.387131:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1049 00000100:00100000:2.0:1788619918.387142:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6002a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+16:2354:x1875503571487488:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619918.387169:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619918.387190:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8078200/0x1cd7fe5368734f7b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c2dd expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.387233:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8078200/0x1cd7fe5368734f7b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2dd expref: 17 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788619918.387249:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:2.0:1788619918.387266:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8078200/0x1cd7fe5368734f7b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2dd expref: 17 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.387281:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619918.387284:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619918.387288:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8078200, rc 0) 00000100:00100000:2.0:1788619918.387404:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6002a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+17:2354:x1875503571487488:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 263us (944us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619918.387411:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1049 00000100:00100000:2.0:1788619918.399791:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619918.400061:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571487616 00000100:00100000:0.0:1788619918.400074:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571487616 00000100:00100000:0.0:1788619918.400149:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 823 00000100:00100000:0.0:1788619918.400159:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571487616:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619918.400179:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619918.400201:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000464c92c3/0x1cd7fe5368734f82 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c2e4 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619918.400268:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6001f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589131904:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619918.400388:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c2c5ef00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619918.418783:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6001f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589131904:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619918.418837:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000464c92c3/0x1cd7fe5368734f82 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2e4 expref: 10 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.418868:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000464c92c3/0x1cd7fe5368734f82 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2e4 expref: 10 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.418884:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78df191000, rc 0) 00000100:00100000:2.0:1788619918.418993:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571487616:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 18834us (19210us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619918.419003:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 823 00000100:00100000:0.0:1788619918.424578:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619918.424680:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571487744 00000100:00100000:3.0:1788619918.424691:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571487744 00000100:00100000:3.0:1788619918.424812:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 824 00000100:00100000:3.0:1788619918.424822:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6000e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571487744:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619918.424841:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619918.424861:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c4fc38e/0x1cd7fe5368734f89 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c2eb expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619918.425033:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b15180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589132032:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619918.425122:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:3.0:1788619918.427655:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b15180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589132032:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619918.427692:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004c4fc38e/0x1cd7fe5368734f89 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2eb expref: 10 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619918.427716:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c4fc38e/0x1cd7fe5368734f89 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2eb expref: 10 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619918.427728:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683ce00, rc 0) 00000100:00100000:3.0:1788619918.427790:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6000e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571487744:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2969us (3223us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619918.427799:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 824 00000100:00100000:2.0:1788619918.430603:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619918.430936:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571487872 00000100:00100000:1.0:1788619918.430946:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571487872 00000100:00100000:1.0:1788619918.431004:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 825 00000100:00100000:1.0:1788619918.431014:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+9:14908:x1875503571487872:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619918.431166:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619918.431186:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e4c897/0x1cd7fe5368734f90 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c2f2 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619918.431219:0:6698:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000007736e663/0x1cd7fe5368734e55 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1150976->1196031) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1e8 expref: 10 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619918.431229:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=1s v=(6 12) 00010000:00010000:1.0:1788619918.431234:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(7 12) 00010000:00010000:1.0:1788619918.431239:0:6698:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e4c897/0x1cd7fe5368734f90 lrc: 4/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1519616->1523711] (req 1519616->1523711) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2f2 expref: 10 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619918.431277:0:6698:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000007736e663/0x1cd7fe5368734e55 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1150976->1196031) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1e8 expref: 10 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619918.431307:0:6698:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000007736e663/0x1cd7fe5368734e55 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1150976->1196031) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1e8 expref: 10 pid: 6697 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619918.431323:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed62a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589132160:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619918.431695:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a240 going to sleep for 11 seconds 00000100:00100000:2.0:1788619918.434463:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619918.434546:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571488000 00000100:00100000:3.0:1788619918.434555:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571488000 00010000:00010000:3.0:1788619918.434604:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000007736e663/0x1cd7fe5368734e55 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1150976->1196031) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1e8 expref: 11 pid: 6697 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619918.434638:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 349 00000100:00100000:3.0:1788619918.434647:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df964700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571488000:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619918.434676:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619918.434698:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000007736e663/0x1cd7fe5368734e55 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1150976->1196031) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1e8 expref: 11 pid: 6697 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619918.434712:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000007736e663/0x1cd7fe5368734e55 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1150976->1196031) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1e8 expref: 11 pid: 6697 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619918.434740:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(7 12) 00010000:00010000:3.0:1788619918.434744:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(7 12) 00010000:00010000:3.0:1788619918.434753:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000029e4c897/0x1cd7fe5368734f90 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1519616->1523711) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2f2 expref: 10 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619918.434772:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e4c897/0x1cd7fe5368734f90 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1519616->1523711) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2f2 expref: 10 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619918.434807:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000029e4c897/0x1cd7fe5368734f90 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1519616->1523711) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2f2 expref: 10 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619918.434822:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b15180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589132288:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619918.434895:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00010000:00010000:1.0:1788619918.435889:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007736e663/0x1cd7fe5368734e55 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1150976->1196031) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c1e8 expref: 11 pid: 6697 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619918.435910:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed62a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589132160:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619918.435941:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000029e4c897/0x1cd7fe5368734f90 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1519616->1523711) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2f2 expref: 10 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619918.435954:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5753800, rc 0) 00000100:00100000:1.0:1788619918.435996:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+10:14908:x1875503571487872:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4983us (5414us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619918.436004:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 825 00000100:00100000:2.0:1788619918.441875:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619918.441931:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619918.442028:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571488256 00000100:00100000:0.0:1788619918.442048:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571488256 00002000:00100000:0.0:1788619918.442103:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df967100 x1875503571488256/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:14/0 lens 488/0 e 0 to 0 dl 1788619929 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619918.442115:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df967100 with x1875503571488256 00000100:00100000:0.0:1788619918.442153:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 663 00000100:00100000:0.0:1788619918.442163:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df967100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2355:x1875503571488256:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619918.442937:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571488128 00000100:00100000:3.0:1788619918.442945:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571488128 00002000:00100000:3.0:1788619918.442977:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1646591) req@ffff8e78df964000 x1875503571488128/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:14/0 lens 488/0 e 0 to 0 dl 1788619929 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619918.442987:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df964000 with x1875503571488128 ext(1048576->1646591) 00002000:00010000:3.0:1788619918.443006:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df964000 00000100:00100000:3.0:1788619918.443042:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 457 00000100:00100000:3.0:1788619918.443051:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df964000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+10:2355:x1875503571488128:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619918.447251:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df967100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2355:x1875503571488256:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 5077us (5307us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619918.447266:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 663 00002000:00100000:1.0:1788619918.447278:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df967100 x1875503571488256/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:14/0 lens 488/440 e 0 to 0 dl 1788619929 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619918.447288:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df967100 with x1875503571488256 00000100:00100000:3.0:1788619918.448777:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b15180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589132288:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619918.448802:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619918.448899:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df964700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571488000:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 14252us (14445us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619918.448910:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 349 00000100:00100000:3.0:1788619918.476833:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df964000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+10:2355:x1875503571488128:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 33769us (34953us total) trans 0 rc 598016/598016 00000100:00100000:3.0:1788619918.476848:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 457 00002000:00100000:3.0:1788619918.476862:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1646591) req@ffff8e78df964000 x1875503571488128/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:14/0 lens 488/440 e 0 to 0 dl 1788619929 ref 1 fl Complete:/600/0 rc 598016/598016 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619918.476872:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df964000 with x1875503571488128 ext(1048576->1646591) 00002000:00010000:3.0:1788619918.476879:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df964000 00000100:00100000:0.0:1788619918.484228:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619918.484449:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571488384 00000100:00100000:2.0:1788619918.484479:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571488384 00000100:00100000:2.0:1788619918.484545:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1050 00000100:00100000:2.0:1788619918.484557:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6000000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+24:14908:x1875503571488384:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619918.484592:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619918.484596:0:5844:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1788619918.484644:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753000/0x1cd7fe5368734f58 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2c1 expref: 24 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.484664:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619918.484669:0:5844:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788619918.484681:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c71f9000/0x1cd7fe5368734f97 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c2f9 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.484708:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8078200/0x1cd7fe5368734f7b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2dd expref: 16 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.484717:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c71f9000/0x1cd7fe5368734f97 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2f9 expref: 24 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.484750:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8078200/0x1cd7fe5368734f7b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2dd expref: 16 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.484772:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8078200/0x1cd7fe5368734f7b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2dd expref: 16 pid: 11397 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619918.484787:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df966a00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589132416:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619918.484927:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619918.488808:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619918.488898:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571488512 00000100:00100000:0.0:1788619918.488919:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571488512 00010000:00010000:0.0:1788619918.488966:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8078200/0x1cd7fe5368734f7b lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2dd expref: 17 pid: 11397 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619918.489008:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 350 00000100:00100000:0.0:1788619918.489018:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df965500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+17:2355:x1875503571488512:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619918.489060:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619918.489089:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8078200/0x1cd7fe5368734f7b lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2dd expref: 17 pid: 11397 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.489101:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8078200/0x1cd7fe5368734f7b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2dd expref: 17 pid: 11397 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.489125:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619918.489131:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c71f9000/0x1cd7fe5368734f97 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2f9 expref: 24 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.489144:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c71f9000/0x1cd7fe5368734f97 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2f9 expref: 24 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.489154:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c71f9000/0x1cd7fe5368734f97 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2f9 expref: 24 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.489197:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c71f9000/0x1cd7fe5368734f97 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2f9 expref: 24 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619918.489210:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6001f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589132544:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619918.489290:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c2c5e300 going to sleep for 11 seconds 00010000:00010000:1.0:1788619918.490208:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8078200/0x1cd7fe5368734f7b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2dd expref: 17 pid: 11397 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619918.490236:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df966a00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589132416:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619918.490260:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c71f9000/0x1cd7fe5368734f97 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2f9 expref: 24 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619918.490272:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619918.490275:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619918.490278:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c71f9000, rc 0) 00000100:00100000:1.0:1788619918.490355:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6000000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+24:14908:x1875503571488384:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5770us (6111us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619918.490363:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1050 00000100:00100000:2.0:1788619918.494383:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619918.494489:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571488640 00000100:00100000:3.0:1788619918.494500:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571488640 00002000:00100000:3.0:1788619918.494552:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df965c00 x1875503571488640/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:14/0 lens 488/0 e 0 to 0 dl 1788619929 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619918.494565:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df965c00 with x1875503571488640 00000100:00100000:3.0:1788619918.494611:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 664 00000100:00100000:3.0:1788619918.494621:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df965c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2354:x1875503571488640:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619918.494930:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df965c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2354:x1875503571488640:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 309us (603us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619918.494971:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 664 00002000:00100000:3.0:1788619918.494980:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df965c00 x1875503571488640/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:14/0 lens 488/4536 e 0 to 0 dl 1788619929 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619918.494987:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df965c00 with x1875503571488640 00000100:00100000:0.0:1788619918.495139:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6001f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589132544:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619918.495164:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619918.495202:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df965500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571488512:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 6185us (6407us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619918.495221:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 350 00000100:00100000:2.0:1788619918.499223:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619918.499308:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571488768 00000100:00100000:1.0:1788619918.499319:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571488768 00002000:00100000:1.0:1788619918.499416:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df964380 x1875503571488768/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:14/0 lens 488/0 e 0 to 0 dl 1788619929 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619918.499428:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df964380 with x1875503571488768 00000100:00100000:1.0:1788619918.499462:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 665 00000100:00100000:1.0:1788619918.499471:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df964380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2354:x1875503571488768:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619918.499799:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df964380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2354:x1875503571488768:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 328us (587us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619918.499808:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 665 00002000:00100000:1.0:1788619918.499815:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df964380 x1875503571488768/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:14/0 lens 488/4536 e 0 to 0 dl 1788619929 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619918.499825:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df964380 with x1875503571488768 00000100:00100000:2.0:1788619918.511056:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619918.511381:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571488896 00000100:00100000:1.0:1788619918.511393:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571488896 00002000:00100000:1.0:1788619918.511444:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df965180 x1875503571488896/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:14/0 lens 504/0 e 0 to 0 dl 1788619929 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619918.511456:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df965180 with x1875503571488896 00000100:00100000:1.0:1788619918.511489:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 666 00000100:00100000:1.0:1788619918.511499:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df965180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2355:x1875503571488896:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619918.520613:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df965180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2355:x1875503571488896:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 9104us (9558us total) trans 0 rc 983040/983040 00000100:00100000:1.0:1788619918.520628:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 666 00002000:00100000:1.0:1788619918.520641:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df965180 x1875503571488896/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:14/0 lens 504/440 e 0 to 0 dl 1788619929 ref 1 fl Complete:/600/0 rc 983040/983040 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619918.520651:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df965180 with x1875503571488896 00000100:00100000:0.0:1788619918.525980:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619918.526065:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619918.526192:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571489152 00000100:00100000:3.0:1788619918.526203:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571489152 00000100:00100000:3.0:1788619918.526258:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 826 00000100:00100000:3.0:1788619918.526268:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571489152:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619918.526287:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619918.526306:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c4fc38e/0x1cd7fe5368734f9e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c307 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619918.526348:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(7 12) 00010000:00010000:3.0:1788619918.526353:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(7 12) 00010000:00010000:3.0:1788619918.526364:0:7654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004c4fc38e/0x1cd7fe5368734f9e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c307 expref: 11 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619918.526382:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004c4fc38e/0x1cd7fe5368734f9e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c307 expref: 11 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619918.526396:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683ce00, rc 0) 00000100:00100000:3.0:1788619918.526461:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571489152:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 194us (395us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619918.526468:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 826 00000100:00100000:2.0:1788619918.527855:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571489024 00000100:00100000:2.0:1788619918.527865:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571489024 00000100:00100000:2.0:1788619918.527914:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1051 00000100:00100000:2.0:1788619918.527922:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6001f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+16:2352:x1875503571489024:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619918.527939:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619918.527956:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679000/0x1cd7fe5368734fa5 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c300 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.527991:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c71f9000/0x1cd7fe5368734f97 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2f9 expref: 23 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619918.528156:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df964a80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589132672:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619918.528236:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc9c0 going to sleep for 11 seconds 00000004:00010000:2.0:1788619918.530531:0:5845:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619916 -> 1788619918 00000100:00100000:2.0:1788619918.530566:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df964a80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589132672:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619918.530602:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:2.0:1788619918.530621:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679000/0x1cd7fe5368734fa5 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c300 expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.530763:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619918.530767:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619918.530775:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679000/0x1cd7fe5368734fa5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c300 expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.530809:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c7679000, rc 0) 00000100:00100000:2.0:1788619918.530914:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6001f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+16:2352:x1875503571489024:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2992us (4941us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619918.530924:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1051 00000100:00100000:0.0:1788619918.533829:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619918.533963:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571489280 00000100:00100000:3.0:1788619918.533987:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571489280 00000100:00100000:3.0:1788619918.534111:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1052 00000100:00100000:3.0:1788619918.534123:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571489280:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619918.534150:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619918.534175:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368734fac lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c30e expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619918.534229:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c71f9000/0x1cd7fe5368734f97 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2f9 expref: 23 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619918.534276:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b15180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589132800:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619918.534590:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619918.537559:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b15180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589132800:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619918.537653:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:2.0:1788619918.537675:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368734fac lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c30e expref: 17 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.537697:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619918.537701:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619918.537707:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368734fac lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c30e expref: 17 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.537721:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d600, rc 0) 00000100:00100000:2.0:1788619918.537831:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571489280:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3710us (4009us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619918.537839:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1052 00000100:00100000:0.0:1788619918.541641:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619918.541742:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571489408 00000100:00100000:1.0:1788619918.541752:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571489408 00000100:00100000:1.0:1788619918.541828:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1053 00000100:00100000:1.0:1788619918.541840:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+16:2353:x1875503571489408:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619918.541861:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619918.541882:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752200/0x1cd7fe5368734fb3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c315 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619918.541924:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c71f9000/0x1cd7fe5368734f97 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2f9 expref: 23 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619918.541966:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed62a00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589132928:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619918.542059:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a900 going to sleep for 11 seconds 00000100:00100000:1.0:1788619918.544355:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed62a00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589132928:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619918.544383:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:1.0:1788619918.544401:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752200/0x1cd7fe5368734fb3 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c315 expref: 17 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619918.544420:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619918.544423:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619918.544427:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752200/0x1cd7fe5368734fb3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c315 expref: 17 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619918.544440:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752200, rc 0) 00000100:00100000:1.0:1788619918.544499:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+16:2353:x1875503571489408:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2660us (2868us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619918.544508:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1053 00000100:00100000:0.0:1788619918.547073:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619918.547143:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571489536 00000100:00100000:2.0:1788619918.547154:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571489536 00000100:00100000:2.0:1788619918.547208:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1054 00000100:00100000:2.0:1788619918.547219:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+16:2352:x1875503571489536:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619918.547239:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619918.547259:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679000/0x1cd7fe5368734fba lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c31c expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.547320:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c71f9000/0x1cd7fe5368734f97 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2f9 expref: 23 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619918.547410:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df964a80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589133056:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619918.547488:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619918.550218:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df964a80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589133056:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619918.550247:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:2.0:1788619918.550266:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679000/0x1cd7fe5368734fba lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c31c expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.550284:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619918.550288:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619918.550293:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679000/0x1cd7fe5368734fba lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c31c expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.550307:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c7679000, rc 0) 00000100:00100000:2.0:1788619918.550390:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+16:2352:x1875503571489536:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3172us (3327us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619918.550398:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1054 00000100:00100000:1.0:1788619918.553785:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619918.553897:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571489664 00000100:00100000:2.0:1788619918.553955:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571489664 00000100:00100000:2.0:1788619918.554084:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1055 00000100:00100000:2.0:1788619918.554096:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed62a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+16:2352:x1875503571489664:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619918.554118:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619918.554139:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679a00/0x1cd7fe5368734fc1 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c323 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.554199:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c71f9000/0x1cd7fe5368734f97 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2f9 expref: 23 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619918.554248:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df964a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589133184:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619918.554496:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619918.557070:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df964a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589133184:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619918.557097:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:2.0:1788619918.557114:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679a00/0x1cd7fe5368734fc1 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c323 expref: 17 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.557134:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619918.557137:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619918.557143:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679a00/0x1cd7fe5368734fc1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c323 expref: 17 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.557157:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c7679a00, rc 0) 00000100:00100000:2.0:1788619918.557209:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed62a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+16:2352:x1875503571489664:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3116us (3435us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619918.557217:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1055 00000100:00100000:0.0:1788619918.562491:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619918.562601:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571489792 00000100:00100000:1.0:1788619918.562611:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571489792 00000100:00100000:1.0:1788619918.562673:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1056 00000100:00100000:1.0:1788619918.562684:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+16:2353:x1875503571489792:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619918.562706:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619918.562729:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753600/0x1cd7fe5368734fc8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c32a expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619918.562777:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c71f9000/0x1cd7fe5368734f97 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2f9 expref: 23 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619918.562817:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed63100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589133312:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619918.562926:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a900 going to sleep for 11 seconds 00000100:00100000:2.0:1788619918.565594:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed63100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589133312:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619918.565628:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:2.0:1788619918.565647:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753600/0x1cd7fe5368734fc8 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c32a expref: 17 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.565666:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619918.565683:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619918.565690:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753600/0x1cd7fe5368734fc8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c32a expref: 17 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.565703:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5753600, rc 0) 00000100:00100000:2.0:1788619918.565753:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+16:2353:x1875503571489792:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3070us (3272us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619918.565773:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1056 00000100:00100000:2.0:1788619918.568308:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619918.568421:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571489920 00000100:00100000:0.0:1788619918.568431:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571489920 00000100:00100000:0.0:1788619918.568477:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 827 00000100:00100000:0.0:1788619918.568486:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df964a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+10:14908:x1875503571489920:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619918.568507:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619918.568526:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000055b3449d/0x1cd7fe5368734fcf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c331 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.568557:0:6696:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000029e4c897/0x1cd7fe5368734f90 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1519616->1523711) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2f2 expref: 11 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.568568:0:6696:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000004c4fc38e/0x1cd7fe5368734f9e lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c307 expref: 10 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.568576:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(7 12) 00010000:00010000:0.0:1788619918.568591:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(7 12) 00010000:00010000:0.0:1788619918.568597:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000055b3449d/0x1cd7fe5368734fcf lrc: 5/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [1392640->1437695] (req 1392640->1437695) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c331 expref: 11 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.568653:0:6696:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000004c4fc38e/0x1cd7fe5368734f9e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c307 expref: 10 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.568667:0:6696:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000004c4fc38e/0x1cd7fe5368734f9e lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c307 expref: 10 pid: 7654 timeout: 511 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619918.568681:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c74f7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589133440:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619918.568783:0:6696:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000029e4c897/0x1cd7fe5368734f90 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1519616->1523711) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2f2 expref: 11 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.568793:0:6696:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000029e4c897/0x1cd7fe5368734f90 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1519616->1523711) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2f2 expref: 11 pid: 6698 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619918.568802:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c74f5500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589133568:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619918.568827:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c2c5e300 going to sleep for 11 seconds 00000100:00100000:2.0:1788619918.571459:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c74f7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589133440:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619918.574463:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619918.574506:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619918.574810:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571490176 00000100:00100000:1.0:1788619918.574821:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571490176 00010000:00010000:1.0:1788619918.574873:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000029e4c897/0x1cd7fe5368734f90 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1519616->1523711) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2f2 expref: 12 pid: 6698 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619918.574920:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571490048 00000100:00100000:1.0:1788619918.574924:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571490048 00010000:00010000:1.0:1788619918.574938:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000004c4fc38e/0x1cd7fe5368734f9e lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c307 expref: 11 pid: 7654 timeout: 511 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619918.574961:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 351 00000100:00100000:1.0:1788619918.574970:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df965880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571490176:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619918.574987:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619918.575010:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000029e4c897/0x1cd7fe5368734f90 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1519616->1523711) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2f2 expref: 12 pid: 6698 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619918.575035:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000029e4c897/0x1cd7fe5368734f90 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1519616->1523711) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2f2 expref: 12 pid: 6698 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619918.575065:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619918.575117:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df965880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571490176:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 148us (664us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619918.575125:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 351 00000100:00100000:1.0:1788619918.575143:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 352 00000100:00100000:1.0:1788619918.575151:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df964e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571490048:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619918.575182:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619918.575193:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000004c4fc38e/0x1cd7fe5368734f9e lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c307 expref: 11 pid: 7654 timeout: 511 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619918.575205:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000004c4fc38e/0x1cd7fe5368734f9e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c307 expref: 11 pid: 7654 timeout: 511 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619918.575219:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c4fc38e/0x1cd7fe5368734f9e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c307 expref: 11 pid: 7654 timeout: 511 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619918.575234:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(7 12) 00010000:00010000:1.0:1788619918.575239:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(7 12) 00010000:00010000:1.0:1788619918.575248:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000055b3449d/0x1cd7fe5368734fcf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1392640->1437695) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c331 expref: 11 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619918.575258:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000055b3449d/0x1cd7fe5368734fcf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1392640->1437695) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c331 expref: 11 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619918.575295:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000055b3449d/0x1cd7fe5368734fcf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1392640->1437695) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c331 expref: 11 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619918.575310:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed60700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589133696:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619918.575401:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a900 going to sleep for 11 seconds 00010000:00010000:0.0:1788619918.577044:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029e4c897/0x1cd7fe5368734f90 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1519616->1523711) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2f2 expref: 11 pid: 6698 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619918.577062:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c74f5500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589133568:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619918.577095:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000055b3449d/0x1cd7fe5368734fcf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1392640->1437695) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c331 expref: 10 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.577109:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78df190000, rc 0) 00000100:00100000:0.0:1788619918.577177:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df964a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+10:14908:x1875503571489920:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8691us (8880us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619918.577186:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 827 00000100:00100000:3.0:1788619918.578547:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed60700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589133696:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619918.578566:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619918.578622:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df964e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571490048:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3470us (4114us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619918.578632:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 352 00000100:00100000:2.0:1788619918.582161:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619918.582244:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571490304 00000100:00100000:0.0:1788619918.582254:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571490304 00000100:00100000:0.0:1788619918.582320:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 458 00000100:00100000:0.0:1788619918.582356:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df967480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+10:2352:x1875503571490304:12345-192.168.201.5@tcp:22:fsx.0 00002000:00010000:0.0:1788619918.582600:0:6701:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1646471 -> 1646471 00002000:00010000:0.0:1788619918.582605:0:6701:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619918 -> 1788619918 00002000:00010000:0.0:1788619918.582608:0:6701:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619916 -> 1788619916 00002000:00010000:0.0:1788619918.582610:0:6701:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619918 -> 1788619918 00002000:00010000:0.0:1788619918.582613:0:6701:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 135 -> 135 00000100:00100000:0.0:1788619918.582665:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df967480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+11:2352:x1875503571490304:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 311us (512us total) trans 4294967435 rc 0/0 00000100:00100000:0.0:1788619918.582673:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 458 00000100:00100000:1.0:1788619918.587370:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619918.587502:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571490432 00000100:00100000:2.0:1788619918.587512:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571490432 00000100:00100000:2.0:1788619918.587571:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1057 00000100:00100000:2.0:1788619918.587582:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed62680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+16:2353:x1875503571490432:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619918.587601:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619918.587620:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678000/0x1cd7fe5368734fd6 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c338 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.587654:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c71f9000/0x1cd7fe5368734f97 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2f9 expref: 23 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619918.587693:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df966300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589133824:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619918.587787:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619918.594172:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619918.594374:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df966300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589133824:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619918.594399:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:3.0:1788619918.594418:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678000/0x1cd7fe5368734fd6 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c338 expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619918.594440:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619918.594443:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619918.594448:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678000/0x1cd7fe5368734fd6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c338 expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619918.594475:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c7678000, rc 0) 00000100:00100000:3.0:1788619918.594530:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed62680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+16:2353:x1875503571490432:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6950us (7208us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619918.594539:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1057 00000100:00100000:0.0:1788619918.595375:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571490560 00000100:00100000:0.0:1788619918.595386:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571490560 00000100:00100000:0.0:1788619918.595445:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 828 00000100:00100000:0.0:1788619918.595456:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df967800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571490560:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619918.595470:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619918.595488:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bad85f32/0x1cd7fe5368734fdd lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c33f expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619918.595546:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c74f5500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589133952:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619918.595635:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c2c5e300 going to sleep for 11 seconds 00002000:00010000:0.0:1788619918.598596:0:7654:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619916 -> 1788619918 00000100:00100000:0.0:1788619918.598649:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c74f5500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589133952:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619918.598686:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bad85f32/0x1cd7fe5368734fdd lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c33f expref: 11 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.598714:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bad85f32/0x1cd7fe5368734fdd lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c33f expref: 11 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.598729:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78df191800, rc 0) 00000100:00100000:0.0:1788619918.598803:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df967800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571490560:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3349us (4643us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619918.598812:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 828 00000100:00100000:2.0:1788619918.603893:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619918.603942:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619918.604042:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571490816 00000100:00100000:1.0:1788619918.604066:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571490816 00000100:00100000:1.0:1788619918.604125:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 829 00000100:00100000:1.0:1788619918.604135:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df965f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571490816:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619918.604151:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619918.604169:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a9ce7747/0x1cd7fe5368734fe4 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c34d expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619918.604230:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed60e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589134080:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619918.604312:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a240 going to sleep for 11 seconds 00000100:00100000:2.0:1788619918.607992:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed60e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589134080:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619918.608047:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a9ce7747/0x1cd7fe5368734fe4 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c34d expref: 11 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.608074:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a9ce7747/0x1cd7fe5368734fe4 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c34d expref: 11 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.608089:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5753000, rc 0) 00000100:00100000:2.0:1788619918.608160:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df965f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571490816:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4025us (4278us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619918.608168:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 829 00000100:00100000:0.0:1788619918.611206:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571490688 00000100:00100000:0.0:1788619918.611225:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571490688 00000100:00100000:0.0:1788619918.611359:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1058 00000100:00100000:0.0:1788619918.611371:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df966300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+16:2352:x1875503571490688:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619918.611392:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619918.611414:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df191800/0x1cd7fe5368734feb lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c346 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.611454:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c71f9000/0x1cd7fe5368734f97 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2f9 expref: 23 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619918.611505:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c74f5500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589134208:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619918.611621:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c2c5e300 going to sleep for 11 seconds 00000100:00100000:0.0:1788619918.614890:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c74f5500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589134208:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619918.615101:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:0.0:1788619918.615124:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df191800/0x1cd7fe5368734feb lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c346 expref: 17 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.615147:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619918.615150:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619918.615156:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78df191800/0x1cd7fe5368734feb lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c346 expref: 17 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.615172:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78df191800, rc 0) 00000100:00100000:0.0:1788619918.615277:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df966300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+16:2352:x1875503571490688:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3907us (11332us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619918.615286:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1058 00000100:00100000:3.0:1788619918.619005:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619918.619054:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619918.619291:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571490944 00000100:00100000:2.0:1788619918.619302:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571490944 00000100:00100000:2.0:1788619918.619385:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1059 00000100:00100000:2.0:1788619918.619397:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b15880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+16:2353:x1875503571490944:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619918.619420:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619918.619442:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678a00/0x1cd7fe5368734ff2 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c354 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.619484:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c71f9000/0x1cd7fe5368734f97 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2f9 expref: 23 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619918.619527:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df966a00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589134336:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619918.619621:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619918.620469:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571491072 00000100:00100000:0.0:1788619918.620477:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571491072 00000100:00100000:0.0:1788619918.620524:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 830 00000100:00100000:0.0:1788619918.620533:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b15f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571491072:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619918.620546:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619918.620560:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d203f2cc/0x1cd7fe5368734ff9 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c35b expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619918.620621:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c74f5500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589134464:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619918.620684:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c2c5e300 going to sleep for 11 seconds 00000100:00100000:0.0:1788619918.633437:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c74f5500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589134464:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619918.633488:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d203f2cc/0x1cd7fe5368734ff9 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c35b expref: 11 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.633538:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d203f2cc/0x1cd7fe5368734ff9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c35b expref: 11 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.633553:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78df190600, rc 0) 00000100:00100000:0.0:1788619918.633639:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b15f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571491072:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 13105us (14582us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619918.633647:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 830 00000100:00100000:2.0:1788619918.634272:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df966a00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589134336:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619918.634291:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:2.0:1788619918.634307:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678a00/0x1cd7fe5368734ff2 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c354 expref: 17 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.634363:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619918.634367:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619918.634374:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678a00/0x1cd7fe5368734ff2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c354 expref: 17 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.634387:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c7678a00, rc 0) 00000100:00100000:2.0:1788619918.634425:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b15880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+16:2353:x1875503571490944:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 15030us (15432us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619918.634432:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1059 00000100:00100000:3.0:1788619918.641618:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619918.641704:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571491200 00000100:00100000:2.0:1788619918.641714:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571491200 00000100:00100000:2.0:1788619918.641782:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1060 00000100:00100000:2.0:1788619918.641793:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b17480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571491200:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619918.641812:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619918.641832:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679800/0x1cd7fe5368735000 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c362 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.641867:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c71f9000/0x1cd7fe5368734f97 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2f9 expref: 23 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619918.641917:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df966a00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589134592:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619918.641996:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619918.644317:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619918.645006:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571491328 00000100:00100000:3.0:1788619918.645018:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571491328 00000100:00100000:3.0:1788619918.645091:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 831 00000100:00100000:3.0:1788619918.645101:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571491328:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619918.645121:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619918.645139:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a803caae/0x1cd7fe5368735007 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c369 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619918.645202:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b14700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589134720:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619918.645291:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaab40 going to sleep for 11 seconds 00000100:00100000:2.0:1788619918.650620:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df966a00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589134592:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619918.650652:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:2.0:1788619918.650677:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679800/0x1cd7fe5368735000 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c362 expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.650701:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619918.650705:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619918.650711:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679800/0x1cd7fe5368735000 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c362 expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.650726:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c7679800, rc 0) 00000100:00100000:2.0:1788619918.650826:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b17480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571491200:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 9035us (9216us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619918.650836:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1060 00000100:00100000:3.0:1788619918.663536:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b14700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589134720:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619918.663579:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a803caae/0x1cd7fe5368735007 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c369 expref: 11 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619918.663606:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a803caae/0x1cd7fe5368735007 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c369 expref: 11 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619918.663643:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d000, rc 0) 00000100:00100000:3.0:1788619918.663707:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571491328:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 18606us (19401us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619918.663717:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 831 00000100:00100000:0.0:1788619918.681985:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619918.682143:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571491456 00000100:00100000:2.0:1788619918.682153:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571491456 00000100:00100000:2.0:1788619918.682214:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1061 00000100:00100000:2.0:1788619918.682226:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+16:14908:x1875503571491456:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619918.682248:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619918.682269:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679800/0x1cd7fe536873500e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c370 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.682301:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c71f9000/0x1cd7fe5368734f97 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2f9 expref: 23 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.682312:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679800/0x1cd7fe536873500e lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c370 expref: 17 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.682412:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c71f9000/0x1cd7fe5368734f97 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2f9 expref: 23 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.682426:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c71f9000/0x1cd7fe5368734f97 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2f9 expref: 23 pid: 5844 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619918.682440:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df966a00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589134848:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619918.682518:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619918.697421:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619918.697513:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571491584 00000100:00100000:3.0:1788619918.697522:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571491584 00002000:00100000:3.0:1788619918.697572:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fed61880 x1875503571491584/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:14/0 lens 504/0 e 0 to 0 dl 1788619929 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619918.697593:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fed61880 with x1875503571491584 00010000:00010000:3.0:1788619918.697602:0:5859:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c71f9000/0x1cd7fe5368734f97 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2f9 expref: 24 pid: 5844 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619918.697618:0:5859:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c71f9000/0x1cd7fe5368734f97 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2f9 expref: 24 pid: 5844 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788619918.697627:0:5859:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e78fed61880 00000100:00100000:3.0:1788619918.697651:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 24 00000100:00100000:3.0:1788619918.697660:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed61880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2355:x1875503571491584:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:2.0:1788619918.698406:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df966a00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589134848:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619918.698436:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679800/0x1cd7fe536873500e lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c370 expref: 17 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.698453:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619918.698456:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619918.698461:0:5846:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679800/0x1cd7fe536873500e lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c370 expref: 17 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.698472:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c7679800, rc 0) 00000100:00100000:2.0:1788619918.698533:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+17:14908:x1875503571491456:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 16310us (16560us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619918.698542:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1061 00000004:00010000:3.0:1788619918.706238:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 994373 -> 994373 00000004:00010000:3.0:1788619918.706250:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619918 -> 1788619918 00000004:00010000:3.0:1788619918.706253:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619918 -> 1788619916 00000004:00010000:3.0:1788619918.706256:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619918 -> 1788619918 00000004:00010000:3.0:1788619918.706259:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 518 -> 518 00000100:00100000:3.0:1788619918.706397:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed61880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+26:2355:x1875503571491584:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 8729us (8977us total) trans 4294967694 rc 0/0 00000100:00100000:3.0:1788619918.706422:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 24 00002000:00100000:3.0:1788619918.706440:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fed61880 x1875503571491584/t4294967694(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:14/0 lens 504/448 e 0 to 0 dl 1788619929 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619918.706448:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fed61880 with x1875503571491584 00010000:00010000:3.0:1788619918.706461:0:5859:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c71f9000/0x1cd7fe5368734f97 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2f9 expref: 26 pid: 5844 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619918.706479:0:5859:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c71f9000/0x1cd7fe5368734f97 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2f9 expref: 26 pid: 5844 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788619918.706501:0:5859:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e78fed61880 00000100:00100000:1.0:1788619918.712423:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619918.714732:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571491712 00000100:00100000:3.0:1788619918.714766:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571491712 00010000:00010000:3.0:1788619918.714822:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c71f9000/0x1cd7fe5368734f97 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2f9 expref: 26 pid: 5844 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619918.714875:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 353 00000100:00100000:3.0:1788619918.714885:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed62d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+26:2355:x1875503571491712:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619918.714902:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619918.714935:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c71f9000/0x1cd7fe5368734f97 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2f9 expref: 26 pid: 5844 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619918.714950:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c71f9000/0x1cd7fe5368734f97 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2f9 expref: 26 pid: 5844 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619918.714976:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c71f9000/0x1cd7fe5368734f97 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2f9 expref: 26 pid: 5844 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619918.715036:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619918.715045:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679800/0x1cd7fe536873500e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c370 expref: 16 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619918.715061:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679800/0x1cd7fe536873500e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c370 expref: 16 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619918.715073:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679800/0x1cd7fe536873500e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c370 expref: 16 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619918.715124:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679800/0x1cd7fe536873500e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c370 expref: 16 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619918.715139:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b14700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589134976:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619918.715254:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaab40 going to sleep for 11 seconds 00000100:00100000:3.0:1788619918.722973:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b14700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589134976:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619918.723002:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619918.723070:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed62d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+25:2355:x1875503571491712:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 8185us (10658us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619918.723080:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 353 00000100:00100000:0.0:1788619918.731445:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619918.731621:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571491840 00000100:00100000:1.0:1788619918.731633:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571491840 00002000:00100000:1.0:1788619918.731690:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c74f7800 x1875503571491840/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:14/0 lens 488/0 e 0 to 0 dl 1788619929 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619918.731702:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c74f7800 with x1875503571491840 00000100:00100000:1.0:1788619918.731742:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 667 00000100:00100000:1.0:1788619918.731762:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+17:2354:x1875503571491840:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619918.795727:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+17:2354:x1875503571491840:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 63965us (64282us total) trans 0 rc 995328/995328 00000100:00100000:1.0:1788619918.795741:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 667 00002000:00100000:1.0:1788619918.795764:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c74f7800 x1875503571491840/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:14/0 lens 488/440 e 0 to 0 dl 1788619929 ref 1 fl Complete:/600/0 rc 995328/995328 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619918.795773:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c74f7800 with x1875503571491840 00000100:00100000:1.0:1788619918.800192:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619918.800314:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571491968 00000100:00100000:2.0:1788619918.800349:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571491968 00000100:00100000:2.0:1788619918.800414:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1062 00000100:00100000:2.0:1788619918.800424:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed62300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+25:2352:x1875503571491968:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619918.800455:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619918.800477:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678a00/0x1cd7fe5368735015 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c377 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.800530:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678a00/0x1cd7fe5368735015 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c377 expref: 26 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788619918.800543:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:2.0:1788619918.800551:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678a00/0x1cd7fe5368735015 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c377 expref: 26 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.800565:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619918.800568:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619918.800583:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c7678a00, rc 0) 00000100:00100000:2.0:1788619918.800647:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed62300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+26:2352:x1875503571491968:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 225us (468us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619918.800654:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1062 00000100:00100000:0.0:1788619918.803927:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619918.804248:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571492224 00000100:00100000:2.0:1788619918.804258:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571492224 00000100:00100000:2.0:1788619918.804316:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1063 00000100:00100000:2.0:1788619918.804353:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+26:14908:x1875503571492224:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:2.0:1788619918.804408:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619918.804435:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079800/0x1cd7fe5368734f66 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2c8 expref: 26 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.804454:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619918.804459:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788619918.804870:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678000/0x1cd7fe536873501c lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.804887:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368734f51 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2ba expref: 16 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.804896:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678000/0x1cd7fe536873501c lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.804991:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368734f51 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2ba expref: 16 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.805009:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368734f51 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2ba expref: 16 pid: 11397 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619918.805209:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df966a00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589135104:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619918.805449:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc240 going to sleep for 11 seconds 00000100:00100000:1.0:1788619918.818950:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619918.819094:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df966a00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589135104:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619918.819128:0:5845:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678000/0x1cd7fe536873501c lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619918.819380:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571492352 00000100:00100000:0.0:1788619918.819389:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571492352 00010000:00010000:0.0:1788619918.819430:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368734f51 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2ba expref: 17 pid: 11397 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619918.819467:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 354 00000100:00100000:0.0:1788619918.819476:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed60e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+17:2354:x1875503571492352:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619918.819504:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619918.819512:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368734f51 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2ba expref: 17 pid: 11397 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.819524:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368734f51 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2ba expref: 17 pid: 11397 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.819567:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368734f51 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c2ba expref: 17 pid: 11397 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.819593:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619918.819598:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678000/0x1cd7fe536873501c lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.819608:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678000/0x1cd7fe536873501c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.819617:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678000/0x1cd7fe536873501c lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.819645:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619918.819706:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed60e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+16:2354:x1875503571492352:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 230us (770us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619918.819713:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 354 00010000:00010000:2.0:1788619918.819976:0:5845:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678000/0x1cd7fe536873501c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.819984:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678000/0x1cd7fe536873501c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788619918.820113:0:5845:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967695 00000004:00010000:2.0:1788619918.820165:0:5845:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 994373 -> 994373 00000004:00010000:2.0:1788619918.820169:0:5845:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619918 -> 1788619918 00000004:00010000:2.0:1788619918.820178:0:5845:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619916 -> 1788619916 00000004:00010000:2.0:1788619918.820180:0:5845:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619918 -> 1788619918 00000004:00010000:2.0:1788619918.820183:0:5845:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 518 -> 518 00000004:00010000:2.0:1788619918.820190:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000ab0dbf13 reply state 00000000830b9dd2 transno 4294967696 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678000/0x1cd7fe536873501c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619918.820199:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788619918.820276:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+28:14908:x1875503571492224:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 15951us (16370us total) trans 4294967696 rc 0/0 00000100:00100000:2.0:1788619918.820283:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1063 00000020:00080000:2.0:1788619918.820287:0:5845:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:1.0:1788619918.826772:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00010000:00010000:2.0:1788619918.827403:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678000/0x1cd7fe536873501c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.827417:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678000/0x1cd7fe536873501c lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.827426:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678000/0x1cd7fe536873501c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.827432:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c7678000 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:2.0:1788619918.827438:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678000/0x1cd7fe536873501c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.827447:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678000/0x1cd7fe536873501c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.827463:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619918.827466:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788619918.827472:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678000/0x1cd7fe536873501c lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619918.827498:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678000/0x1cd7fe536873501c lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619918.827661:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571492480 00000100:00100000:0.0:1788619918.827671:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571492480 00000100:00100000:0.0:1788619918.827769:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1064 00000100:00100000:0.0:1788619918.827780:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed60700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+27:14908:x1875503571492480:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619918.827804:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619918.827807:0:5846:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:0.0:1788619918.827842:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678a00/0x1cd7fe5368735015 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c377 expref: 27 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.827853:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619918.827863:0:5846:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788619918.827876:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368735023 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c37e expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.827900:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679800/0x1cd7fe536873500e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c370 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.827921:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368735023 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c37e expref: 27 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.827960:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679800/0x1cd7fe536873500e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c370 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.827972:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679800/0x1cd7fe536873500e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c370 expref: 15 pid: 5846 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619918.827986:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c74f7480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589135232:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619918.828084:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c2c5e300 going to sleep for 11 seconds 00000100:00100000:1.0:1788619918.832783:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619918.833364:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571492608 00000100:00100000:3.0:1788619918.833374:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571492608 00010000:00010000:3.0:1788619918.833422:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679800/0x1cd7fe536873500e lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c370 expref: 16 pid: 5846 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619918.833495:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 355 00000100:00100000:3.0:1788619918.833506:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8558c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+16:2354:x1875503571492608:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619918.833520:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619918.833547:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679800/0x1cd7fe536873500e lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c370 expref: 16 pid: 5846 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619918.833561:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679800/0x1cd7fe536873500e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c370 expref: 16 pid: 5846 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619918.833596:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619918.833604:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368735023 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c37e expref: 27 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619918.833617:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368735023 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c37e expref: 27 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619918.833627:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368735023 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c37e expref: 27 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619918.833666:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368735023 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c37e expref: 27 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619918.833681:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b14700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589135360:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619918.833774:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaab40 going to sleep for 11 seconds 00010000:00010000:0.0:1788619918.834506:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679800/0x1cd7fe536873500e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c370 expref: 16 pid: 5846 timeout: 511 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619918.834529:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c74f7480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589135232:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619918.834558:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368735023 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c37e expref: 27 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619918.834570:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619918.834573:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619918.834598:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6f000, rc 0) 00000100:00100000:0.0:1788619918.834638:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed60700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+27:14908:x1875503571492480:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6859us (7884us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619918.834646:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1064 00000100:00100000:1.0:1788619918.841288:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619918.841702:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b14700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589135360:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619918.841739:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619918.841824:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8558c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+15:2354:x1875503571492608:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 8318us (9047us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619918.841835:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 355 00000100:00100000:3.0:1788619918.841931:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571492736 00000100:00100000:3.0:1788619918.841938:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571492736 00002000:00010000:3.0:1788619918.841972:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1869263->18446744073709551615) 00002000:00100000:3.0:1788619918.841976:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fed63100 with x1875503571492736 ext(1869263->18446744073709551615) 00002000:00010000:3.0:1788619918.841987:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fed63100. 00000100:00100000:3.0:1788619918.842028:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 459 00000100:00100000:3.0:1788619918.842038:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed63100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+11:2355:x1875503571492736:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:3.0:1788619918.842278:0:6703:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1646471 -> 1869263 00002000:00010000:3.0:1788619918.842282:0:6703:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619918 -> 1788619918 00002000:00010000:3.0:1788619918.842285:0:6703:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619918 -> 1788619916 00002000:00010000:3.0:1788619918.842287:0:6703:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619918 -> 1788619918 00002000:00010000:3.0:1788619918.842289:0:6703:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 135 -> 135 00000100:00100000:3.0:1788619918.842315:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed63100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2355:x1875503571492736:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 278us (1035us total) trans 4294967436 rc 0/0 00000100:00100000:3.0:1788619918.842323:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 459 00002000:00010000:3.0:1788619918.842357:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1869263->18446744073709551615) 00002000:00100000:3.0:1788619918.842360:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fed63100 with x1875503571492736 ext(1869263->18446744073709551615) 00002000:00010000:3.0:1788619918.842365:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fed63100. 00000100:00100000:1.0:1788619918.848375:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619918.848762:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571492864 00000100:00100000:2.0:1788619918.848774:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571492864 00002000:00100000:2.0:1788619918.848826:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1646592->1871871) req@ffff8e78fed60000 x1875503571492864/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:14/0 lens 488/0 e 0 to 0 dl 1788619929 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619918.848839:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fed60000 with x1875503571492864 ext(1646592->1871871) 00002000:00010000:2.0:1788619918.848846:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fed60000 00000100:00100000:2.0:1788619918.848885:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 460 00000100:00100000:2.0:1788619918.848896:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed60000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2352:x1875503571492864:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619918.854816:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed60000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2352:x1875503571492864:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 5910us (6440us total) trans 0 rc 225280/225280 00000100:00100000:2.0:1788619918.854829:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 460 00002000:00100000:2.0:1788619918.854841:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1646592->1871871) req@ffff8e78fed60000 x1875503571492864/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:14/0 lens 488/440 e 0 to 0 dl 1788619929 ref 1 fl Complete:/600/0 rc 225280/225280 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619918.854851:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fed60000 with x1875503571492864 ext(1646592->1871871) 00002000:00010000:2.0:1788619918.854857:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fed60000 00000100:00100000:3.0:1788619918.864573:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619918.864745:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571492992 00000100:00100000:1.0:1788619918.864766:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571492992 00002000:00100000:1.0:1788619918.864820:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1818624->1869262) req@ffff8e77c6b14700 x1875503571492992/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:14/0 lens 488/0 e 0 to 0 dl 1788619929 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619918.864831:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6b14700 with x1875503571492992 ext(1818624->1869262) 00002000:00010000:1.0:1788619918.864836:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6b14700 00000100:00100000:1.0:1788619918.864874:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 461 00000100:00100000:1.0:1788619918.864884:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b14700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2355:x1875503571492992:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:2.0:1788619918.873464:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b14700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571492992:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 8568us (8887us total) trans 4294967437 rc 0/0 00000100:00100000:2.0:1788619918.873478:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 461 00002000:00100000:2.0:1788619918.873492:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1818624->1869262) req@ffff8e77c6b14700 x1875503571492992/t4294967437(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:14/0 lens 488/448 e 0 to 0 dl 1788619929 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619918.873503:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6b14700 with x1875503571492992 ext(1818624->1869262) 00002000:00010000:2.0:1788619918.873518:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6b14700 00000100:00100000:0.0:1788619918.878296:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619918.883173:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571493120 00000100:00100000:1.0:1788619918.883193:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571493120 00000100:00100000:1.0:1788619918.883363:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 832 00000100:00100000:1.0:1788619918.883375:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571493120:12345-192.168.201.5@tcp:16:fsx.0 00000100:00100000:3.0:1788619918.947199:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b17100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589135872:0@lo:400:kworker.0 00000100:00100000:3.0:1788619918.947275:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619918.947374:0:3318:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:3.0:1788619918.947381:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b14000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:3318:1875503589135488:0@lo:400:kworker.0 00000100:00100000:3.0:1788619918.947413:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619918.947437:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3318:1875503589136000:0@lo:400:kworker.0 00000100:00100000:3.0:1788619918.947465:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619918.947494:0:3317:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b15180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3317:1875503589135744:0@lo:400:kworker.0 00000100:00100000:3.0:1788619918.947515:0:3317:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619918.947533:0:3317:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1788619918.947539:0:3317:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b16a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3317:1875503589135616:0@lo:400:kworker.0 00000100:00100000:3.0:1788619918.947557:0:3317:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619918.947852:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589136000 00000100:00100000:1.0:1788619918.947860:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589136000 00000100:00100000:1.0:1788619918.947913:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589135744 00000100:00100000:1.0:1788619918.947916:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589135744 00000100:00100000:1.0:1788619918.947935:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589135616 00000100:00100000:1.0:1788619918.947938:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589135616 00000100:00100000:1.0:1788619918.947952:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 186 00000100:00100000:1.0:1788619918.947960:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3318:x1875503589136000:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788619918.948026:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3318:x1875503589136000:12345-0@lo:400:kworker.0 Request processed in 55us (548us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619918.948044:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 186 00000100:00100000:1.0:1788619918.948061:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 187 00000100:00100000:1.0:1788619918.948067:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3317:x1875503589135744:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788619918.948098:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3317:x1875503589135744:12345-0@lo:400:kworker.0 Request processed in 30us (581us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619918.948103:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 187 00000100:00100000:1.0:1788619918.948115:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 188 00000100:00100000:1.0:1788619918.948120:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3317:x1875503589135616:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788619918.948147:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3317:x1875503589135616:12345-0@lo:400:kworker.0 Request processed in 27us (588us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619918.948152:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 188 00000100:00080000:1.0:1788619918.948167:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788619918.948381:0:3317:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3317:1875503589136128:0@lo:400:kworker.0 00000100:00100000:3.0:1788619918.948406:0:3317:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619918.948441:0:3317:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b15180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3317:1875503589135744:0@lo:400:kworker.0 00000100:00100000:3.0:1788619918.948462:0:3317:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b16a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3317:1875503589135616:0@lo:400:kworker.0 00000100:00100000:3.0:1788619918.948479:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589135488 00000100:00100000:3.0:1788619918.948483:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589135488 00000100:00100000:3.0:1788619918.948507:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 290 00000100:00100000:3.0:1788619918.948514:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5ce2c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+19:3318:x1875503589135488:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788619918.948544:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5ce2c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+19:3318:x1875503589135488:12345-0@lo:400:kworker.0 Request processed in 30us (1129us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619918.948550:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 290 00000100:00100000:3.0:1788619918.948597:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589135872 00000100:00100000:3.0:1788619918.948602:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589135872 00000100:00100000:3.0:1788619918.948633:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589136128 00000100:00100000:3.0:1788619918.948636:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589136128 00000100:00100000:3.0:1788619918.948649:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 126 00000100:00100000:3.0:1788619918.948661:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa15880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589135872:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788619918.948692:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa15880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589135872:12345-0@lo:400:kworker.0 Request processed in 31us (1418us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619918.948697:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 126 00000100:00100000:3.0:1788619918.948709:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 127 00000100:00100000:3.0:1788619918.948714:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3317:x1875503589136128:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788619918.948743:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3317:x1875503589136128:12345-0@lo:400:kworker.0 Request processed in 29us (335us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619918.948759:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 127 00000100:00100000:3.0:1788619918.948789:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b14000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:3318:1875503589135488:0@lo:400:kworker.0 00000100:00100000:3.0:1788619918.948799:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010ff800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3318:1875503589136000:0@lo:400:kworker.0 00000100:00100000:3.0:1788619918.948808:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b17100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589135872:0@lo:400:kworker.0 00000100:00100000:3.0:1788619918.948833:0:3317:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3317:1875503589136128:0@lo:400:kworker.0 00000100:00080000:1.0:1788619918.956536:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1788619918.956546:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:1.0:1788619919.038282:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967435 is committed 00000001:00080000:1.0:1788619919.038300:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967434 is committed 00000001:00080000:1.0:1788619919.038363:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967437 is committed 00000100:00100000:0.0:1788619919.038640:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571493120:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 155261us (160349us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619919.038652:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 832 00000100:00100000:0.0:1788619919.042959:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619919.043511:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571493248 00000100:00100000:1.0:1788619919.043521:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571493248 00000100:00100000:1.0:1788619919.043592:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1065 00000100:00100000:1.0:1788619919.043603:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+27:14908:x1875503571493248:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619919.043624:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619919.043646:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753c00/0x1cd7fe536873502a lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c385 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.043728:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753400/0x1cd7fe5368735031 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.043770:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753400/0x1cd7fe5368735031 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.043783:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753400/0x1cd7fe5368735031 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788619919.043813:0:9417:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1869263 00000004:00200000:1.0:1788619919.043817:0:9417:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788619919.043834:0:9417:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753400/0x1cd7fe5368735031 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.043861:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753c00/0x1cd7fe536873502a lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c385 expref: 29 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.043878:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753400/0x1cd7fe5368735031 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c385 expref: 28 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.043886:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619919.043888:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788619919.043891:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5753400, rc 0) 00000100:00100000:1.0:1788619919.043949:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+28:14908:x1875503571493248:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 348us (1000us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619919.043956:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1065 00000100:00100000:1.0:1788619919.055166:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619919.055253:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571493376 00000100:00100000:2.0:1788619919.055262:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571493376 00000100:00100000:2.0:1788619919.055319:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1066 00000100:00100000:2.0:1788619919.055352:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed61180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+15:14908:x1875503571493376:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619919.055373:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619919.055391:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678c00/0x1cd7fe5368735038 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c38c expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619919.055480:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678e00/0x1cd7fe536873503f lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619919.055498:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678e00/0x1cd7fe536873503f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619919.055510:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678e00/0x1cd7fe536873503f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788619919.055539:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1869263 00000004:00200000:2.0:1788619919.055543:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788619919.055562:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678e00/0x1cd7fe536873503f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619919.055604:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678c00/0x1cd7fe5368735038 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c38c expref: 17 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619919.055622:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678e00/0x1cd7fe536873503f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c38c expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619919.055629:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619919.055646:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788619919.055649:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c7678e00, rc 0) 00000100:00100000:2.0:1788619919.055714:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed61180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+16:14908:x1875503571493376:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 387us (559us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619919.055721:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1066 00000100:00100000:0.0:1788619919.058536:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619919.060594:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571493504 00000100:00100000:3.0:1788619919.060611:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571493504 00000100:00100000:3.0:1788619919.060687:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1067 00000100:00100000:3.0:1788619919.060698:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571493504:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619919.060721:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619919.060776:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368735046 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c393 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619919.060817:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368735023 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c37e expref: 27 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619919.060859:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df19ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589136256:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619919.060986:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619919.077789:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619919.078145:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571493632 00000100:00100000:3.0:1788619919.078156:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571493632 00000100:00100000:3.0:1788619919.078214:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 833 00000100:00100000:3.0:1788619919.078224:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571493632:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619919.078243:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619919.078262:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe536873504d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c39a expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619919.078381:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df19f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589136384:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619919.078458:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000004:00010000:3.0:1788619919.087797:0:5845:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619916 -> 1788619918 00000100:00100000:3.0:1788619919.087830:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df19ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589136256:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619919.087854:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:3.0:1788619919.087873:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368735046 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c393 expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619919.087895:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619919.087898:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619919.087904:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368735046 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c393 expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619919.087921:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d000, rc 0) 00000100:00100000:3.0:1788619919.087987:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571493504:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 27290us (29461us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619919.087996:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1067 00002000:00010000:3.0:1788619919.090551:0:6696:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619916 -> 1788619918 00002000:00010000:3.0:1788619919.090560:0:6696:0:(ofd_lvb.c:284:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks: 135 -> 263 00000100:00100000:3.0:1788619919.090594:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df19f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589136384:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619919.090630:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe536873504d lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c39a expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619919.090654:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe536873504d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c39a expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619919.090668:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c600, rc 0) 00000100:00100000:3.0:1788619919.090725:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571493632:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 12501us (12945us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619919.090733:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 833 00000100:00100000:0.0:1788619919.111668:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619919.111710:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619919.111810:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571493760 00000100:00100000:1.0:1788619919.111820:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571493760 00000100:00100000:1.0:1788619919.111885:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1068 00000100:00100000:1.0:1788619919.111896:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571493760:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619919.111936:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619919.111959:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753800/0x1cd7fe5368735054 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c3a1 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.111999:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368735023 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c37e expref: 27 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619919.112051:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed62d80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589136512:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619919.112136:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a900 going to sleep for 11 seconds 00000100:00100000:3.0:1788619919.112315:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571493888 00000100:00100000:3.0:1788619919.112320:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571493888 00000100:00100000:3.0:1788619919.112386:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 834 00000100:00100000:3.0:1788619919.112394:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571493888:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619919.112402:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619919.112413:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe536873505b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c3a8 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619919.112452:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df19f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589136640:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619919.112486:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:3.0:1788619919.118815:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df19f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589136640:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619919.118860:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe536873505b lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c3a8 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619919.118900:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe536873505b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c3a8 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619919.118913:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c600, rc 0) 00000100:00100000:3.0:1788619919.118975:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571493888:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6581us (7263us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619919.118983:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 834 00000100:00100000:1.0:1788619919.119114:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed62d80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589136512:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619919.119134:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:1.0:1788619919.119148:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753800/0x1cd7fe5368735054 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c3a1 expref: 17 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.119160:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619919.119164:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619919.119168:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753800/0x1cd7fe5368735054 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c3a1 expref: 17 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.119178:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5753800, rc 0) 00000100:00100000:1.0:1788619919.119203:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571493760:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7310us (7546us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619919.119210:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1068 00000100:00100000:0.0:1788619919.128973:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619919.129060:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571494016 00000100:00100000:3.0:1788619919.129071:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571494016 00000100:00100000:3.0:1788619919.129135:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1069 00000100:00100000:3.0:1788619919.129145:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571494016:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619919.129168:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619919.129188:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368735062 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c3af expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619919.129226:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368735023 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c37e expref: 27 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619919.129264:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df19f480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589136768:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619919.129369:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:3.0:1788619919.131312:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df19f480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589136768:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619919.131370:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:3.0:1788619919.131389:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368735062 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c3af expref: 17 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619919.131438:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619919.131442:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619919.131447:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368735062 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c3af expref: 17 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619919.131462:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c600, rc 0) 00000100:00100000:3.0:1788619919.131507:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571494016:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2362us (2544us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619919.131515:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1069 00000100:00100000:2.0:1788619919.135319:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619919.135463:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571494144 00000100:00100000:0.0:1788619919.135474:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571494144 00000100:00100000:0.0:1788619919.135531:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 835 00000100:00100000:0.0:1788619919.135541:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df966a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571494144:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619919.135558:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619919.135584:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000738676cd/0x1cd7fe5368735069 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c3b6 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619919.135640:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589136896:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619919.135720:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb840 going to sleep for 11 seconds 00000100:00100000:1.0:1788619919.139295:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589136896:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619919.139364:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000738676cd/0x1cd7fe5368735069 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c3b6 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.139422:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000738676cd/0x1cd7fe5368735069 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c3b6 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.139436:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6ec00, rc 0) 00000100:00100000:1.0:1788619919.139491:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df966a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571494144:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3951us (4182us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619919.139499:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 835 00010000:00010000:1.0:1788619919.140696:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788619919.142671:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619919.142713:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619919.142791:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571494400 00000100:00100000:1.0:1788619919.142801:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571494400 00000100:00100000:1.0:1788619919.142860:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 836 00000100:00100000:1.0:1788619919.142870:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df965500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571494400:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619919.142887:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619919.142912:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e4c897/0x1cd7fe5368735070 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c3c4 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619919.142971:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed62d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589137024:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619919.143061:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a900 going to sleep for 11 seconds 00000100:00100000:3.0:1788619919.143855:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571494272 00000100:00100000:3.0:1788619919.143861:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571494272 00000100:00100000:3.0:1788619919.143900:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1070 00000100:00100000:3.0:1788619919.143908:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df964700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+16:2354:x1875503571494272:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619919.143919:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619919.143933:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368735077 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c3bd expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619919.143964:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368735023 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c37e expref: 27 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619919.143990:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df19f480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589137152:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619919.144053:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:1.0:1788619919.146533:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed62d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589137024:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619919.146591:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000029e4c897/0x1cd7fe5368735070 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c3c4 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.146616:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029e4c897/0x1cd7fe5368735070 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c3c4 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.146648:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5753800, rc 0) 00000100:00100000:1.0:1788619919.146709:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df965500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571494400:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3840us (3994us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619919.146718:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 836 00000100:00100000:3.0:1788619919.148283:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df19f480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589137152:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619919.148309:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:3.0:1788619919.148362:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368735077 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c3bd expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619919.148399:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619919.148402:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619919.148408:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368735077 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c3bd expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619919.148423:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c600, rc 0) 00000100:00100000:3.0:1788619919.148478:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df964700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+16:2354:x1875503571494272:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4571us (5815us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619919.148487:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1070 00000100:00100000:0.0:1788619919.154970:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619919.155106:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571494528 00000100:00100000:1.0:1788619919.155118:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571494528 00000100:00100000:1.0:1788619919.155185:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 462 00000100:00100000:1.0:1788619919.155196:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571494528:12345-192.168.201.5@tcp:22:fsx.0 00002000:00010000:1.0:1788619919.155545:0:6702:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1869263 -> 1869263 00002000:00010000:1.0:1788619919.155549:0:6702:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619918 -> 1788619918 00002000:00010000:1.0:1788619919.155552:0:6702:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619918 -> 1788619916 00002000:00010000:1.0:1788619919.155555:0:6702:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619918 -> 1788619918 00002000:00010000:1.0:1788619919.155557:0:6702:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 263 -> 263 00000100:00100000:1.0:1788619919.155624:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2352:x1875503571494528:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 429us (663us total) trans 4294967438 rc 0/0 00000100:00100000:1.0:1788619919.155632:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 462 00000100:00100000:2.0:1788619919.164412:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619919.164507:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619919.164622:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571494656 00000100:00100000:1.0:1788619919.164631:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571494656 00000100:00100000:1.0:1788619919.164684:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1071 00000100:00100000:1.0:1788619919.164693:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df965880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+16:2353:x1875503571494656:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619919.164714:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619919.164733:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753800/0x1cd7fe536873507e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c3cb expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.164801:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368735023 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c37e expref: 27 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619919.164840:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed62d80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589137280:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619919.164909:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a900 going to sleep for 11 seconds 00000100:00100000:3.0:1788619919.168288:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571494784 00000100:00100000:3.0:1788619919.168304:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571494784 00000004:00010000:1.0:1788619919.169598:0:5846:0:(mdt_lvb.c:195:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime: 1788619918 -> 1788619919 00000100:00100000:1.0:1788619919.169630:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed62d80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589137280:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619919.169659:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:1.0:1788619919.169678:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753800/0x1cd7fe536873507e lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c3cb expref: 17 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.169698:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619919.169701:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619919.169706:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753800/0x1cd7fe536873507e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c3cb expref: 17 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.169720:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5753800, rc 0) 00000100:00100000:1.0:1788619919.169792:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df965880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+16:2353:x1875503571494656:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5100us (5390us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619919.169801:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1071 00000100:00100000:2.0:1788619919.174933:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 837 00000100:00100000:2.0:1788619919.174948:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df964e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571494784:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619919.174968:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619919.174987:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002a87657c/0x1cd7fe5368735085 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c3d2 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619919.175065:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df96b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589137408:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619919.175157:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619919.175740:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619919.175901:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571494912 00000100:00100000:0.0:1788619919.175909:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571494912 00000100:00100000:0.0:1788619919.175961:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 291 00000100:00100000:0.0:1788619919.175969:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5ce0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:597450a2-e90c-408c-b401-a0ce48eed204+13:2354:x1875503571494912:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:0.0:1788619919.176021:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5ce0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:597450a2-e90c-408c-b401-a0ce48eed204+13:2354:x1875503571494912:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 53us (283us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619919.176029:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 291 00002000:00010000:2.0:1788619919.177691:0:6696:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619916 -> 1788619918 00002000:00010000:2.0:1788619919.177699:0:6696:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime: 1788619918 -> 1788619919 00000100:00100000:2.0:1788619919.177723:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df96b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589137408:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619919.177766:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002a87657c/0x1cd7fe5368735085 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c3d2 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619919.177789:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002a87657c/0x1cd7fe5368735085 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c3d2 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619919.177818:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c7678400, rc 0) 00000100:00100000:2.0:1788619919.177865:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df964e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571494784:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2920us (13357us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619919.177872:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 837 00000100:00100000:0.0:1788619919.187513:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619919.187557:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619919.187576:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571495040 00000100:00100000:3.0:1788619919.187586:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571495040 00000100:00100000:3.0:1788619919.187647:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1072 00000100:00100000:3.0:1788619919.187657:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+16:2352:x1875503571495040:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619919.187677:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619919.187697:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe536873508c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c3d9 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619919.187734:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571495168 00010000:00010000:3.0:1788619919.187736:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368735023 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c37e expref: 27 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619919.187738:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571495168 00000100:00100000:3.0:1788619919.187787:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df19f480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589137536:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619919.187790:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 838 00000100:00100000:2.0:1788619919.187797:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571495168:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619919.187804:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619919.187814:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fed5453b/0x1cd7fe5368735093 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c3e0 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619919.187849:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df96b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589137664:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619919.187865:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:2.0:1788619919.187872:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619919.190191:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df96b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589137664:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619919.190247:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000fed5453b/0x1cd7fe5368735093 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c3e0 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.190320:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fed5453b/0x1cd7fe5368735093 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c3e0 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.190369:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c7678000, rc 0) 00000100:00100000:1.0:1788619919.190436:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571495168:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2639us (2877us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619919.190444:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 838 00000100:00100000:0.0:1788619919.191296:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df19f480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589137536:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619919.191357:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:0.0:1788619919.191378:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe536873508c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c3d9 expref: 17 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619919.191402:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619919.191407:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619919.191415:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe536873508c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c3d9 expref: 17 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619919.191434:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d000, rc 0) 00000100:00100000:0.0:1788619919.191491:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+16:2352:x1875503571495040:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3836us (3988us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619919.191499:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1072 00000100:00100000:3.0:1788619919.196008:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619919.196084:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571495296 00000100:00100000:2.0:1788619919.196095:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571495296 00002000:00100000:2.0:1788619919.196146:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df19ea00 x1875503571495296/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:15/0 lens 488/0 e 0 to 0 dl 1788619930 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619919.196157:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df19ea00 with x1875503571495296 00000100:00100000:2.0:1788619919.196199:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 668 00000100:00100000:2.0:1788619919.196209:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+28:2353:x1875503571495296:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619919.196452:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+28:2353:x1875503571495296:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 243us (452us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619919.196460:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 668 00002000:00100000:2.0:1788619919.196473:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df19ea00 x1875503571495296/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:15/0 lens 488/4536 e 0 to 0 dl 1788619930 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619919.196481:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df19ea00 with x1875503571495296 00000100:00100000:3.0:1788619919.200062:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619919.200161:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571495424 00000100:00100000:1.0:1788619919.200171:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571495424 00002000:00100000:1.0:1788619919.200214:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df19c700 x1875503571495424/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:15/0 lens 488/0 e 0 to 0 dl 1788619930 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619919.200225:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df19c700 with x1875503571495424 00000100:00100000:1.0:1788619919.200261:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 669 00000100:00100000:1.0:1788619919.200270:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+28:2352:x1875503571495424:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619919.200494:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+28:2352:x1875503571495424:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 224us (442us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619919.200502:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 669 00002000:00100000:1.0:1788619919.200509:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df19c700 x1875503571495424/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:15/0 lens 488/4536 e 0 to 0 dl 1788619930 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619919.200516:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df19c700 with x1875503571495424 00000100:00100000:3.0:1788619919.203676:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619919.203762:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571495552 00000100:00100000:0.0:1788619919.203771:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571495552 00002000:00100000:0.0:1788619919.203820:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df19dc00 x1875503571495552/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:15/0 lens 488/0 e 0 to 0 dl 1788619930 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619919.203831:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df19dc00 with x1875503571495552 00000100:00100000:0.0:1788619919.203870:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 670 00000100:00100000:0.0:1788619919.203895:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+28:2352:x1875503571495552:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619919.204218:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+28:2352:x1875503571495552:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 323us (551us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619919.204225:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 670 00002000:00100000:0.0:1788619919.204232:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df19dc00 x1875503571495552/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:15/0 lens 488/4536 e 0 to 0 dl 1788619930 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619919.204240:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df19dc00 with x1875503571495552 00000100:00100000:0.0:1788619919.208839:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619919.213206:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571495680 00000100:00100000:3.0:1788619919.213222:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571495680 00002000:00100000:3.0:1788619919.213287:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e67f3b80 x1875503571495680/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:15/0 lens 488/0 e 0 to 0 dl 1788619930 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619919.213299:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e67f3b80 with x1875503571495680 00000100:00100000:3.0:1788619919.213371:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 671 00000100:00100000:3.0:1788619919.213382:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+28:2352:x1875503571495680:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619919.213655:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+28:2352:x1875503571495680:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 274us (4825us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619919.213663:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 671 00002000:00100000:3.0:1788619919.213670:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e67f3b80 x1875503571495680/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:15/0 lens 488/4536 e 0 to 0 dl 1788619930 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619919.213677:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e67f3b80 with x1875503571495680 00000100:00100000:0.0:1788619919.217144:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619919.220187:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571495808 00000100:00100000:1.0:1788619919.220203:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571495808 00002000:00100000:0.0:1788619919.222385:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e67f1f80 x1875503571495808/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:15/0 lens 488/0 e 0 to 0 dl 1788619930 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619919.222400:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e67f1f80 with x1875503571495808 00000100:00100000:0.0:1788619919.222492:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 672 00000100:00100000:0.0:1788619919.222503:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+28:2353:x1875503571495808:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619919.222754:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+28:2353:x1875503571495808:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 243us (5611us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619919.222762:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 672 00002000:00100000:0.0:1788619919.222768:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e67f1f80 x1875503571495808/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:15/0 lens 488/4536 e 0 to 0 dl 1788619930 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619919.222776:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e67f1f80 with x1875503571495808 00000100:00100000:3.0:1788619919.228614:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619919.228668:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571495936 00000100:00100000:2.0:1788619919.228680:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571495936 00002000:00100000:2.0:1788619919.228733:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df19c380 x1875503571495936/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:15/0 lens 488/0 e 0 to 0 dl 1788619930 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619919.228753:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df19c380 with x1875503571495936 00000100:00100000:2.0:1788619919.228788:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 673 00000100:00100000:2.0:1788619919.228797:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+28:2354:x1875503571495936:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619919.229012:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+28:2354:x1875503571495936:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 215us (407us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619919.229031:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 673 00002000:00100000:2.0:1788619919.229039:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df19c380 x1875503571495936/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:15/0 lens 488/4536 e 0 to 0 dl 1788619930 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619919.229047:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df19c380 with x1875503571495936 00000100:00100000:2.0:1788619919.237902:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619919.241235:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571496064 00000100:00100000:3.0:1788619919.241252:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571496064 00002000:00100000:3.0:1788619919.241312:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df969500 x1875503571496064/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:15/0 lens 488/0 e 0 to 0 dl 1788619930 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619919.241337:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df969500 with x1875503571496064 00000100:00100000:3.0:1788619919.241438:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 674 00000100:00100000:3.0:1788619919.241448:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df969500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+28:2352:x1875503571496064:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619919.241681:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df969500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+28:2352:x1875503571496064:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 234us (3790us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619919.241704:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 674 00002000:00100000:3.0:1788619919.241711:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df969500 x1875503571496064/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:15/0 lens 488/4536 e 0 to 0 dl 1788619930 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619919.241719:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df969500 with x1875503571496064 00000100:00100000:0.0:1788619919.249145:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619919.249970:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571496192 00000100:00100000:2.0:1788619919.249983:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571496192 00002000:00100000:2.0:1788619919.250110:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e67f2300 x1875503571496192/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:15/0 lens 488/0 e 0 to 0 dl 1788619930 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619919.250122:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e67f2300 with x1875503571496192 00000100:00100000:2.0:1788619919.250165:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 675 00000100:00100000:2.0:1788619919.250176:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+28:2354:x1875503571496192:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:3.0:1788619919.262411:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 994373 -> 994373 00000004:00010000:3.0:1788619919.262421:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619918 -> 1788619919 00000004:00010000:3.0:1788619919.262424:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619918 -> 1788619916 00000004:00010000:3.0:1788619919.262426:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619919 -> 1788619919 00000004:00010000:3.0:1788619919.262427:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 518 -> 518 00000100:00100000:3.0:1788619919.265555:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+30:2354:x1875503571496192:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 15368us (16407us total) trans 4294967697 rc 0/0 00000100:00100000:3.0:1788619919.265572:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 675 00002000:00100000:3.0:1788619919.265586:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e67f2300 x1875503571496192/t4294967697(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:15/0 lens 488/448 e 0 to 0 dl 1788619930 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619919.265598:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e67f2300 with x1875503571496192 00000100:00100000:2.0:1788619919.268459:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619919.268552:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571496320 00000100:00100000:3.0:1788619919.268562:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571496320 00000100:00100000:3.0:1788619919.268619:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1073 00000100:00100000:3.0:1788619919.268630:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df96b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+30:2354:x1875503571496320:12345-192.168.201.5@tcp:16:fsx.0 00000001:00080000:1.0:1788619919.345938:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967689 is committed 00000001:00080000:1.0:1788619919.345972:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967686 is committed 00000001:00080000:1.0:1788619919.346006:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967696 is committed 00000001:00080000:1.0:1788619919.347151:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967692 is committed 00000100:00100000:1.0:1788619919.347347:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df96b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2354:x1875503571496320:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 78696us (78874us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619919.347360:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1073 00000001:00080000:0.0:1788619919.348252:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967697 is committed 00000001:00080000:0.0:1788619919.348270:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967693 is committed 00000100:00100000:0.0:1788619919.358197:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619919.358360:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571496448 00000100:00100000:3.0:1788619919.358372:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571496448 00002000:00100000:3.0:1788619919.358422:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e67f0a80 x1875503571496448/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:15/0 lens 488/0 e 0 to 0 dl 1788619930 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619919.358434:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e67f0a80 with x1875503571496448 00000100:00100000:3.0:1788619919.358467:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 676 00000100:00100000:3.0:1788619919.358476:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571496448:12345-192.168.201.5@tcp:3:fsx.0 00000004:00010000:3.0:1788619919.358755:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 518 -> 646 00000100:00100000:3.0:1788619919.358839:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571496448:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 363us (654us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619919.358848:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 676 00002000:00100000:3.0:1788619919.358855:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e67f0a80 x1875503571496448/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:15/0 lens 488/4536 e 0 to 0 dl 1788619930 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619919.358864:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e67f0a80 with x1875503571496448 00000100:00100000:1.0:1788619919.366425:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619919.369204:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571496576 00000100:00100000:3.0:1788619919.369220:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571496576 00002000:00100000:3.0:1788619919.369279:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fed62d80 x1875503571496576/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:15/0 lens 488/0 e 0 to 0 dl 1788619930 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619919.369292:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fed62d80 with x1875503571496576 00000100:00100000:3.0:1788619919.369384:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 677 00000100:00100000:3.0:1788619919.369394:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed62d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571496576:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619919.371130:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed62d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571496576:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 1729us (4706us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619919.371143:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 677 00002000:00100000:3.0:1788619919.371153:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fed62d80 x1875503571496576/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:15/0 lens 488/4536 e 0 to 0 dl 1788619930 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619919.371163:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fed62d80 with x1875503571496576 00000100:00100000:0.0:1788619919.378991:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619919.380690:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571496704 00000100:00100000:1.0:1788619919.380703:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571496704 00002000:00100000:1.0:1788619919.380764:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e67f1180 x1875503571496704/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:15/0 lens 488/0 e 0 to 0 dl 1788619930 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619919.380776:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e67f1180 with x1875503571496704 00000100:00100000:1.0:1788619919.380879:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 678 00000100:00100000:1.0:1788619919.380889:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571496704:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619919.381174:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571496704:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 285us (2193us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619919.381182:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 678 00002000:00100000:1.0:1788619919.381190:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e67f1180 x1875503571496704/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:15/0 lens 488/4536 e 0 to 0 dl 1788619930 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619919.381199:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e67f1180 with x1875503571496704 00000100:00100000:0.0:1788619919.386655:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619919.390527:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571496832 00000100:00100000:2.0:1788619919.390543:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571496832 00002000:00100000:2.0:1788619919.390613:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e67f3100 x1875503571496832/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:15/0 lens 488/0 e 0 to 0 dl 1788619930 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619919.390624:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e67f3100 with x1875503571496832 00000100:00100000:2.0:1788619919.390651:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 679 00000100:00100000:2.0:1788619919.390661:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571496832:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619919.390941:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571496832:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 280us (4294us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619919.390949:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 679 00002000:00100000:2.0:1788619919.390956:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e67f3100 x1875503571496832/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:15/0 lens 488/4536 e 0 to 0 dl 1788619930 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619919.390964:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e67f3100 with x1875503571496832 00000100:00100000:0.0:1788619919.400771:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619919.403233:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571496960 00000100:00100000:2.0:1788619919.403252:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571496960 00002000:00100000:2.0:1788619919.403316:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e67f0700 x1875503571496960/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:15/0 lens 488/0 e 0 to 0 dl 1788619930 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619919.403390:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e67f0700 with x1875503571496960 00000100:00100000:2.0:1788619919.403498:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 680 00000100:00100000:2.0:1788619919.403511:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571496960:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619919.403859:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571496960:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 350us (3096us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619919.403869:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 680 00002000:00100000:2.0:1788619919.403877:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e67f0700 x1875503571496960/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:15/0 lens 488/4536 e 0 to 0 dl 1788619930 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619919.403897:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e67f0700 with x1875503571496960 00000100:00100000:0.0:1788619919.413023:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619919.414003:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571497088 00000100:00100000:1.0:1788619919.414024:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571497088 00002000:00100000:1.0:1788619919.414077:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e67f1500 x1875503571497088/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:15/0 lens 488/0 e 0 to 0 dl 1788619930 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619919.414088:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e67f1500 with x1875503571497088 00000100:00100000:1.0:1788619919.414112:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 681 00000100:00100000:1.0:1788619919.414135:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571497088:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619919.414418:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571497088:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 282us (1418us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619919.414426:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 681 00002000:00100000:1.0:1788619919.414432:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e67f1500 x1875503571497088/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:15/0 lens 488/4536 e 0 to 0 dl 1788619930 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619919.414440:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e67f1500 with x1875503571497088 00000100:00100000:0.0:1788619919.422213:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619919.423683:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571497216 00000100:00100000:1.0:1788619919.423697:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571497216 00002000:00100000:0.0:1788619919.423838:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e67f3480 x1875503571497216/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:15/0 lens 488/0 e 0 to 0 dl 1788619930 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619919.423851:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e67f3480 with x1875503571497216 00000100:00100000:0.0:1788619919.423946:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 682 00000100:00100000:0.0:1788619919.423957:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571497216:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619919.424235:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571497216:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 280us (2033us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619919.424243:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 682 00002000:00100000:0.0:1788619919.424250:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e67f3480 x1875503571497216/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:15/0 lens 488/4536 e 0 to 0 dl 1788619930 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619919.424259:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e67f3480 with x1875503571497216 00000100:00100000:3.0:1788619919.431344:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619919.431836:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571497344 00000100:00100000:2.0:1788619919.431846:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571497344 00002000:00100000:2.0:1788619919.431898:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df19fb80 x1875503571497344/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:15/0 lens 488/0 e 0 to 0 dl 1788619930 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619919.431909:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df19fb80 with x1875503571497344 00000100:00100000:2.0:1788619919.432035:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 683 00000100:00100000:2.0:1788619919.432045:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571497344:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619919.432304:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571497344:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 259us (987us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619919.432312:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 683 00002000:00100000:2.0:1788619919.432319:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df19fb80 x1875503571497344/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:15/0 lens 488/4536 e 0 to 0 dl 1788619930 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619919.432352:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df19fb80 with x1875503571497344 00000100:00100000:1.0:1788619919.439877:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619919.439990:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571497472 00000100:00100000:2.0:1788619919.440000:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571497472 00002000:00100000:2.0:1788619919.440305:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fed60e00 x1875503571497472/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:15/0 lens 488/0 e 0 to 0 dl 1788619930 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619919.440317:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fed60e00 with x1875503571497472 00000100:00100000:2.0:1788619919.440410:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 684 00000100:00100000:2.0:1788619919.440420:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed60e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571497472:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619919.440697:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed60e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571497472:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 277us (828us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619919.440705:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 684 00002000:00100000:2.0:1788619919.440723:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fed60e00 x1875503571497472/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:15/0 lens 488/4536 e 0 to 0 dl 1788619930 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619919.440731:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fed60e00 with x1875503571497472 00000100:00100000:1.0:1788619919.447552:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619919.447606:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619919.447638:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571497728 00000100:00100000:1.0:1788619919.447646:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571497728 00000100:00100000:1.0:1788619919.447695:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 839 00000100:00100000:1.0:1788619919.447705:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571497728:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619919.447723:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619919.447753:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cfe71009/0x1cd7fe536873509a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c3ee expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619919.447811:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df8b1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589137792:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619919.447890:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a240 going to sleep for 11 seconds 00000100:00100000:3.0:1788619919.451785:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571497600 00000100:00100000:3.0:1788619919.451816:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571497600 00000100:00100000:3.0:1788619919.452020:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1074 00000100:00100000:3.0:1788619919.452030:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571497600:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619919.452050:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619919.452070:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe53687350a1 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c3e7 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619919.452104:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368735023 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c37e expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619919.452141:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df19e300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589137920:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619919.455335:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78ce7f3540 going to sleep for 11 seconds 00002000:00010000:1.0:1788619919.461766:0:6697:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime: 1788619918 -> 1788619919 00002000:00010000:1.0:1788619919.461776:0:6697:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619918 -> 1788619919 00000100:00100000:1.0:1788619919.461800:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df8b1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589137792:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619919.461835:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000cfe71009/0x1cd7fe536873509a lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c3ee expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.461875:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cfe71009/0x1cd7fe536873509a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c3ee expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.461888:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5753e00, rc 0) 00000100:00100000:1.0:1788619919.461944:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571497728:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 14239us (14336us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619919.461952:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 839 00000100:00100000:0.0:1788619919.469294:0:3317:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df19ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3317:1875503589138048:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1788619919.469385:0:3317:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619919.470927:0:6700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589138048 00000100:00100000:1.0:1788619919.470939:0:6700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589138048 00000100:00100000:1.0:1788619919.470994:0:6700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 88 00000100:00100000:1.0:1788619919.471004:0:6700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3317:x1875503589138048:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1788619919.471050:0:6700:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3856662528 avail 3854565376 00000100:00100000:1.0:1788619919.475528:0:6700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3317:x1875503589138048:12345-0@lo:13:osp-pre-1-0.0 Request processed in 4512us (6129us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619919.475548:0:6700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 88 00000004:00010000:3.0:1788619919.475893:0:5846:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619916 -> 1788619919 00000100:00100000:3.0:1788619919.475918:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df19e300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589137920:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619919.475938:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:3.0:1788619919.475958:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe53687350a1 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c3e7 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619919.475979:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619919.475982:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619919.475988:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe53687350a1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c3e7 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619919.476000:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c600, rc 0) 00000100:00100000:3.0:1788619919.476070:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571497600:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 24042us (28529us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619919.476077:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1074 00000100:00100000:1.0:1788619919.480481:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619919.480534:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619919.480677:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571497984 00000100:00100000:2.0:1788619919.480688:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571497984 00000100:00100000:2.0:1788619919.480758:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 840 00000100:00100000:2.0:1788619919.480769:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571497984:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619919.480788:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619919.480805:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000aeaf52b/0x1cd7fe53687350a8 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c3fc expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619919.480867:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df968a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589138176:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619919.480967:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc000 going to sleep for 11 seconds 00000100:00100000:3.0:1788619919.481443:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571497856 00000100:00100000:3.0:1788619919.481449:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571497856 00000100:00100000:3.0:1788619919.481493:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1075 00000100:00100000:3.0:1788619919.481501:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571497856:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619919.481511:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619919.481525:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe53687350af lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c3f5 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619919.481553:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368735023 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c37e expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619919.481591:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df19e300 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589138304:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619919.481625:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78ce7f3540 going to sleep for 11 seconds 00000100:00100000:3.0:1788619919.485585:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df19e300 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589138304:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619919.485610:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:3.0:1788619919.485629:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe53687350af lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c3f5 expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619919.485650:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619919.485653:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619919.485658:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe53687350af lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c3f5 expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619919.485672:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c200, rc 0) 00000100:00100000:3.0:1788619919.485736:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571497856:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4235us (5200us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619919.485755:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1075 00000100:00100000:2.0:1788619919.486344:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df968a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589138176:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619919.486366:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000aeaf52b/0x1cd7fe53687350a8 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c3fc expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619919.486385:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000aeaf52b/0x1cd7fe53687350a8 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c3fc expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619919.486396:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c7678a00, rc 0) 00000100:00100000:2.0:1788619919.486421:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571497984:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5654us (5950us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619919.486429:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 840 00000100:00100000:0.0:1788619919.488515:0:3317:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df19ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3317:1875503589138048:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1788619919.496173:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619919.496255:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571498112 00000100:00100000:2.0:1788619919.496277:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571498112 00000100:00100000:2.0:1788619919.496371:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 841 00000100:00100000:2.0:1788619919.496381:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:14908:x1875503571498112:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619919.496399:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619919.496417:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005bd564dc/0x1cd7fe53687350b6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c403 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619919.496449:0:6696:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000055b3449d/0x1cd7fe5368734fcf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1392640->1437695) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c331 expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619919.496458:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=1s v=(7 12) 00010000:00010000:2.0:1788619919.496463:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(1 7) 00010000:00010000:2.0:1788619919.496470:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005bd564dc/0x1cd7fe53687350b6 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1454080->1503231] (req 1454080->1503231) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c403 expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619919.496501:0:6696:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000055b3449d/0x1cd7fe5368734fcf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1392640->1437695) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c331 expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619919.496516:0:6696:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000055b3449d/0x1cd7fe5368734fcf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1392640->1437695) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c331 expref: 6 pid: 6696 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619919.496530:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df968a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589138432:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619919.496616:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc000 going to sleep for 11 seconds 00000100:00100000:2.0:1788619919.498448:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df968a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589138432:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619919.498481:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005bd564dc/0x1cd7fe53687350b6 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1454080->1503231] (req 1454080->1503231) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c403 expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619919.498519:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c7679a00, rc 0) 00000100:00100000:2.0:1788619919.498579:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:14908:x1875503571498112:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2199us (2414us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619919.498588:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 841 00000100:00100000:3.0:1788619919.500728:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619919.500837:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571498240 00000100:00100000:0.0:1788619919.500847:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571498240 00010000:00010000:0.0:1788619919.500890:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000055b3449d/0x1cd7fe5368734fcf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1392640->1437695) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c331 expref: 7 pid: 6696 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619919.500942:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 356 00000100:00100000:0.0:1788619919.500968:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571498240:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619919.500984:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619919.501018:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000055b3449d/0x1cd7fe5368734fcf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1392640->1437695) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c331 expref: 7 pid: 6696 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619919.501031:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000055b3449d/0x1cd7fe5368734fcf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1392640->1437695) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c331 expref: 7 pid: 6696 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619919.501055:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000055b3449d/0x1cd7fe5368734fcf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1392640->1437695) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c331 expref: 7 pid: 6696 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619919.501071:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(1 7) 00010000:00010000:0.0:1788619919.501075:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(1 7) 00010000:00010000:0.0:1788619919.501082:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000005bd564dc/0x1cd7fe53687350b6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1454080->1503231) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c403 expref: 5 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619919.501091:0:13330:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005bd564dc/0x1cd7fe53687350b6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1454080->1503231) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c403 expref: 5 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619919.501130:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000005bd564dc/0x1cd7fe53687350b6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1454080->1503231) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c403 expref: 5 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619919.501143:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be5880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589138560:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619919.501218:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb540 going to sleep for 11 seconds 00000100:00100000:0.0:1788619919.504616:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be5880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589138560:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619919.504659:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619919.504719:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571498240:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3752us (3998us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619919.504729:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 356 00000100:00100000:1.0:1788619919.505487:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619919.505776:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571498368 00000100:00100000:2.0:1788619919.505784:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571498368 00002000:00100000:2.0:1788619919.505831:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1454080->1458175) req@ffff8e78df8b1f80 x1875503571498368/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:15/0 lens 488/0 e 0 to 0 dl 1788619930 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619919.505842:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df8b1f80 with x1875503571498368 ext(1454080->1458175) 00002000:00010000:2.0:1788619919.505847:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df8b1f80 00000100:00100000:2.0:1788619919.505878:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 463 00000100:00100000:2.0:1788619919.505887:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571498368:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619919.506228:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571498368:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 340us (742us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619919.506234:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 463 00002000:00100000:2.0:1788619919.506241:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1454080->1458175) req@ffff8e78df8b1f80 x1875503571498368/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:15/0 lens 488/4536 e 0 to 0 dl 1788619930 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619919.506249:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df8b1f80 with x1875503571498368 ext(1454080->1458175) 00002000:00010000:2.0:1788619919.506253:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df8b1f80 00000100:00100000:3.0:1788619919.509585:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619919.510256:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571498496 00000100:00100000:0.0:1788619919.510267:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571498496 00002000:00100000:0.0:1788619919.510320:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1499136->1503231) req@ffff8e78df19e680 x1875503571498496/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:15/0 lens 488/0 e 0 to 0 dl 1788619930 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619919.510374:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df19e680 with x1875503571498496 ext(1499136->1503231) 00002000:00010000:0.0:1788619919.510380:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df19e680 00000100:00100000:0.0:1788619919.510404:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 464 00000100:00100000:0.0:1788619919.510413:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571498496:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619919.510760:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571498496:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 347us (1197us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619919.510769:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 464 00002000:00100000:0.0:1788619919.510777:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1499136->1503231) req@ffff8e78df19e680 x1875503571498496/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:15/0 lens 488/4536 e 0 to 0 dl 1788619930 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619919.510786:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df19e680 with x1875503571498496 ext(1499136->1503231) 00002000:00010000:0.0:1788619919.510789:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df19e680 00000100:00100000:1.0:1788619919.518242:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619919.522177:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571498624 00000100:00100000:0.0:1788619919.522190:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571498624 00000100:00100000:0.0:1788619919.522265:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1076 00000100:00100000:0.0:1788619919.522277:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571498624:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619919.522297:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619919.522371:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e200/0x1cd7fe53687350bd lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c40a expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619919.522410:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368735023 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c37e expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619919.522450:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be5880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589138688:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619919.522530:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619919.527838:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be5880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589138688:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619919.527866:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:0.0:1788619919.527886:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e200/0x1cd7fe53687350bd lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c40a expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619919.527908:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619919.527912:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619919.527917:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e200/0x1cd7fe53687350bd lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c40a expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619919.527933:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6e200, rc 0) 00000100:00100000:0.0:1788619919.528000:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571498624:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5724us (9766us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619919.528017:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1076 00000100:00100000:3.0:1788619919.530946:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619919.531046:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571498752 00000100:00100000:0.0:1788619919.531057:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571498752 00000100:00100000:0.0:1788619919.531119:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1077 00000100:00100000:0.0:1788619919.531131:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571498752:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619919.531150:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619919.531179:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe53687350c4 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c411 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619919.531215:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368735023 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c37e expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619919.531252:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be5880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589138816:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619919.531357:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619919.537308:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be5880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589138816:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619919.537362:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:0.0:1788619919.537382:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe53687350c4 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c411 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619919.537404:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619919.537407:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619919.537413:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ea00/0x1cd7fe53687350c4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c411 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619919.537427:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6ea00, rc 0) 00000100:00100000:0.0:1788619919.537489:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571498752:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6360us (6553us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619919.537499:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1077 00000100:00100000:3.0:1788619919.540462:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619919.540603:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571498880 00000100:00100000:1.0:1788619919.540613:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571498880 00000100:00100000:1.0:1788619919.540672:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 842 00000100:00100000:1.0:1788619919.540683:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571498880:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619919.540720:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619919.540748:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000022bdbaff/0x1cd7fe53687350cb lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c418 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619919.540808:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df8b3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589138944:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619919.540892:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a240 going to sleep for 11 seconds 00000100:00100000:1.0:1788619919.543366:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df8b3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589138944:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619919.543406:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000022bdbaff/0x1cd7fe53687350cb lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c418 expref: 7 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.543432:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000022bdbaff/0x1cd7fe53687350cb lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c418 expref: 7 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.543445:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5753a00, rc 0) 00000100:00100000:1.0:1788619919.543500:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571498880:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2818us (3047us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619919.543508:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 842 00000100:00100000:3.0:1788619919.558188:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619919.558550:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571499008 00000100:00100000:1.0:1788619919.558560:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571499008 00000100:00100000:1.0:1788619919.558628:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 843 00000100:00100000:1.0:1788619919.558638:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571499008:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619919.558655:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619919.558674:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000022bdbaff/0x1cd7fe53687350d2 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c41f expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619919.558767:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df8b3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589139072:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619919.558854:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a240 going to sleep for 11 seconds 00000100:00100000:1.0:1788619919.561237:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df8b3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589139072:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619919.561277:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000022bdbaff/0x1cd7fe53687350d2 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c41f expref: 7 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.561303:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000022bdbaff/0x1cd7fe53687350d2 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c41f expref: 7 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.561318:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5753a00, rc 0) 00000100:00100000:1.0:1788619919.561418:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571499008:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2781us (3241us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619919.561427:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 843 00000100:00100000:3.0:1788619919.564279:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619919.564381:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571499136 00000100:00100000:2.0:1788619919.564391:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571499136 00000100:00100000:2.0:1788619919.564443:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 844 00000100:00100000:2.0:1788619919.564453:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571499136:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619919.564484:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619919.564502:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003fefefac/0x1cd7fe53687350d9 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c426 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619919.564560:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df968a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589139200:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619919.564641:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccd80 going to sleep for 11 seconds 00000100:00100000:2.0:1788619919.572990:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df968a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589139200:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619919.573045:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003fefefac/0x1cd7fe53687350d9 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c426 expref: 7 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619919.573075:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003fefefac/0x1cd7fe53687350d9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c426 expref: 7 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619919.573091:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c7679c00, rc 0) 00000100:00100000:2.0:1788619919.573169:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571499136:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8717us (8897us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619919.573180:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 844 00000100:00100000:3.0:1788619919.590068:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619919.590209:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571499264 00000100:00100000:2.0:1788619919.590219:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571499264 00000100:00100000:2.0:1788619919.590281:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 845 00000100:00100000:2.0:1788619919.590291:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe138450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571499264:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619919.590309:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619919.590361:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003fefefac/0x1cd7fe53687350e0 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c42d expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619919.590442:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df968a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589139328:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619919.590523:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccd80 going to sleep for 11 seconds 00000100:00100000:2.0:1788619919.599232:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df968a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589139328:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619919.599278:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003fefefac/0x1cd7fe53687350e0 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c42d expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619919.599305:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003fefefac/0x1cd7fe53687350e0 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c42d expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619919.599319:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c7679c00, rc 0) 00000100:00100000:2.0:1788619919.599428:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe138450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571499264:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 9137us (9369us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619919.599437:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 845 00000100:00100000:3.0:1788619919.607525:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619919.607640:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571499392 00000100:00100000:2.0:1788619919.607651:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571499392 00002000:00100000:2.0:1788619919.607701:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1327104->1331199) req@ffff8e78df19c000 x1875503571499392/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:15/0 lens 488/0 e 0 to 0 dl 1788619930 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619919.607711:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df19c000 with x1875503571499392 ext(1327104->1331199) 00002000:00010000:2.0:1788619919.607717:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df19c000 00000100:00100000:2.0:1788619919.607758:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 465 00000100:00100000:2.0:1788619919.607767:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571499392:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619919.608046:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571499392:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 279us (530us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619919.608060:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 465 00002000:00100000:2.0:1788619919.608067:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1327104->1331199) req@ffff8e78df19c000 x1875503571499392/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:15/0 lens 488/4536 e 0 to 0 dl 1788619930 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619919.608075:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df19c000 with x1875503571499392 ext(1327104->1331199) 00002000:00010000:2.0:1788619919.608079:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df19c000 00000100:00100000:3.0:1788619919.612979:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619919.614375:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571499520 00000100:00100000:1.0:1788619919.614389:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571499520 00002000:00100000:1.0:1788619919.614446:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1331200->1335295) req@ffff8e78df19f100 x1875503571499520/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:15/0 lens 488/0 e 0 to 0 dl 1788619930 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619919.614456:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df19f100 with x1875503571499520 ext(1331200->1335295) 00002000:00010000:1.0:1788619919.614462:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df19f100 00000100:00100000:1.0:1788619919.614503:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 466 00000100:00100000:1.0:1788619919.614513:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571499520:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619919.614804:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571499520:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 292us (1835us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619919.614821:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 466 00002000:00100000:1.0:1788619919.614829:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1331200->1335295) req@ffff8e78df19f100 x1875503571499520/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:15/0 lens 488/4536 e 0 to 0 dl 1788619930 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619919.614838:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df19f100 with x1875503571499520 ext(1331200->1335295) 00002000:00010000:1.0:1788619919.614841:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df19f100 00000100:00100000:1.0:1788619919.618835:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619919.618931:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571499648 00000100:00100000:0.0:1788619919.618941:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571499648 00002000:00100000:0.0:1788619919.618995:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1335296->1339391) req@ffff8e78df8b3b80 x1875503571499648/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:15/0 lens 488/0 e 0 to 0 dl 1788619930 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619919.619019:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df8b3b80 with x1875503571499648 ext(1335296->1339391) 00002000:00010000:0.0:1788619919.619038:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df8b3b80 00000100:00100000:0.0:1788619919.619074:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 467 00000100:00100000:0.0:1788619919.619084:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571499648:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619919.619360:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571499648:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 243us (498us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619919.619369:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 467 00002000:00100000:0.0:1788619919.619377:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1335296->1339391) req@ffff8e78df8b3b80 x1875503571499648/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:15/0 lens 488/4536 e 0 to 0 dl 1788619930 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619919.619385:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df8b3b80 with x1875503571499648 ext(1335296->1339391) 00002000:00010000:0.0:1788619919.619389:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df8b3b80 00000100:00100000:3.0:1788619919.622155:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619919.622255:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571499776 00000100:00100000:2.0:1788619919.622265:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571499776 00002000:00100000:2.0:1788619919.622309:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1339392->1343487) req@ffff8e78df19d880 x1875503571499776/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:15/0 lens 488/0 e 0 to 0 dl 1788619930 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619919.622320:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df19d880 with x1875503571499776 ext(1339392->1343487) 00002000:00010000:2.0:1788619919.622346:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df19d880 00000100:00100000:2.0:1788619919.622380:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 468 00000100:00100000:2.0:1788619919.622389:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571499776:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619919.622619:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571499776:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 231us (470us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619919.622627:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 468 00002000:00100000:2.0:1788619919.622633:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1339392->1343487) req@ffff8e78df19d880 x1875503571499776/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:15/0 lens 488/4536 e 0 to 0 dl 1788619930 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619919.622641:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df19d880 with x1875503571499776 ext(1339392->1343487) 00002000:00010000:2.0:1788619919.622645:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df19d880 00000100:00100000:3.0:1788619919.624839:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619919.624932:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571499904 00000100:00100000:1.0:1788619919.624940:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571499904 00002000:00100000:1.0:1788619919.624983:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1343488->1347583) req@ffff8e78df19ca80 x1875503571499904/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:15/0 lens 488/0 e 0 to 0 dl 1788619930 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619919.624994:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df19ca80 with x1875503571499904 ext(1343488->1347583) 00002000:00010000:1.0:1788619919.624999:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df19ca80 00000100:00100000:1.0:1788619919.625038:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 469 00000100:00100000:1.0:1788619919.625048:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571499904:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619919.625274:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571499904:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 226us (441us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619919.625281:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 469 00002000:00100000:1.0:1788619919.625288:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1343488->1347583) req@ffff8e78df19ca80 x1875503571499904/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:15/0 lens 488/4536 e 0 to 0 dl 1788619930 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619919.625297:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df19ca80 with x1875503571499904 ext(1343488->1347583) 00002000:00010000:1.0:1788619919.625302:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df19ca80 00000100:00100000:1.0:1788619919.628461:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619919.628556:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571500032 00000100:00100000:0.0:1788619919.628572:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571500032 00002000:00100000:0.0:1788619919.628619:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1347584->1351679) req@ffff8e78df8b0e00 x1875503571500032/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:15/0 lens 488/0 e 0 to 0 dl 1788619930 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619919.628630:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df8b0e00 with x1875503571500032 ext(1347584->1351679) 00002000:00010000:0.0:1788619919.628635:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df8b0e00 00000100:00100000:0.0:1788619919.628668:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 470 00000100:00100000:0.0:1788619919.628678:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571500032:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619919.628937:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571500032:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 259us (484us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619919.628944:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 470 00002000:00100000:0.0:1788619919.628950:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1347584->1351679) req@ffff8e78df8b0e00 x1875503571500032/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:15/0 lens 488/4536 e 0 to 0 dl 1788619930 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619919.628958:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df8b0e00 with x1875503571500032 ext(1347584->1351679) 00002000:00010000:0.0:1788619919.628976:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df8b0e00 00000100:00100000:3.0:1788619919.631187:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619919.631275:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571500160 00000100:00100000:2.0:1788619919.631283:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571500160 00002000:00100000:2.0:1788619919.631322:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1351680->1355775) req@ffff8e78df19ed80 x1875503571500160/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:15/0 lens 488/0 e 0 to 0 dl 1788619930 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619919.631383:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df19ed80 with x1875503571500160 ext(1351680->1355775) 00002000:00010000:2.0:1788619919.631388:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df19ed80 00000100:00100000:2.0:1788619919.631408:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 471 00000100:00100000:2.0:1788619919.631417:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571500160:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619919.631634:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571500160:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 218us (455us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619919.631641:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 471 00002000:00100000:2.0:1788619919.631647:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1351680->1355775) req@ffff8e78df19ed80 x1875503571500160/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:15/0 lens 488/4536 e 0 to 0 dl 1788619930 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619919.631656:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df19ed80 with x1875503571500160 ext(1351680->1355775) 00002000:00010000:2.0:1788619919.631659:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df19ed80 00000100:00100000:3.0:1788619919.634052:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619919.634363:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571500288 00000100:00100000:1.0:1788619919.634379:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571500288 00002000:00100000:1.0:1788619919.634419:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1355776->1359871) req@ffff8e78df19f480 x1875503571500288/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:15/0 lens 488/0 e 0 to 0 dl 1788619930 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619919.634430:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df19f480 with x1875503571500288 ext(1355776->1359871) 00002000:00010000:1.0:1788619919.634434:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df19f480 00000100:00100000:1.0:1788619919.634464:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 472 00000100:00100000:1.0:1788619919.634471:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571500288:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619919.634705:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571500288:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 233us (659us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619919.634712:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 472 00002000:00100000:1.0:1788619919.634718:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1355776->1359871) req@ffff8e78df19f480 x1875503571500288/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:15/0 lens 488/4536 e 0 to 0 dl 1788619930 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619919.634726:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df19f480 with x1875503571500288 ext(1355776->1359871) 00002000:00010000:1.0:1788619919.634729:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df19f480 00000100:00100000:1.0:1788619919.637383:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619919.637472:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571500416 00000100:00100000:2.0:1788619919.637481:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571500416 00002000:00100000:2.0:1788619919.637530:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1359872->1363967) req@ffff8e78df8b0000 x1875503571500416/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:15/0 lens 488/0 e 0 to 0 dl 1788619930 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619919.637542:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df8b0000 with x1875503571500416 ext(1359872->1363967) 00002000:00010000:2.0:1788619919.637548:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df8b0000 00000100:00100000:2.0:1788619919.637590:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 473 00000100:00100000:2.0:1788619919.637600:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571500416:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619919.637849:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571500416:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 250us (476us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619919.637857:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 473 00002000:00100000:2.0:1788619919.637875:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1359872->1363967) req@ffff8e78df8b0000 x1875503571500416/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:15/0 lens 488/4536 e 0 to 0 dl 1788619930 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619919.637884:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df8b0000 with x1875503571500416 ext(1359872->1363967) 00002000:00010000:2.0:1788619919.637888:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df8b0000 00000100:00100000:3.0:1788619919.640996:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619919.641195:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571500544 00000100:00100000:2.0:1788619919.641205:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571500544 00002000:00100000:2.0:1788619919.641251:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1363968->1368063) req@ffff8e79010fd880 x1875503571500544/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:15/0 lens 488/0 e 0 to 0 dl 1788619930 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619919.641262:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e79010fd880 with x1875503571500544 ext(1363968->1368063) 00002000:00010000:2.0:1788619919.641268:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e79010fd880 00000100:00100000:2.0:1788619919.641303:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 474 00000100:00100000:2.0:1788619919.641313:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010fd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571500544:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619919.641621:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010fd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571500544:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 308us (634us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619919.641630:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 474 00002000:00100000:2.0:1788619919.641637:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1363968->1368063) req@ffff8e79010fd880 x1875503571500544/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:15/0 lens 488/4536 e 0 to 0 dl 1788619930 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619919.641645:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e79010fd880 with x1875503571500544 ext(1363968->1368063) 00002000:00010000:2.0:1788619919.641649:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e79010fd880 00000100:00100000:3.0:1788619919.644210:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619919.644316:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571500672 00000100:00100000:1.0:1788619919.644362:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571500672 00002000:00100000:1.0:1788619919.644409:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1368064->1372159) req@ffff8e79010fed80 x1875503571500672/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:15/0 lens 488/0 e 0 to 0 dl 1788619930 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619919.644420:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e79010fed80 with x1875503571500672 ext(1368064->1372159) 00002000:00010000:1.0:1788619919.644425:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e79010fed80 00000100:00100000:1.0:1788619919.644458:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 475 00000100:00100000:1.0:1788619919.644467:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571500672:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619919.644718:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571500672:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 251us (516us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619919.644750:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 475 00002000:00100000:1.0:1788619919.644758:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1368064->1372159) req@ffff8e79010fed80 x1875503571500672/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:15/0 lens 488/4536 e 0 to 0 dl 1788619930 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619919.644766:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e79010fed80 with x1875503571500672 ext(1368064->1372159) 00002000:00010000:1.0:1788619919.644770:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e79010fed80 00000100:00100000:1.0:1788619919.647524:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619919.647676:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571500800 00000100:00100000:0.0:1788619919.647686:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571500800 00002000:00100000:0.0:1788619919.647734:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1372160->1376255) req@ffff8e78df8b2a00 x1875503571500800/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:15/0 lens 488/0 e 0 to 0 dl 1788619930 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619919.647756:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df8b2a00 with x1875503571500800 ext(1372160->1376255) 00002000:00010000:0.0:1788619919.647762:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df8b2a00 00000100:00100000:0.0:1788619919.647794:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 476 00000100:00100000:0.0:1788619919.647803:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571500800:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619919.648046:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571500800:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 243us (532us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619919.648054:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 476 00002000:00100000:0.0:1788619919.648061:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1372160->1376255) req@ffff8e78df8b2a00 x1875503571500800/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:15/0 lens 488/4536 e 0 to 0 dl 1788619930 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619919.648070:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df8b2a00 with x1875503571500800 ext(1372160->1376255) 00002000:00010000:0.0:1788619919.648074:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df8b2a00 00000100:00100000:0.0:1788619919.650851:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619919.650948:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571500928 00000100:00100000:2.0:1788619919.650957:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571500928 00002000:00100000:2.0:1788619919.651002:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1376256->1380351) req@ffff8e77c4be5880 x1875503571500928/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:15/0 lens 488/0 e 0 to 0 dl 1788619930 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619919.651023:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4be5880 with x1875503571500928 ext(1376256->1380351) 00002000:00010000:2.0:1788619919.651043:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4be5880 00000100:00100000:2.0:1788619919.651078:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 477 00000100:00100000:2.0:1788619919.651087:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571500928:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619919.651372:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571500928:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 240us (483us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619919.651380:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 477 00002000:00100000:2.0:1788619919.651388:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1376256->1380351) req@ffff8e77c4be5880 x1875503571500928/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:15/0 lens 488/4536 e 0 to 0 dl 1788619930 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619919.651398:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4be5880 with x1875503571500928 ext(1376256->1380351) 00002000:00010000:2.0:1788619919.651401:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4be5880 00000100:00100000:3.0:1788619919.654283:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619919.654406:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571501056 00000100:00100000:1.0:1788619919.654416:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571501056 00002000:00100000:1.0:1788619919.654460:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1380352->1384447) req@ffff8e79010ff800 x1875503571501056/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:15/0 lens 488/0 e 0 to 0 dl 1788619930 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619919.654472:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e79010ff800 with x1875503571501056 ext(1380352->1384447) 00002000:00010000:1.0:1788619919.654477:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e79010ff800 00000100:00100000:1.0:1788619919.654513:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 478 00000100:00100000:1.0:1788619919.654522:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010ff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571501056:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619919.654810:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010ff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571501056:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 289us (537us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619919.654818:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 478 00002000:00100000:1.0:1788619919.654825:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1380352->1384447) req@ffff8e79010ff800 x1875503571501056/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:15/0 lens 488/4536 e 0 to 0 dl 1788619930 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619919.654834:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e79010ff800 with x1875503571501056 ext(1380352->1384447) 00002000:00010000:1.0:1788619919.654838:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e79010ff800 00000100:00100000:1.0:1788619919.657728:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619919.657841:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571501184 00000100:00100000:2.0:1788619919.657851:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571501184 00000100:00100000:2.0:1788619919.657909:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 846 00000100:00100000:2.0:1788619919.657919:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571501184:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619919.657936:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619919.657953:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003fefefac/0x1cd7fe53687350e7 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c434 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619919.658022:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df968a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589139456:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619919.658106:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc240 going to sleep for 11 seconds 00000100:00100000:2.0:1788619919.659992:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df968a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589139456:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619919.660043:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003fefefac/0x1cd7fe53687350e7 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c434 expref: 7 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619919.660091:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003fefefac/0x1cd7fe53687350e7 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c434 expref: 7 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619919.660107:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c7679c00, rc 0) 00000100:00100000:2.0:1788619919.660165:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571501184:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2246us (2446us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619919.660175:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 846 00000100:00100000:3.0:1788619919.663054:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619919.663196:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571501312 00000100:00100000:1.0:1788619919.663218:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571501312 00000100:00100000:1.0:1788619919.663276:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 847 00000100:00100000:1.0:1788619919.663286:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b16300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571501312:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619919.663302:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619919.663319:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c60d0346/0x1cd7fe53687350ee lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c43b expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619919.663397:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df8b0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589139584:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619919.663477:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a240 going to sleep for 11 seconds 00000100:00100000:2.0:1788619919.665315:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df8b0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589139584:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619919.665377:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c60d0346/0x1cd7fe53687350ee lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c43b expref: 7 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619919.665403:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c60d0346/0x1cd7fe53687350ee lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c43b expref: 7 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619919.665417:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752e00, rc 0) 00000100:00100000:2.0:1788619919.665473:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b16300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571501312:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2187us (2431us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619919.665481:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 847 00000100:00100000:0.0:1788619919.668309:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619919.668600:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571501440 00000100:00100000:1.0:1788619919.668610:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571501440 00000100:00100000:1.0:1788619919.668670:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 848 00000100:00100000:1.0:1788619919.668680:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571501440:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619919.668698:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619919.668716:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd2e9c29/0x1cd7fe53687350f5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c442 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619919.668787:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df8b1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589139712:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619919.668871:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a240 going to sleep for 11 seconds 00000100:00100000:1.0:1788619919.670923:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df8b1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589139712:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619919.670960:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bd2e9c29/0x1cd7fe53687350f5 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c442 expref: 7 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.670985:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd2e9c29/0x1cd7fe53687350f5 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c442 expref: 7 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.671028:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752600, rc 0) 00000100:00100000:1.0:1788619919.671122:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571501440:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2443us (2821us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619919.671130:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 848 00000100:00100000:1.0:1788619919.674280:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619919.674432:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571501568 00000100:00100000:2.0:1788619919.674442:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571501568 00000100:00100000:2.0:1788619919.674697:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 849 00000100:00100000:2.0:1788619919.674709:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571501568:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619919.674727:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619919.674768:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003fefefac/0x1cd7fe53687350fc lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c449 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619919.674824:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df968a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589139840:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619919.674913:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc240 going to sleep for 11 seconds 00000100:00100000:2.0:1788619919.677308:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df968a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589139840:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619919.677391:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003fefefac/0x1cd7fe53687350fc lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c449 expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619919.677415:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003fefefac/0x1cd7fe53687350fc lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c449 expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619919.677429:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c7679c00, rc 0) 00000100:00100000:2.0:1788619919.677485:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571501568:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2778us (3214us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619919.677493:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 849 00000100:00100000:3.0:1788619919.680906:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619919.680994:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571501696 00000100:00100000:2.0:1788619919.681013:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571501696 00000100:00100000:2.0:1788619919.681078:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 850 00000100:00100000:2.0:1788619919.681089:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b14a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571501696:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619919.681110:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619919.681130:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cc4d367f/0x1cd7fe5368735103 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c450 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619919.681190:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df968a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589139968:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619919.681506:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc240 going to sleep for 11 seconds 00000100:00100000:2.0:1788619919.683993:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df968a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589139968:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619919.684048:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000cc4d367f/0x1cd7fe5368735103 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c450 expref: 7 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619919.684075:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc4d367f/0x1cd7fe5368735103 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c450 expref: 7 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619919.684087:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c7678800, rc 0) 00000100:00100000:2.0:1788619919.684145:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b14a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571501696:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3057us (3248us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619919.684154:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 850 00000100:00100000:0.0:1788619919.686944:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619919.687030:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571501824 00000100:00100000:1.0:1788619919.687041:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571501824 00000100:00100000:1.0:1788619919.687097:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 851 00000100:00100000:1.0:1788619919.687107:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571501824:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619919.687124:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619919.687143:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd2e9c29/0x1cd7fe536873510a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c457 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619919.687203:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df8b0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589140096:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619919.687287:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a240 going to sleep for 11 seconds 00000100:00100000:2.0:1788619919.689192:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df8b0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589140096:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619919.689229:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bd2e9c29/0x1cd7fe536873510a lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c457 expref: 7 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619919.689266:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd2e9c29/0x1cd7fe536873510a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c457 expref: 7 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619919.689281:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752600, rc 0) 00000100:00100000:2.0:1788619919.689349:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571501824:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2244us (2414us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619919.689358:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 851 00000100:00100000:3.0:1788619919.692465:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619919.692607:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571501952 00000100:00100000:1.0:1788619919.692617:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571501952 00000100:00100000:1.0:1788619919.692706:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 852 00000100:00100000:1.0:1788619919.692716:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b16680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571501952:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619919.692745:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619919.692763:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001786a1ee/0x1cd7fe5368735111 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c45e expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619919.692824:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df8b0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589140224:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619919.692919:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a900 going to sleep for 11 seconds 00000100:00100000:1.0:1788619919.695059:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df8b0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589140224:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619919.695098:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001786a1ee/0x1cd7fe5368735111 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c45e expref: 7 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.695122:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001786a1ee/0x1cd7fe5368735111 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c45e expref: 7 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.695134:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5753200, rc 0) 00000100:00100000:1.0:1788619919.695186:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b16680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571501952:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2472us (2731us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619919.695194:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 852 00000100:00100000:3.0:1788619919.697987:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619919.698121:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571502080 00000100:00100000:2.0:1788619919.698130:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571502080 00000100:00100000:2.0:1788619919.698188:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 853 00000100:00100000:2.0:1788619919.698197:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b15f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571502080:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619919.698213:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619919.698230:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cc4d367f/0x1cd7fe5368735118 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c465 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619919.698286:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df968a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589140352:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619919.698390:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc240 going to sleep for 11 seconds 00000100:00100000:2.0:1788619919.700157:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df968a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589140352:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619919.700192:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000cc4d367f/0x1cd7fe5368735118 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c465 expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619919.700217:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc4d367f/0x1cd7fe5368735118 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c465 expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619919.700231:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c7678800, rc 0) 00000100:00100000:2.0:1788619919.700287:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b15f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571502080:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2091us (2309us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619919.700297:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 853 00000100:00100000:0.0:1788619919.703295:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619919.703410:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571502208 00000100:00100000:3.0:1788619919.703419:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571502208 00000100:00100000:3.0:1788619919.703488:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 685 00000100:00100000:3.0:1788619919.703499:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571502208:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:3.0:1788619919.703809:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 994373 -> 994373 00000004:00010000:3.0:1788619919.703814:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619919 -> 1788619919 00000004:00010000:3.0:1788619919.703816:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619919 -> 1788619916 00000004:00010000:3.0:1788619919.703819:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619919 -> 1788619919 00000004:00010000:3.0:1788619919.703835:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 646 -> 646 00000100:00100000:3.0:1788619919.703910:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2353:x1875503571502208:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 412us (623us total) trans 4294967698 rc 0/0 00000100:00100000:3.0:1788619919.703919:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 685 00000100:00100000:3.0:1788619919.706773:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619919.706870:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571502336 00000100:00100000:2.0:1788619919.706879:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571502336 00000100:00100000:2.0:1788619919.706935:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 854 00000100:00100000:2.0:1788619919.706946:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b17100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571502336:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619919.706962:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619919.706979:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cc4d367f/0x1cd7fe536873511f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c46c expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619919.707046:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df968a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589140480:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619919.707125:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc240 going to sleep for 11 seconds 00000100:00100000:2.0:1788619919.708955:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df968a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589140480:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619919.708991:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000cc4d367f/0x1cd7fe536873511f lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c46c expref: 7 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619919.709027:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc4d367f/0x1cd7fe536873511f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c46c expref: 7 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619919.709042:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c7678800, rc 0) 00000100:00100000:2.0:1788619919.709134:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b17100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571502336:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2188us (2370us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619919.709142:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 854 00000100:00100000:0.0:1788619919.712056:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619919.712148:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571502464 00000100:00100000:1.0:1788619919.712157:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571502464 00000100:00100000:1.0:1788619919.712213:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 855 00000100:00100000:1.0:1788619919.712223:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571502464:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619919.712239:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619919.712255:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c03a8843/0x1cd7fe5368735126 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c473 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619919.712310:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df8b0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589140608:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619919.712421:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a900 going to sleep for 11 seconds 00000100:00100000:1.0:1788619919.714131:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df8b0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589140608:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619919.714165:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c03a8843/0x1cd7fe5368735126 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c473 expref: 7 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.714206:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c03a8843/0x1cd7fe5368735126 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c473 expref: 7 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.714218:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752a00, rc 0) 00000100:00100000:1.0:1788619919.714261:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571502464:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2040us (2215us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619919.714269:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 855 00000100:00100000:3.0:1788619919.716714:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619919.716798:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571502592 00000100:00100000:1.0:1788619919.716807:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571502592 00000100:00100000:1.0:1788619919.716863:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 856 00000100:00100000:1.0:1788619919.716872:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b14000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571502592:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619919.716889:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619919.716907:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000022bdbaff/0x1cd7fe536873512d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c47a expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619919.716962:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df8b0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589140736:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619919.717047:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a900 going to sleep for 11 seconds 00000100:00100000:1.0:1788619919.718698:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df8b0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589140736:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619919.718744:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000022bdbaff/0x1cd7fe536873512d lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c47a expref: 7 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.718771:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000022bdbaff/0x1cd7fe536873512d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c47a expref: 7 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.718802:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5753a00, rc 0) 00000100:00100000:1.0:1788619919.718857:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b14000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571502592:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 1985us (2150us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619919.718865:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 856 00000100:00100000:3.0:1788619919.725936:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619919.726046:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571502720 00000100:00100000:2.0:1788619919.726057:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571502720 00000100:00100000:2.0:1788619919.726118:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 857 00000100:00100000:2.0:1788619919.726129:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b16a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571502720:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619919.726145:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619919.726161:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cc4d367f/0x1cd7fe5368735134 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c481 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619919.726219:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df968a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589140864:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619919.726306:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc240 going to sleep for 11 seconds 00000100:00100000:2.0:1788619919.728415:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df968a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589140864:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619919.728451:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000cc4d367f/0x1cd7fe5368735134 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c481 expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619919.728475:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc4d367f/0x1cd7fe5368735134 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c481 expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619919.728488:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c7678800, rc 0) 00000100:00100000:2.0:1788619919.728541:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b16a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571502720:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2414us (2615us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619919.728549:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 857 00000100:00100000:0.0:1788619919.731728:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619919.731813:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571502848 00000100:00100000:3.0:1788619919.731822:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571502848 00000100:00100000:3.0:1788619919.731878:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1078 00000100:00100000:3.0:1788619919.731888:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571502848:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619919.731907:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619919.731928:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe536873513b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c488 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619919.731964:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368735023 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c37e expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619919.732003:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b15180 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589140992:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619919.732098:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78ce7f3540 going to sleep for 11 seconds 00000004:00010000:0.0:1788619919.734106:0:11397:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619916 -> 1788619919 00000100:00100000:0.0:1788619919.734132:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b15180 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589140992:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619919.734151:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:0.0:1788619919.734179:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe536873513b lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c488 expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619919.734198:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619919.734201:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619919.734205:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe536873513b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c488 expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619919.734218:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c200, rc 0) 00000100:00100000:0.0:1788619919.734267:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571502848:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2381us (2548us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619919.734275:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1078 00000100:00100000:1.0:1788619919.736954:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619919.737074:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571502976 00000100:00100000:0.0:1788619919.737084:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571502976 00000100:00100000:0.0:1788619919.737148:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1079 00000100:00100000:0.0:1788619919.737158:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571502976:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619919.737178:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619919.737199:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fc00/0x1cd7fe5368735142 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c48f expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619919.737232:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368735023 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c37e expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619919.737269:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be5f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589141120:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619919.737369:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebb40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619919.739435:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be5f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589141120:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619919.739459:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:0.0:1788619919.739490:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fc00/0x1cd7fe5368735142 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c48f expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619919.739511:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619919.739514:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619919.739519:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fc00/0x1cd7fe5368735142 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c48f expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619919.739535:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6fc00, rc 0) 00000100:00100000:0.0:1788619919.739699:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571502976:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2541us (2752us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619919.739708:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1079 00000100:00100000:3.0:1788619919.743094:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619919.743186:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571503104 00000100:00100000:0.0:1788619919.743196:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571503104 00000100:00100000:0.0:1788619919.743297:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1080 00000100:00100000:0.0:1788619919.743308:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b15180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571503104:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619919.743344:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619919.743367:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368735149 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c496 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619919.743404:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368735023 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c37e expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619919.743443:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be5f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589141248:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619919.743520:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebb40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619919.745488:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be5f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589141248:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619919.745515:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:0.0:1788619919.745548:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368735149 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c496 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619919.745579:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619919.745582:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619919.745589:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368735149 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c496 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619919.745603:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6e000, rc 0) 00000100:00100000:0.0:1788619919.745660:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b15180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571503104:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2355us (2764us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619919.745669:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1080 00000100:00100000:3.0:1788619919.748430:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619919.748517:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571503232 00000100:00100000:1.0:1788619919.748526:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571503232 00000100:00100000:1.0:1788619919.748586:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1081 00000100:00100000:1.0:1788619919.748598:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7432d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571503232:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619919.748618:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619919.748639:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752000/0x1cd7fe5368735150 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c49d expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.748677:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368735023 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c37e expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619919.748716:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df8b0380 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589141376:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619919.748809:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a900 going to sleep for 11 seconds 00000100:00100000:1.0:1788619919.750820:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df8b0380 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589141376:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619919.750845:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:1.0:1788619919.750864:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752000/0x1cd7fe5368735150 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c49d expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.750883:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619919.750886:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619919.750906:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5752000/0x1cd7fe5368735150 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c49d expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.750920:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752000, rc 0) 00000100:00100000:1.0:1788619919.750972:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7432d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571503232:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2376us (2550us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619919.750981:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1081 00000100:00100000:3.0:1788619919.753774:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619919.753941:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571503360 00000100:00100000:0.0:1788619919.753951:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571503360 00000100:00100000:0.0:1788619919.754015:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1082 00000100:00100000:0.0:1788619919.754025:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7432a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571503360:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619919.754044:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619919.754063:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fc00/0x1cd7fe5368735157 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c4a4 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619919.754092:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368735023 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c37e expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619919.754101:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fc00/0x1cd7fe5368735157 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4a4 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619919.754135:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368735023 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c37e expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619919.754167:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368735023 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c37e expref: 13 pid: 5846 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619919.754180:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be5f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589141504:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619919.754259:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebb40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619919.756700:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be5f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589141504:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619919.756744:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fc00/0x1cd7fe5368735157 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4a4 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619919.756762:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619919.756765:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619919.756769:0:5845:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fc00/0x1cd7fe5368735157 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4a4 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619919.756779:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6fc00, rc 0) 00000100:00100000:0.0:1788619919.756836:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7432a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:14908:x1875503571503360:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2812us (3071us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619919.756844:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1082 00000100:00100000:3.0:1788619919.758886:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619919.758970:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571503488 00000100:00100000:1.0:1788619919.758979:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571503488 00010000:00010000:1.0:1788619919.759030:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368735023 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c37e expref: 14 pid: 5846 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619919.759067:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 357 00000100:00100000:1.0:1788619919.759078:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7433480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571503488:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619919.759091:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619919.759120:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368735023 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c37e expref: 14 pid: 5846 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.759135:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368735023 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c37e expref: 14 pid: 5846 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.759158:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368735023 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c37e expref: 14 pid: 5846 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.759174:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619919.759180:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fc00/0x1cd7fe5368735157 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4a4 expref: 9 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.759211:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fc00/0x1cd7fe5368735157 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4a4 expref: 9 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.759221:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fc00/0x1cd7fe5368735157 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4a4 expref: 9 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.759258:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fc00/0x1cd7fe5368735157 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4a4 expref: 9 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619919.759271:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df8b0380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589141632:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619919.759436:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a900 going to sleep for 11 seconds 00000100:00100000:1.0:1788619919.765376:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df8b0380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589141632:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619919.765404:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619919.765464:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7433480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571503488:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 6387us (6586us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619919.765475:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 357 00000100:00100000:0.0:1788619919.767709:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619919.767816:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571503616 00000100:00100000:2.0:1788619919.767826:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571503616 00000100:00100000:2.0:1788619919.767898:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 686 00000100:00100000:2.0:1788619919.767919:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571503616:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:2.0:1788619919.768166:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 994373 -> 994373 00000004:00010000:2.0:1788619919.768170:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619919 -> 1788619919 00000004:00010000:2.0:1788619919.768173:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619919 -> 1788619919 00000004:00010000:2.0:1788619919.768175:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619919 -> 1788619919 00000004:00010000:2.0:1788619919.768177:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 646 -> 646 00000100:00100000:2.0:1788619919.768248:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571503616:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 332us (548us total) trans 4294967699 rc 0/0 00000100:00100000:2.0:1788619919.768255:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 686 00000100:00100000:2.0:1788619919.775704:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619919.775811:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571503744 00000100:00100000:0.0:1788619919.775821:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571503744 00000100:00100000:0.0:1788619919.775884:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1083 00000100:00100000:0.0:1788619919.775895:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df968a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571503744:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619919.775929:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619919.775965:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e400/0x1cd7fe536873515e lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c4ab expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619919.775996:0:9417:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fc00/0x1cd7fe5368735157 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4a4 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619919.776018:0:9417:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e400/0x1cd7fe536873515e lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4ab expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619919.776052:0:9417:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fc00/0x1cd7fe5368735157 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4a4 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619919.776066:0:9417:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fc00/0x1cd7fe5368735157 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4a4 expref: 10 pid: 5845 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619919.776080:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be7480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589141760:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619919.776167:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebb40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619919.778637:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be7480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589141760:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619919.778674:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e400/0x1cd7fe536873515e lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4ab expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619919.778693:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619919.778696:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619919.778701:0:9417:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e400/0x1cd7fe536873515e lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4ab expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619919.778712:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6e400, rc 0) 00000100:00100000:0.0:1788619919.778806:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df968a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571503744:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2911us (3109us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619919.778815:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1083 00000100:00100000:3.0:1788619919.781158:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619919.781289:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571503872 00000100:00100000:1.0:1788619919.781298:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571503872 00010000:00010000:1.0:1788619919.781350:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fc00/0x1cd7fe5368735157 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4a4 expref: 11 pid: 5845 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619919.781390:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 358 00000100:00100000:1.0:1788619919.781398:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7430700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571503872:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619919.781412:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619919.781438:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fc00/0x1cd7fe5368735157 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4a4 expref: 11 pid: 5845 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.781468:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fc00/0x1cd7fe5368735157 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4a4 expref: 11 pid: 5845 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.781492:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6fc00/0x1cd7fe5368735157 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4a4 expref: 11 pid: 5845 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.781510:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619919.781515:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e400/0x1cd7fe536873515e lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4ab expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.781525:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e400/0x1cd7fe536873515e lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4ab expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.781534:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e400/0x1cd7fe536873515e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4ab expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.781585:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e400/0x1cd7fe536873515e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4ab expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619919.781597:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df8b0380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589141888:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619919.781671:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a900 going to sleep for 11 seconds 00000100:00100000:2.0:1788619919.788143:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619919.788277:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571504000 00000100:00100000:0.0:1788619919.788286:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571504000 00000100:00100000:0.0:1788619919.788378:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 687 00000100:00100000:0.0:1788619919.788388:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df96aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571504000:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:0.0:1788619919.788631:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 994373 -> 994373 00000004:00010000:0.0:1788619919.788636:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619919 -> 1788619919 00000004:00010000:0.0:1788619919.788638:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619919 -> 1788619919 00000004:00010000:0.0:1788619919.788640:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619919 -> 1788619919 00000004:00010000:0.0:1788619919.788643:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 646 -> 646 00000100:00100000:0.0:1788619919.788713:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df96aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571504000:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 326us (580us total) trans 4294967700 rc 0/0 00000100:00100000:0.0:1788619919.788722:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 687 00000100:00100000:1.0:1788619919.788829:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df8b0380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589141888:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619919.788850:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619919.788871:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7430700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571503872:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 7474us (7722us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619919.788899:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 358 00000100:00100000:3.0:1788619919.791938:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619919.792200:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571504128 00000100:00100000:2.0:1788619919.792210:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571504128 00000100:00100000:2.0:1788619919.792269:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 858 00000100:00100000:2.0:1788619919.792279:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7430a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571504128:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619919.792297:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619919.792314:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003fefefac/0x1cd7fe5368735165 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c4b2 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619919.792414:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df96a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589142016:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619919.792535:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc240 going to sleep for 11 seconds 00000100:00100000:2.0:1788619919.794933:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df96a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589142016:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619919.794970:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003fefefac/0x1cd7fe5368735165 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4b2 expref: 7 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619919.794996:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003fefefac/0x1cd7fe5368735165 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4b2 expref: 7 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619919.795019:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c7679c00, rc 0) 00000100:00100000:2.0:1788619919.795072:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7430a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571504128:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2794us (3143us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619919.795081:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 858 00000100:00100000:3.0:1788619919.797959:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619919.798124:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571504256 00000100:00100000:1.0:1788619919.798134:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571504256 00000100:00100000:1.0:1788619919.798192:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 859 00000100:00100000:1.0:1788619919.798202:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7430000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571504256:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619919.798222:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619919.798239:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d40dc9c9/0x1cd7fe536873516c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c4b9 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619919.798295:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df8b0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589142144:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619919.798405:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a900 going to sleep for 11 seconds 00000100:00100000:1.0:1788619919.801576:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df8b0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589142144:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619919.801608:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d40dc9c9/0x1cd7fe536873516c lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4b9 expref: 7 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.801629:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d40dc9c9/0x1cd7fe536873516c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4b9 expref: 7 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.801640:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5752c00, rc 0) 00000100:00100000:1.0:1788619919.801687:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7430000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571504256:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3486us (3738us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619919.801696:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 859 00000100:00100000:0.0:1788619919.808159:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619919.808259:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571504384 00000100:00100000:3.0:1788619919.808269:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571504384 00000100:00100000:3.0:1788619919.808438:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1084 00000100:00100000:3.0:1788619919.808448:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571504384:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619919.808482:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619919.808504:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368735173 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c4c0 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619919.808542:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e400/0x1cd7fe536873515e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4ab expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619919.808592:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c7433100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589142272:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619919.808677:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78ce7f3000 going to sleep for 11 seconds 00000100:00100000:0.0:1788619919.819127:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c7433100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589142272:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619919.819156:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:0.0:1788619919.819175:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368735173 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4c0 expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619919.819195:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619919.819198:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619919.819204:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368735173 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4c0 expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619919.819237:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c600, rc 0) 00000100:00100000:0.0:1788619919.819560:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571504384:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 11112us (11407us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619919.819569:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1084 00000100:00100000:1.0:1788619919.826130:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619919.826239:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571504512 00000100:00100000:0.0:1788619919.826250:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571504512 00000100:00100000:0.0:1788619919.826372:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1085 00000100:00100000:0.0:1788619919.826382:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:14908:x1875503571504512:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619919.826403:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619919.826424:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ec00/0x1cd7fe536873517a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c4c7 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619919.826456:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e400/0x1cd7fe536873515e lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4ab expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619919.826477:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ec00/0x1cd7fe536873517a lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4c7 expref: 11 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619919.826508:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e400/0x1cd7fe536873515e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4ab expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619919.826520:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e400/0x1cd7fe536873515e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4ab expref: 14 pid: 9417 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619919.826532:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be5180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589142400:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619919.826627:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebb40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619919.829885:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be5180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589142400:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619919.829923:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ec00/0x1cd7fe536873517a lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4c7 expref: 11 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619919.829943:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619919.829946:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619919.829950:0:5845:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ec00/0x1cd7fe536873517a lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4c7 expref: 11 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619919.829962:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6ec00, rc 0) 00000100:00100000:0.0:1788619919.830035:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571504512:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3654us (3914us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619919.830044:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1085 00000100:00100000:3.0:1788619919.831446:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619919.831544:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571504640 00000100:00100000:2.0:1788619919.831552:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571504640 00010000:00010000:2.0:1788619919.831600:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e400/0x1cd7fe536873515e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4ab expref: 15 pid: 9417 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619919.831650:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 359 00000100:00100000:2.0:1788619919.831659:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7431f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571504640:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619919.831673:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619919.831696:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e400/0x1cd7fe536873515e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4ab expref: 15 pid: 9417 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619919.831709:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e400/0x1cd7fe536873515e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4ab expref: 15 pid: 9417 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619919.831730:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e400/0x1cd7fe536873515e lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4ab expref: 15 pid: 9417 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619919.831758:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619919.831764:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ec00/0x1cd7fe536873517a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4c7 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619919.831775:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ec00/0x1cd7fe536873517a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4c7 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619919.831784:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ec00/0x1cd7fe536873517a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4c7 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619919.831817:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ec00/0x1cd7fe536873517a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4c7 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619919.831836:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df96a680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589142528:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619919.831915:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc240 going to sleep for 11 seconds 00000100:00100000:2.0:1788619919.835801:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df96a680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589142528:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619919.835827:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619919.835882:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7431f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571504640:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 4223us (4442us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619919.835894:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 359 00000100:00100000:0.0:1788619919.842360:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619919.842390:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619919.842447:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571504896 00000100:00100000:2.0:1788619919.842456:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571504896 00002000:00100000:2.0:1788619919.842501:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1871871) req@ffff8e77c4be4700 x1875503571504896/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:15/0 lens 520/0 e 0 to 0 dl 1788619930 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619919.842509:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4be4700 with x1875503571504896 ext(1048576->1871871) 00002000:00010000:2.0:1788619919.842513:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4be4700 00000100:00100000:2.0:1788619919.842531:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 479 00000100:00100000:2.0:1788619919.842537:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571504896:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619919.843357:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571504768 00000100:00100000:3.0:1788619919.843364:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571504768 00002000:00100000:3.0:1788619919.843394:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be5180 x1875503571504768/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:15/0 lens 488/0 e 0 to 0 dl 1788619930 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619919.843403:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be5180 with x1875503571504768 00000100:00100000:3.0:1788619919.843485:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 688 00000100:00100000:3.0:1788619919.843495:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571504768:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619919.867893:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571504896:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 25340us (25486us total) trans 0 rc 716800/716800 00000100:00100000:2.0:1788619919.867909:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 479 00002000:00100000:2.0:1788619919.867924:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1871871) req@ffff8e77c4be4700 x1875503571504896/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:15/0 lens 520/440 e 0 to 0 dl 1788619930 ref 1 fl Complete:/600/0 rc 716800/716800 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619919.867937:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4be4700 with x1875503571504896 ext(1048576->1871871) 00002000:00010000:2.0:1788619919.867943:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4be4700 00000100:00100000:2.0:1788619919.869779:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571504768:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 26278us (27450us total) trans 0 rc 995328/995328 00000100:00100000:2.0:1788619919.869792:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 688 00002000:00100000:2.0:1788619919.869804:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be5180 x1875503571504768/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:15/0 lens 488/440 e 0 to 0 dl 1788619930 ref 1 fl Complete:/600/0 rc 995328/995328 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619919.869814:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be5180 with x1875503571504768 00000100:00100000:0.0:1788619919.874313:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619919.874521:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571505024 00000100:00100000:2.0:1788619919.874533:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571505024 00002000:00100000:2.0:1788619919.874588:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1454080->1556479) req@ffff8e77c4be5500 x1875503571505024/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:15/0 lens 504/0 e 0 to 0 dl 1788619930 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619919.874600:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4be5500 with x1875503571505024 ext(1454080->1556479) 00002000:00010000:2.0:1788619919.874605:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4be5500 00000100:00100000:2.0:1788619919.874687:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 480 00000100:00100000:2.0:1788619919.874698:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571505024:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:1.0:1788619919.883410:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+8:2354:x1875503571505024:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 8703us (9095us total) trans 4294967439 rc 0/0 00000100:00100000:1.0:1788619919.883434:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 480 00002000:00100000:1.0:1788619919.883448:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1454080->1556479) req@ffff8e77c4be5500 x1875503571505024/t4294967439(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:15/0 lens 504/448 e 0 to 0 dl 1788619930 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619919.883458:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4be5500 with x1875503571505024 ext(1454080->1556479) 00002000:00010000:1.0:1788619919.883598:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4be5500 00000100:00100000:2.0:1788619919.886207:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619919.886264:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571505152 00000100:00100000:1.0:1788619919.886284:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571505152 00000100:00100000:1.0:1788619919.886368:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 860 00000100:00100000:1.0:1788619919.886379:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df96a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+8:2353:x1875503571505152:12345-192.168.201.5@tcp:16:fsx.0 00000001:00080000:0.0:1788619919.916935:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967439 is committed 00000001:00080000:0.0:1788619919.916955:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967438 is committed 00000100:00100000:0.0:1788619919.919873:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df96a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571505152:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 33485us (33660us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619919.919887:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 860 00000100:00100000:2.0:1788619919.932672:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619919.932719:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619919.932798:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571505408 00000100:00100000:1.0:1788619919.932809:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571505408 00000100:00100000:1.0:1788619919.932868:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 861 00000100:00100000:1.0:1788619919.932878:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df968000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571505408:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619919.932895:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619919.932913:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0b04224/0x1cd7fe5368735181 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c4d5 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619919.932974:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4ba0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589142656:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619919.933062:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a240 going to sleep for 11 seconds 00000100:00100000:0.0:1788619919.933284:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571505280 00000100:00100000:0.0:1788619919.933290:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571505280 00000100:00100000:0.0:1788619919.933380:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1086 00000100:00100000:0.0:1788619919.933388:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df969f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571505280:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619919.933415:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619919.933428:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368735188 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c4ce expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619919.933456:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368735188 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4ce expref: 15 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788619919.933480:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:0.0:1788619919.933488:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368735188 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4ce expref: 15 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619919.933501:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619919.933504:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619919.933507:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6e000, rc 0) 00000100:00100000:0.0:1788619919.933546:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df969f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571505280:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 159us (881us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619919.933553:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1086 00002000:00010000:1.0:1788619919.938613:0:6696:0:(ofd_lvb.c:284:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks: 263 -> 519 00000100:00100000:1.0:1788619919.938645:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4ba0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589142656:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619919.938812:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e0b04224/0x1cd7fe5368735181 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4d5 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.938859:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0b04224/0x1cd7fe5368735181 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4d5 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.938872:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa06800, rc 0) 00000100:00100000:1.0:1788619919.938934:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df968000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571505408:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6057us (6213us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619919.938942:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 861 00000100:00100000:3.0:1788619919.942241:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619919.942370:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571505536 00000100:00100000:2.0:1788619919.942381:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571505536 00000100:00100000:2.0:1788619919.942451:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 862 00000100:00100000:2.0:1788619919.942462:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7430380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571505536:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619919.942479:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619919.942495:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000003a33281/0x1cd7fe536873518f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c4dc expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619919.942552:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df969880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589142784:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619919.947785:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc240 going to sleep for 11 seconds 00000100:00100000:2.0:1788619919.947856:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df969880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589142784:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619919.947888:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000003a33281/0x1cd7fe536873518f lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4dc expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619919.947911:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000003a33281/0x1cd7fe536873518f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4dc expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619919.947924:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c7679600, rc 0) 00000100:00100000:2.0:1788619919.947987:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7430380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571505536:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5526us (5756us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619919.947995:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 862 00000100:00100000:3.0:1788619919.954830:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619919.954964:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571505664 00000100:00100000:0.0:1788619919.954976:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571505664 00000100:00100000:0.0:1788619919.955036:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1087 00000100:00100000:0.0:1788619919.955046:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7431500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571505664:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619919.955067:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619919.955070:0:5846:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:0.0:1788619919.955112:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6ec00/0x1cd7fe536873517a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4c7 expref: 11 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619919.955129:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619919.955147:0:5846:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788619919.955160:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368735196 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c4e3 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619919.955179:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368735188 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4ce expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619919.955185:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368735196 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4e3 expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619919.955208:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368735188 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4ce expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619919.955217:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368735188 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4ce expref: 14 pid: 9417 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619919.955226:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be7800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589142912:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619919.955291:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebb40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619919.966162:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be7800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589142912:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619919.966203:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368735196 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4e3 expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619919.966223:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619919.966226:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619919.966230:0:5846:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368735196 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4e3 expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619919.966261:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6b6f000, rc 0) 00000100:00100000:0.0:1788619919.966384:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7431500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571505664:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 11338us (11564us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619919.966393:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1087 00000100:00100000:0.0:1788619919.966783:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619919.967188:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571505792 00000100:00100000:2.0:1788619919.967196:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571505792 00010000:00010000:2.0:1788619919.967248:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368735188 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4ce expref: 15 pid: 9417 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619919.967285:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 360 00000100:00100000:2.0:1788619919.967889:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571505792:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619919.967906:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619919.967934:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368735188 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4ce expref: 15 pid: 9417 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619919.967947:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368735188 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4ce expref: 15 pid: 9417 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619919.967973:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6e000/0x1cd7fe5368735188 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4ce expref: 15 pid: 9417 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619919.967989:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619919.967995:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368735196 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4e3 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619919.968074:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368735196 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4e3 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619919.968083:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368735196 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4e3 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619919.968121:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368735196 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4e3 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619919.968134:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df969880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589143040:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619919.968248:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc240 going to sleep for 11 seconds 00000100:00100000:2.0:1788619919.974561:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df969880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589143040:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619919.974587:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619919.974654:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571505792:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 7360us (7871us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619919.974665:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 360 00000100:00100000:3.0:1788619919.975583:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619919.975991:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571505920 00000100:00100000:2.0:1788619919.975999:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571505920 00002000:00100000:2.0:1788619919.976048:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c7433800 x1875503571505920/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:15/0 lens 488/0 e 0 to 0 dl 1788619930 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619919.976059:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c7433800 with x1875503571505920 00000100:00100000:2.0:1788619919.976093:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 689 00000100:00100000:2.0:1788619919.976103:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7433800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571505920:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619919.976390:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7433800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571505920:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 288us (809us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619919.976397:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 689 00002000:00100000:2.0:1788619919.976404:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c7433800 x1875503571505920/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:15/0 lens 488/4536 e 0 to 0 dl 1788619930 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619919.976614:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c7433800 with x1875503571505920 00000100:00100000:3.0:1788619919.983506:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619919.983942:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571506048 00000100:00100000:2.0:1788619919.983952:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571506048 00002000:00100000:2.0:1788619919.984026:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c7431880 x1875503571506048/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:15/0 lens 488/0 e 0 to 0 dl 1788619930 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619919.984037:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c7431880 with x1875503571506048 00000100:00100000:2.0:1788619919.984075:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 690 00000100:00100000:2.0:1788619919.984084:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7431880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571506048:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619919.984375:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7431880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571506048:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 291us (877us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619919.984383:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 690 00002000:00100000:2.0:1788619919.984390:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c7431880 x1875503571506048/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:15/0 lens 488/4536 e 0 to 0 dl 1788619930 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619919.984398:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c7431880 with x1875503571506048 00000100:00100000:0.0:1788619919.991474:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619919.991629:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571506176 00000100:00100000:1.0:1788619919.991640:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571506176 00000100:00100000:1.0:1788619919.991829:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1088 00000100:00100000:1.0:1788619919.991839:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571506176:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619919.991860:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619919.991880:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06200/0x1cd7fe536873519d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c4ea expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.991910:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368735196 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4e3 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.991921:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06200/0x1cd7fe536873519d lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4ea expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.992401:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368735196 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4e3 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619919.992417:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368735196 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4e3 expref: 10 pid: 5846 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619919.992445:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4ba0700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589143168:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619919.992699:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a240 going to sleep for 11 seconds 00000100:00100000:3.0:1788619919.996116:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619919.996203:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4ba0700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589143168:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619919.996236:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06200/0x1cd7fe536873519d lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4ea expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619919.996254:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619919.996257:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619919.996262:0:5845:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06200/0x1cd7fe536873519d lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4ea expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619919.996273:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa06200, rc 0) 00000100:00100000:2.0:1788619919.996364:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571506304 00000100:00100000:2.0:1788619919.996373:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571506304 00002000:00100000:2.0:1788619919.996421:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c7433b80 x1875503571506304/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:15/0 lens 488/0 e 0 to 0 dl 1788619930 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619919.996431:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c7433b80 with x1875503571506304 00010000:00010000:2.0:1788619919.996438:0:5857:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368735196 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4e3 expref: 11 pid: 5846 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619919.996451:0:5857:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368735196 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4e3 expref: 11 pid: 5846 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788619919.996458:0:5857:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e78c7433b80 00000100:00100000:2.0:1788619919.996499:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 25 00000100:00100000:2.0:1788619919.996508:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7433b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571506304:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:0.0:1788619919.997516:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:14908:x1875503571506176:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5673us (6046us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619919.997525:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1088 00000004:00010000:2.0:1788619920.006575:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 994373 -> 994373 00000004:00010000:2.0:1788619920.006583:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619919 -> 1788619919 00000004:00010000:2.0:1788619920.006586:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619919 -> 1788619919 00000004:00010000:2.0:1788619920.006588:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619919 -> 1788619919 00000004:00010000:2.0:1788619920.006591:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 646 -> 646 00000100:00100000:2.0:1788619920.006657:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7433b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571506304:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 10145us (10545us total) trans 4294967701 rc 0/0 00000100:00100000:2.0:1788619920.006667:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 25 00002000:00100000:2.0:1788619920.006795:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c7433b80 x1875503571506304/t4294967701(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:15/0 lens 488/448 e 0 to 0 dl 1788619930 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619920.006804:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c7433b80 with x1875503571506304 00010000:00010000:2.0:1788619920.006815:0:5857:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368735196 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4e3 expref: 13 pid: 5846 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619920.006831:0:5857:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368735196 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4e3 expref: 13 pid: 5846 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788619920.006839:0:5857:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e78c7433b80 00000100:00100000:3.0:1788619920.010320:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619920.010481:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571506432 00000100:00100000:1.0:1788619920.010492:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571506432 00010000:00010000:1.0:1788619920.010565:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368735196 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4e3 expref: 13 pid: 5846 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619920.010609:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 361 00000100:00100000:1.0:1788619920.010620:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7432300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+13:2355:x1875503571506432:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619920.010635:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619920.010660:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368735196 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4e3 expref: 13 pid: 5846 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.010885:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368735196 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4e3 expref: 13 pid: 5846 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.010910:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6b6f000/0x1cd7fe5368735196 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4e3 expref: 13 pid: 5846 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.010928:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619920.010934:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06200/0x1cd7fe536873519d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4ea expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.010946:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06200/0x1cd7fe536873519d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4ea expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.010969:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06200/0x1cd7fe536873519d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4ea expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.011016:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06200/0x1cd7fe536873519d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4ea expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619920.011029:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4ba0380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589143296:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619920.011216:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a6c0 going to sleep for 11 seconds 00000100:00080000:2.0:1788619920.011429:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1788619920.019894:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619920.019976:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571506560 00000100:00100000:3.0:1788619920.019986:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571506560 00002000:00100000:3.0:1788619920.020062:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be6d80 x1875503571506560/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/0 e 0 to 0 dl 1788619931 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619920.020076:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be6d80 with x1875503571506560 00000100:00100000:3.0:1788619920.020114:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 691 00000100:00100000:3.0:1788619920.020125:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571506560:12345-192.168.201.5@tcp:3:fsx.0 00000100:00080000:2.0:1788619920.024301:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1788619920.029881:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4ba0380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589143296:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619920.029909:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619920.032462:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571506560:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 12328us (12567us total) trans 0 rc 995328/995328 00000100:00100000:1.0:1788619920.032476:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 691 00002000:00100000:1.0:1788619920.032489:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be6d80 x1875503571506560/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/440 e 0 to 0 dl 1788619931 ref 1 fl Complete:/600/0 rc 995328/995328 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619920.032500:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be6d80 with x1875503571506560 00000100:00100000:0.0:1788619920.034755:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7432300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571506432:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 24125us (24432us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619920.034769:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 361 00000100:00100000:2.0:1788619920.041435:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619920.041532:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571506688 00000100:00100000:1.0:1788619920.041542:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571506688 00000100:00100000:1.0:1788619920.041621:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 863 00000100:00100000:1.0:1788619920.041632:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df969880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571506688:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619920.041654:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619920.041674:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001b18afa8/0x1cd7fe53687351a4 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c4f1 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619920.041764:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4ba0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589143424:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619920.041844:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a6c0 going to sleep for 11 seconds 00000100:00080000:2.0:1788619920.042008:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1788619920.045697:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4ba0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589143424:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619920.045748:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001b18afa8/0x1cd7fe53687351a4 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4f1 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.045774:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001b18afa8/0x1cd7fe53687351a4 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4f1 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.045790:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa07200, rc 0) 00000100:00100000:1.0:1788619920.045855:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df969880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571506688:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4225us (4429us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619920.045864:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 863 00000100:00100000:0.0:1788619920.048744:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619920.048823:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571506816 00000100:00100000:1.0:1788619920.048834:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571506816 00000100:00100000:1.0:1788619920.048893:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 864 00000100:00100000:1.0:1788619920.048902:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e02a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571506816:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619920.048919:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619920.048937:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004ec41216/0x1cd7fe53687351ab lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c4f8 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619920.048993:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4ba0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589143552:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619920.049089:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a6c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619920.051804:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4ba0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589143552:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619920.051852:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004ec41216/0x1cd7fe53687351ab lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4f8 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619920.051877:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004ec41216/0x1cd7fe53687351ab lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4f8 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619920.051891:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa07400, rc 0) 00000100:00100000:0.0:1788619920.051951:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e02a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571506816:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3049us (3225us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619920.051960:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 864 00000100:00100000:3.0:1788619920.055412:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619920.055507:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571506944 00000100:00100000:1.0:1788619920.055519:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571506944 00000100:00100000:1.0:1788619920.055584:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 865 00000100:00100000:1.0:1788619920.055595:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7431c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571506944:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619920.055612:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619920.055630:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001b18afa8/0x1cd7fe53687351b2 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c4ff expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619920.055687:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4ba2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589143680:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619920.055778:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a900 going to sleep for 11 seconds 00000100:00100000:1.0:1788619920.057757:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4ba2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589143680:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619920.057797:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001b18afa8/0x1cd7fe53687351b2 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4ff expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.057820:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001b18afa8/0x1cd7fe53687351b2 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4ff expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.057833:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa07200, rc 0) 00000100:00100000:1.0:1788619920.057887:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7431c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571506944:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2294us (2485us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619920.057895:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 865 00000100:00100000:3.0:1788619920.062415:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619920.062699:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571507072 00000100:00100000:0.0:1788619920.062710:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571507072 00000100:00100000:0.0:1788619920.062780:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1089 00000100:00100000:0.0:1788619920.062789:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7432680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:14908:x1875503571507072:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619920.062811:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619920.062814:0:9417:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:0.0:1788619920.062857:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06200/0x1cd7fe536873519d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c4ea expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619920.062876:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619920.062881:0:9417:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788619920.062895:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe53687351b9 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c506 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619920.062920:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe53687351b9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c506 expref: 15 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619920.062957:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe53687351b9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c506 expref: 15 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619920.062968:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619920.062971:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619920.062974:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297e00, rc 0) 00000100:00100000:0.0:1788619920.063042:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7432680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:14908:x1875503571507072:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 254us (637us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619920.063049:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1089 00000100:00100000:3.0:1788619920.078927:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619920.079046:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571507200 00000100:00100000:2.0:1788619920.079056:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571507200 00002000:00100000:2.0:1788619920.079106:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c7433100 x1875503571507200/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/0 e 0 to 0 dl 1788619931 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619920.079117:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c7433100 with x1875503571507200 00000100:00100000:2.0:1788619920.079156:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 692 00000100:00100000:2.0:1788619920.079165:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7433100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571507200:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619920.079517:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7433100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571507200:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 351us (598us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619920.079526:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 692 00002000:00100000:2.0:1788619920.079533:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c7433100 x1875503571507200/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/4536 e 0 to 0 dl 1788619931 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619920.079541:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c7433100 with x1875503571507200 00000100:00100000:3.0:1788619920.085117:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619920.085454:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571507328 00000100:00100000:1.0:1788619920.085465:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571507328 00002000:00100000:1.0:1788619920.085516:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fe000000 x1875503571507328/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/0 e 0 to 0 dl 1788619931 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619920.085527:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fe000000 with x1875503571507328 00000100:00100000:1.0:1788619920.085592:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 693 00000100:00100000:1.0:1788619920.085601:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe000000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571507328:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619920.086111:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe000000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571507328:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 509us (1003us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619920.086119:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 693 00002000:00100000:1.0:1788619920.086126:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fe000000 x1875503571507328/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/4536 e 0 to 0 dl 1788619931 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619920.086134:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fe000000 with x1875503571507328 00000100:00100000:3.0:1788619920.090212:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619920.090420:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571507456 00000100:00100000:0.0:1788619920.090431:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571507456 00002000:00100000:0.0:1788619920.090485:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fe001c00 x1875503571507456/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/0 e 0 to 0 dl 1788619931 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619920.090498:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fe001c00 with x1875503571507456 00000100:00100000:0.0:1788619920.090538:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 694 00000100:00100000:0.0:1788619920.090548:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe001c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571507456:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619920.090862:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe001c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571507456:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 314us (659us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619920.090870:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 694 00002000:00100000:0.0:1788619920.090877:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fe001c00 x1875503571507456/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/4536 e 0 to 0 dl 1788619931 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619920.090886:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fe001c00 with x1875503571507456 00000100:00100000:3.0:1788619920.094824:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619920.095055:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571507584 00000100:00100000:1.0:1788619920.095065:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571507584 00002000:00100000:1.0:1788619920.095114:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fe003800 x1875503571507584/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/0 e 0 to 0 dl 1788619931 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619920.095125:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fe003800 with x1875503571507584 00000100:00100000:1.0:1788619920.095165:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 695 00000100:00100000:1.0:1788619920.095174:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe003800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2352:x1875503571507584:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619920.095493:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe003800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2352:x1875503571507584:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 318us (679us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619920.095500:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 695 00002000:00100000:1.0:1788619920.095508:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fe003800 x1875503571507584/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/4536 e 0 to 0 dl 1788619931 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619920.095515:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fe003800 with x1875503571507584 00000100:00100000:2.0:1788619920.100077:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619920.100785:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571507712 00000100:00100000:3.0:1788619920.100798:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571507712 00002000:00100000:3.0:1788619920.100850:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df969c00 x1875503571507712/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/0 e 0 to 0 dl 1788619931 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619920.100861:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df969c00 with x1875503571507712 00000100:00100000:3.0:1788619920.100948:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 696 00000100:00100000:3.0:1788619920.100958:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df969c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2353:x1875503571507712:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619920.101243:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df969c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2353:x1875503571507712:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 286us (1177us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619920.101251:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 696 00002000:00100000:3.0:1788619920.101259:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df969c00 x1875503571507712/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/4536 e 0 to 0 dl 1788619931 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619920.101288:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df969c00 with x1875503571507712 00000100:00100000:1.0:1788619920.105838:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619920.105926:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571507840 00000100:00100000:0.0:1788619920.105936:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571507840 00002000:00100000:0.0:1788619920.105994:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4ba2a00 x1875503571507840/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/0 e 0 to 0 dl 1788619931 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619920.106059:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4ba2a00 with x1875503571507840 00000100:00100000:0.0:1788619920.106120:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 697 00000100:00100000:0.0:1788619920.106129:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571507840:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619920.106607:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571507840:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 477us (776us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619920.106623:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 697 00002000:00100000:0.0:1788619920.106631:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4ba2a00 x1875503571507840/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/4536 e 0 to 0 dl 1788619931 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619920.106639:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4ba2a00 with x1875503571507840 00000100:00100000:2.0:1788619920.110584:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619920.110784:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571507968 00000100:00100000:3.0:1788619920.110794:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571507968 00002000:00100000:3.0:1788619920.110842:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df969180 x1875503571507968/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/0 e 0 to 0 dl 1788619931 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619920.110853:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df969180 with x1875503571507968 00000100:00100000:3.0:1788619920.110893:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 698 00000100:00100000:3.0:1788619920.110902:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df969180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2352:x1875503571507968:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619920.111317:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df969180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2352:x1875503571507968:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 414us (740us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619920.111344:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 698 00002000:00100000:3.0:1788619920.111352:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df969180 x1875503571507968/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/4536 e 0 to 0 dl 1788619931 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619920.111360:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df969180 with x1875503571507968 00000100:00100000:0.0:1788619920.114935:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619920.115233:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571508096 00000100:00100000:2.0:1788619920.115243:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571508096 00002000:00100000:2.0:1788619920.115290:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900e03100 x1875503571508096/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/0 e 0 to 0 dl 1788619931 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619920.115301:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900e03100 with x1875503571508096 00000100:00100000:2.0:1788619920.115368:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 699 00000100:00100000:2.0:1788619920.115378:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e03100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571508096:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619920.115958:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e03100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571508096:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 579us (1030us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619920.115966:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 699 00002000:00100000:2.0:1788619920.115973:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900e03100 x1875503571508096/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/4536 e 0 to 0 dl 1788619931 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619920.115982:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900e03100 with x1875503571508096 00000100:00100000:0.0:1788619920.120268:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619920.120351:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571508224 00000100:00100000:3.0:1788619920.120361:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571508224 00002000:00100000:3.0:1788619920.120486:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900e01880 x1875503571508224/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/0 e 0 to 0 dl 1788619931 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619920.120496:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900e01880 with x1875503571508224 00000100:00100000:3.0:1788619920.120534:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 700 00000100:00100000:3.0:1788619920.120543:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e01880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2353:x1875503571508224:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619920.120985:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e01880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2353:x1875503571508224:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 441us (724us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619920.121003:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 700 00002000:00100000:3.0:1788619920.121026:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900e01880 x1875503571508224/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/4536 e 0 to 0 dl 1788619931 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619920.121035:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900e01880 with x1875503571508224 00000100:00100000:3.0:1788619920.124941:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619920.125102:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571508352 00000100:00100000:2.0:1788619920.125111:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571508352 00002000:00100000:2.0:1788619920.125157:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fe001180 x1875503571508352/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/0 e 0 to 0 dl 1788619931 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619920.125168:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fe001180 with x1875503571508352 00000100:00100000:2.0:1788619920.125244:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 701 00000100:00100000:2.0:1788619920.125254:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe001180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2352:x1875503571508352:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619920.125544:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe001180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2352:x1875503571508352:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 291us (613us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619920.125564:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 701 00002000:00100000:2.0:1788619920.125571:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fe001180 x1875503571508352/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/4536 e 0 to 0 dl 1788619931 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619920.125578:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fe001180 with x1875503571508352 00000100:00100000:0.0:1788619920.130117:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619920.130248:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571508480 00000100:00100000:2.0:1788619920.130258:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571508480 00002000:00100000:2.0:1788619920.130306:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900e03800 x1875503571508480/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/0 e 0 to 0 dl 1788619931 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619920.130317:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900e03800 with x1875503571508480 00000100:00100000:2.0:1788619920.130365:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 702 00000100:00100000:2.0:1788619920.130374:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e03800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2352:x1875503571508480:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619920.130968:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e03800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2352:x1875503571508480:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 594us (860us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619920.130976:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 702 00002000:00100000:2.0:1788619920.130983:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900e03800 x1875503571508480/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/4536 e 0 to 0 dl 1788619931 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619920.130991:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900e03800 with x1875503571508480 00000100:00100000:0.0:1788619920.134975:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619920.135142:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571508608 00000100:00100000:2.0:1788619920.135153:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571508608 00002000:00100000:2.0:1788619920.135206:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900e01c00 x1875503571508608/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/0 e 0 to 0 dl 1788619931 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619920.135217:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900e01c00 with x1875503571508608 00000100:00100000:2.0:1788619920.135302:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 703 00000100:00100000:2.0:1788619920.135312:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e01c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2352:x1875503571508608:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619920.135886:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e01c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2352:x1875503571508608:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 574us (918us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619920.135895:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 703 00002000:00100000:2.0:1788619920.135902:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900e01c00 x1875503571508608/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/4536 e 0 to 0 dl 1788619931 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619920.135917:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900e01c00 with x1875503571508608 00000100:00100000:0.0:1788619920.139542:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619920.139749:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571508736 00000100:00100000:3.0:1788619920.139760:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571508736 00002000:00100000:3.0:1788619920.139811:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900e03480 x1875503571508736/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/0 e 0 to 0 dl 1788619931 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619920.139822:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900e03480 with x1875503571508736 00000100:00100000:3.0:1788619920.139859:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 704 00000100:00100000:3.0:1788619920.139870:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e03480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2352:x1875503571508736:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:3.0:1788619920.144693:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 994373 -> 994373 00000004:00010000:3.0:1788619920.144704:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619919 -> 1788619920 00000004:00010000:3.0:1788619920.144707:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619919 -> 1788619919 00000004:00010000:3.0:1788619920.144709:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619919 -> 1788619920 00000004:00010000:3.0:1788619920.144712:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 646 -> 646 00000100:00100000:3.0:1788619920.144851:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e03480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2352:x1875503571508736:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 4977us (5313us total) trans 4294967702 rc 0/0 00000100:00100000:3.0:1788619920.144863:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 704 00002000:00100000:3.0:1788619920.144874:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900e03480 x1875503571508736/t4294967702(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/448 e 0 to 0 dl 1788619931 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619920.144883:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900e03480 with x1875503571508736 00000100:00100000:1.0:1788619920.150612:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619920.150760:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571508864 00000100:00100000:0.0:1788619920.150770:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571508864 00000100:00100000:0.0:1788619920.150887:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1090 00000100:00100000:0.0:1788619920.150899:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2354:x1875503571508864:12345-192.168.201.5@tcp:16:fsx.0 00010000:00010000:1.0:1788619920.166011:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1788619920.172936:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967700 is committed 00000001:00080000:1.0:1788619920.172965:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967701 is committed 00000001:00080000:1.0:1788619920.172975:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967702 is committed 00000100:00100000:0.0:1788619920.173164:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571508864:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 22263us (22557us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619920.173175:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1090 00000100:00100000:0.0:1788619920.175713:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619920.175785:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571508992 00000100:00100000:2.0:1788619920.175795:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571508992 00000100:00100000:2.0:1788619920.175845:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 866 00000100:00100000:2.0:1788619920.175855:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571508992:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619920.175873:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619920.175899:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1a53256/0x1cd7fe53687351c0 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c50d expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619920.175945:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df968380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589143808:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619920.176020:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc300 going to sleep for 11 seconds 00000100:00100000:2.0:1788619920.177804:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df968380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589143808:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619920.177833:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e1a53256/0x1cd7fe53687351c0 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c50d expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619920.177851:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1a53256/0x1cd7fe53687351c0 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c50d expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619920.177861:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c7679400, rc 0) 00000100:00100000:2.0:1788619920.177952:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571508992:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2099us (2245us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619920.177960:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 866 00000100:00100000:1.0:1788619920.181431:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619920.181469:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571509120 00000100:00100000:2.0:1788619920.181479:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571509120 00000100:00100000:2.0:1788619920.181602:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 867 00000100:00100000:2.0:1788619920.181613:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571509120:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619920.181631:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619920.181649:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003fefefac/0x1cd7fe53687351c7 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c514 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619920.181709:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df968380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589143936:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619920.181835:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc300 going to sleep for 11 seconds 00000100:00100000:2.0:1788619920.184359:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df968380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589143936:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619920.184413:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003fefefac/0x1cd7fe53687351c7 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c514 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619920.184435:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003fefefac/0x1cd7fe53687351c7 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c514 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619920.184445:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c7679c00, rc 0) 00000100:00100000:2.0:1788619920.184500:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571509120:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2889us (3075us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619920.184508:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 867 00000100:00100000:1.0:1788619920.187289:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619920.188214:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571509248 00000100:00100000:0.0:1788619920.188235:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571509248 00000100:00100000:0.0:1788619920.188286:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 868 00000100:00100000:0.0:1788619920.188295:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571509248:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619920.188312:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619920.188350:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6252eaa/0x1cd7fe53687351ce lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c51b expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619920.188424:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900e00e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589144064:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619920.188515:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeba80 going to sleep for 11 seconds 00000100:00100000:0.0:1788619920.193857:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900e00e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589144064:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619920.193896:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a6252eaa/0x1cd7fe53687351ce lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c51b expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619920.193927:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6252eaa/0x1cd7fe53687351ce lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c51b expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619920.193937:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297600, rc 0) 00000100:00100000:0.0:1788619920.193985:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571509248:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5691us (6705us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619920.193993:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 868 00000100:00100000:1.0:1788619920.196841:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619920.196930:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571509376 00000100:00100000:3.0:1788619920.196939:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571509376 00000100:00100000:3.0:1788619920.197006:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 869 00000100:00100000:3.0:1788619920.197033:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571509376:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619920.197051:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619920.197068:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad29727f/0x1cd7fe53687351d5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c522 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619920.197122:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fe000380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589144192:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619920.197201:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c900 going to sleep for 11 seconds 00000100:00100000:3.0:1788619920.200625:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe000380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589144192:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619920.200666:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ad29727f/0x1cd7fe53687351d5 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c522 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619920.200692:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad29727f/0x1cd7fe53687351d5 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c522 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619920.200705:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c000, rc 0) 00000100:00100000:3.0:1788619920.200777:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571509376:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3746us (3947us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619920.200786:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 869 00000100:00100000:1.0:1788619920.205150:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619920.205251:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571509504 00000100:00100000:2.0:1788619920.205262:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571509504 00000100:00100000:2.0:1788619920.205322:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 870 00000100:00100000:2.0:1788619920.205470:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571509504:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619920.205489:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619920.205507:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003fefefac/0x1cd7fe53687351dc lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c529 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619920.205568:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df968380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589144320:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619920.205654:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc300 going to sleep for 11 seconds 00000100:00100000:2.0:1788619920.208151:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df968380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589144320:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619920.208193:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003fefefac/0x1cd7fe53687351dc lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c529 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619920.208216:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003fefefac/0x1cd7fe53687351dc lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c529 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619920.208228:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c7679c00, rc 0) 00000100:00100000:2.0:1788619920.208288:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571509504:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2822us (3148us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619920.208297:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 870 00000100:00100000:1.0:1788619920.211597:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619920.211661:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571509632 00000100:00100000:2.0:1788619920.211671:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571509632 00000100:00100000:2.0:1788619920.211746:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 871 00000100:00100000:2.0:1788619920.211756:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:14908:x1875503571509632:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619920.211778:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619920.211796:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1a53256/0x1cd7fe53687351e3 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c530 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619920.211828:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000005bd564dc/0x1cd7fe53687350b6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1454080->1503231) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c403 expref: 5 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619920.211842:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(1 7) 00010000:00010000:2.0:1788619920.211846:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 7) 00010000:00010000:2.0:1788619920.211852:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1a53256/0x1cd7fe53687351e3 lrc: 4/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1249280->1253375] (req 1249280->1253375) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c530 expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619920.211885:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000005bd564dc/0x1cd7fe53687350b6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1454080->1503231) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c403 expref: 5 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619920.211899:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000005bd564dc/0x1cd7fe53687350b6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1454080->1503231) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c403 expref: 5 pid: 6696 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619920.211913:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df968380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589144448:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619920.211998:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc300 going to sleep for 11 seconds 00000100:00100000:1.0:1788619920.219947:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619920.220039:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571509760 00000100:00100000:3.0:1788619920.220049:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571509760 00010000:00010000:3.0:1788619920.220108:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000005bd564dc/0x1cd7fe53687350b6 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1454080->1503231) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c403 expref: 6 pid: 6696 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619920.220157:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 362 00000100:00100000:3.0:1788619920.220167:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c855b450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571509760:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619920.220196:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619920.220218:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000005bd564dc/0x1cd7fe53687350b6 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1454080->1503231) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c403 expref: 6 pid: 6696 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619920.220233:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000005bd564dc/0x1cd7fe53687350b6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1454080->1503231) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c403 expref: 6 pid: 6696 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619920.220258:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 7) 00010000:00010000:3.0:1788619920.220264:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 7) 00010000:00010000:3.0:1788619920.220272:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000e1a53256/0x1cd7fe53687351e3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1249280->1253375) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c530 expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619920.220282:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1a53256/0x1cd7fe53687351e3 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1249280->1253375) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c530 expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619920.220319:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000e1a53256/0x1cd7fe53687351e3 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1249280->1253375) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c530 expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619920.220365:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fe000380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589144576:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619920.220451:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c900 going to sleep for 11 seconds 00000100:00080000:1.0:1788619920.220502:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:2.0:1788619920.221247:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005bd564dc/0x1cd7fe53687350b6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1454080->1503231) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c403 expref: 6 pid: 6696 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619920.221265:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df968380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589144448:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619920.221290:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e1a53256/0x1cd7fe53687351e3 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1249280->1253375) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c530 expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619920.221302:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c7679400, rc 0) 00000100:00100000:2.0:1788619920.221376:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:14908:x1875503571509632:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 9622us (9789us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619920.221388:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 871 00000100:00100000:3.0:1788619920.236189:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe000380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589144576:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619920.236230:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619920.236288:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c855b450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571509760:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 16122us (16350us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619920.236309:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 362 00000100:00100000:1.0:1788619920.244439:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619920.244488:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619920.244599:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571509888 00000100:00100000:2.0:1788619920.244610:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571509888 00002000:00100000:2.0:1788619920.244661:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1871871) req@ffff8e77c4ba0a80 x1875503571509888/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/0 e 0 to 0 dl 1788619931 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619920.244671:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4ba0a80 with x1875503571509888 ext(1048576->1871871) 00002000:00010000:2.0:1788619920.244677:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4ba0a80 00000100:00100000:2.0:1788619920.244710:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 481 00000100:00100000:2.0:1788619920.244719:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571509888:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619920.245789:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571510016 00000100:00100000:3.0:1788619920.245799:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571510016 00002000:00100000:3.0:1788619920.245831:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4ba3800 x1875503571510016/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/0 e 0 to 0 dl 1788619931 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619920.245841:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4ba3800 with x1875503571510016 00000100:00100000:3.0:1788619920.245874:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 705 00000100:00100000:3.0:1788619920.245883:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571510016:12345-192.168.201.5@tcp:3:fsx.0 00000004:00010000:3.0:1788619920.259215:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 646 -> 1030 00000100:00100000:3.0:1788619920.259385:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571510016:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 13497us (14891us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619920.259398:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 705 00002000:00100000:3.0:1788619920.259420:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4ba3800 x1875503571510016/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/440 e 0 to 0 dl 1788619931 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619920.259429:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4ba3800 with x1875503571510016 00000100:00100000:2.0:1788619920.278499:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571509888:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 33770us (34059us total) trans 0 rc 823296/823296 00000100:00100000:2.0:1788619920.278512:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 481 00002000:00100000:2.0:1788619920.278525:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1871871) req@ffff8e77c4ba0a80 x1875503571509888/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/440 e 0 to 0 dl 1788619931 ref 1 fl Complete:/600/0 rc 823296/823296 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619920.278535:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4ba0a80 with x1875503571509888 ext(1048576->1871871) 00002000:00010000:2.0:1788619920.278541:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4ba0a80 00000100:00100000:3.0:1788619920.283810:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619920.283842:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619920.283922:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571510272 00000100:00100000:1.0:1788619920.283931:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571510272 00000100:00100000:1.0:1788619920.283978:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 872 00000100:00100000:1.0:1788619920.283988:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe000380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571510272:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619920.284017:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619920.284034:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001e71691f/0x1cd7fe53687351ea lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c53e expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.284058:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 7) 00010000:00010000:1.0:1788619920.284061:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 7) 00010000:00010000:1.0:1788619920.284069:0:7654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001e71691f/0x1cd7fe53687351ea lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c53e expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.284082:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001e71691f/0x1cd7fe53687351ea lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c53e expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.284093:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa06a00, rc 0) 00000100:00100000:1.0:1788619920.284157:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe000380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571510272:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 171us (354us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619920.284164:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 872 00000100:00100000:0.0:1788619920.286526:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571510144 00000100:00100000:0.0:1788619920.286539:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571510144 00000100:00100000:0.0:1788619920.286606:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1091 00000100:00100000:0.0:1788619920.286616:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe003b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571510144:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619920.286638:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619920.286671:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe53687351f1 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c537 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619920.286711:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe53687351b9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c506 expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619920.286778:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900e00e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589144704:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619920.286873:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb840 going to sleep for 11 seconds 00000004:00010000:0.0:1788619920.290546:0:5845:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619919 -> 1788619920 00000100:00100000:0.0:1788619920.290579:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900e00e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589144704:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619920.290604:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:0.0:1788619920.290624:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe53687351f1 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c537 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619920.290645:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619920.290648:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619920.290653:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe53687351f1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c537 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619920.290669:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed0400, rc 0) 00000100:00100000:0.0:1788619920.290751:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe003b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571510144:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4126us (6896us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619920.290761:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1091 00000100:00100000:1.0:1788619920.297696:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619920.297786:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571510400 00000100:00100000:0.0:1788619920.297796:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571510400 00000100:00100000:0.0:1788619920.297860:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1092 00000100:00100000:0.0:1788619920.297870:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571510400:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619920.297916:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619920.297937:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe53687351f8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c545 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619920.297974:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe53687351b9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c506 expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619920.298016:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900e00e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589144832:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619920.298097:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb840 going to sleep for 11 seconds 00000100:00100000:0.0:1788619920.304317:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900e00e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589144832:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619920.304387:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:0.0:1788619920.304419:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe53687351f8 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c545 expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619920.304442:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619920.304445:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619920.304451:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe53687351f8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c545 expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619920.304467:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed0400, rc 0) 00000100:00100000:0.0:1788619920.304542:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571510400:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6673us (6855us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619920.304550:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1092 00000100:00100000:3.0:1788619920.318886:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619920.319626:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571510528 00000100:00100000:1.0:1788619920.319649:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571510528 00000100:00100000:1.0:1788619920.319734:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 873 00000100:00100000:1.0:1788619920.319744:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe001500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:14908:x1875503571510528:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619920.319761:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619920.319778:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000a9a342/0x1cd7fe53687351ff lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c54c expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.319809:0:6696:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000e1a53256/0x1cd7fe53687351e3 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1249280->1253375) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c530 expref: 7 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.319820:0:6696:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000001e71691f/0x1cd7fe53687351ea lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c53e expref: 5 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.319829:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 7) 00010000:00010000:1.0:1788619920.319833:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 7) 00010000:00010000:1.0:1788619920.319839:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000a9a342/0x1cd7fe53687351ff lrc: 5/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [1544192->1548287] (req 1544192->1548287) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c54c expref: 7 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.319874:0:6696:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000001e71691f/0x1cd7fe53687351ea lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c53e expref: 5 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.319978:0:6696:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000001e71691f/0x1cd7fe53687351ea lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c53e expref: 5 pid: 7654 timeout: 512 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619920.320011:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4ba0000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589144960:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619920.320109:0:6696:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000e1a53256/0x1cd7fe53687351e3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1249280->1253375) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c530 expref: 7 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.320119:0:6696:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000e1a53256/0x1cd7fe53687351e3 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1249280->1253375) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c530 expref: 7 pid: 6697 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619920.320128:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4ba1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589145088:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619920.320150:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7480 going to sleep for 11 seconds 00000100:00100000:3.0:1788619920.332309:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4ba0000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589144960:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619920.332366:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4ba1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589145088:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619920.332398:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000000a9a342/0x1cd7fe53687351ff lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [1544192->1548287] (req 1544192->1548287) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c54c expref: 7 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619920.332428:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa07a00, rc 0) 00000100:00100000:3.0:1788619920.332504:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe001500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:14908:x1875503571510528:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 12760us (13627us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619920.332529:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 873 00000100:00100000:3.0:1788619920.341445:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619920.341490:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619920.341586:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571510656 00000100:00100000:0.0:1788619920.341595:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571510656 00010000:00010000:0.0:1788619920.341633:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000001e71691f/0x1cd7fe53687351ea lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c53e expref: 6 pid: 7654 timeout: 512 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619920.341710:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571510784 00000100:00100000:0.0:1788619920.341714:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571510784 00010000:00010000:0.0:1788619920.341741:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e1a53256/0x1cd7fe53687351e3 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1249280->1253375) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c530 expref: 7 pid: 6697 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619920.341771:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 363 00000100:00100000:0.0:1788619920.341781:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe000700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571510656:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619920.341798:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619920.341822:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 1s ns: filter-lustre-OST0000_UUID lock: 000000001e71691f/0x1cd7fe53687351ea lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c53e expref: 6 pid: 7654 timeout: 512 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619920.341837:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000001e71691f/0x1cd7fe53687351ea lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c53e expref: 6 pid: 7654 timeout: 512 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619920.341862:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001e71691f/0x1cd7fe53687351ea lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c53e expref: 6 pid: 7654 timeout: 512 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619920.341881:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619920.341928:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe000700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571510656:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 150us (495us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619920.341956:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 364 00000100:00100000:1.0:1788619920.341969:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe000a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571510784:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619920.341978:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619920.341989:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 1s ns: filter-lustre-OST0000_UUID lock: 00000000e1a53256/0x1cd7fe53687351e3 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1249280->1253375) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c530 expref: 7 pid: 6697 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.342001:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e1a53256/0x1cd7fe53687351e3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1249280->1253375) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c530 expref: 7 pid: 6697 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.342015:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1a53256/0x1cd7fe53687351e3 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1249280->1253375) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c530 expref: 7 pid: 6697 timeout: 512 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.342027:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=1s v=(2 7) 00010000:00010000:1.0:1788619920.342031:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 7) 00010000:00010000:1.0:1788619920.342046:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000000a9a342/0x1cd7fe53687351ff lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1544192->1548287) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c54c expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.342056:0:13330:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000a9a342/0x1cd7fe53687351ff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1544192->1548287) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c54c expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.342095:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000000a9a342/0x1cd7fe53687351ff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1544192->1548287) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c54c expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619920.342110:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4ba0e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589145216:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619920.342165:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7f00 going to sleep for 11 seconds 00000100:00100000:0.0:1788619920.342882:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 363 00000100:00100000:0.0:1788619920.344810:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4ba0e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589145216:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619920.344838:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619920.344892:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe000a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571510784:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 2925us (3400us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619920.344916:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 364 00000100:00100000:1.0:1788619920.359563:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619920.359695:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571510912 00000100:00100000:2.0:1788619920.359706:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571510912 00002000:00100000:2.0:1788619920.359766:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1544192->1548287) req@ffff8e77c4ba1180 x1875503571510912/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/0 e 0 to 0 dl 1788619931 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619920.359778:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4ba1180 with x1875503571510912 ext(1544192->1548287) 00002000:00010000:2.0:1788619920.359783:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4ba1180 00000100:00100000:2.0:1788619920.359815:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 482 00000100:00100000:2.0:1788619920.359824:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571510912:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619920.360045:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571510912:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 220us (501us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619920.360053:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 482 00002000:00100000:2.0:1788619920.360060:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1544192->1548287) req@ffff8e77c4ba1180 x1875503571510912/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/4536 e 0 to 0 dl 1788619931 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619920.360069:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4ba1180 with x1875503571510912 ext(1544192->1548287) 00002000:00010000:2.0:1788619920.360072:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4ba1180 00000100:00100000:3.0:1788619920.365311:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619920.365496:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571511040 00000100:00100000:2.0:1788619920.365507:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571511040 00002000:00100000:2.0:1788619920.365568:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1548288->1552383) req@ffff8e78fe001880 x1875503571511040/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/0 e 0 to 0 dl 1788619931 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619920.365579:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fe001880 with x1875503571511040 ext(1548288->1552383) 00002000:00010000:2.0:1788619920.365584:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fe001880 00000100:00100000:2.0:1788619920.365621:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 483 00000100:00100000:2.0:1788619920.365632:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe001880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571511040:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619920.365837:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe001880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571511040:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 206us (535us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619920.365844:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 483 00002000:00100000:2.0:1788619920.365851:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1548288->1552383) req@ffff8e78fe001880 x1875503571511040/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/4536 e 0 to 0 dl 1788619931 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619920.365860:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fe001880 with x1875503571511040 ext(1548288->1552383) 00002000:00010000:2.0:1788619920.365864:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fe001880 00000100:00100000:3.0:1788619920.376100:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619920.376218:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571511168 00000100:00100000:1.0:1788619920.376229:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571511168 00002000:00100000:1.0:1788619920.376280:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1552384->1556479) req@ffff8e78fe000e00 x1875503571511168/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/0 e 0 to 0 dl 1788619931 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619920.376292:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fe000e00 with x1875503571511168 ext(1552384->1556479) 00002000:00010000:1.0:1788619920.376298:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fe000e00 00000100:00100000:1.0:1788619920.376375:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 484 00000100:00100000:1.0:1788619920.376386:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe000e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571511168:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619920.376602:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe000e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571511168:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 218us (513us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619920.376609:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 484 00002000:00100000:1.0:1788619920.376616:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1552384->1556479) req@ffff8e78fe000e00 x1875503571511168/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/4536 e 0 to 0 dl 1788619931 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619920.376645:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fe000e00 with x1875503571511168 ext(1552384->1556479) 00002000:00010000:1.0:1788619920.376648:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fe000e00 00000100:00100000:1.0:1788619920.395190:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619920.395299:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571511296 00000100:00100000:0.0:1788619920.395310:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571511296 00002000:00100000:0.0:1788619920.395396:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1556480->1560575) req@ffff8e77c4ba0e00 x1875503571511296/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/0 e 0 to 0 dl 1788619931 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619920.395409:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4ba0e00 with x1875503571511296 ext(1556480->1560575) 00002000:00010000:0.0:1788619920.395415:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4ba0e00 00000100:00100000:0.0:1788619920.395436:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 485 00000100:00100000:0.0:1788619920.395446:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571511296:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619920.395655:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571511296:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 210us (474us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619920.395664:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 485 00002000:00100000:0.0:1788619920.395672:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1556480->1560575) req@ffff8e77c4ba0e00 x1875503571511296/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/4536 e 0 to 0 dl 1788619931 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619920.395680:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4ba0e00 with x1875503571511296 ext(1556480->1560575) 00002000:00010000:0.0:1788619920.395684:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4ba0e00 00000100:00100000:3.0:1788619920.405603:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619920.405826:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571511424 00000100:00100000:2.0:1788619920.405836:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571511424 00002000:00100000:2.0:1788619920.405884:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1560576->1564671) req@ffff8e78fe003480 x1875503571511424/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/0 e 0 to 0 dl 1788619931 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619920.405896:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fe003480 with x1875503571511424 ext(1560576->1564671) 00002000:00010000:2.0:1788619920.405901:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fe003480 00000100:00100000:2.0:1788619920.405936:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 486 00000100:00100000:2.0:1788619920.405946:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe003480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571511424:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619920.406234:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe003480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571511424:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 289us (641us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619920.406242:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 486 00002000:00100000:2.0:1788619920.406249:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1560576->1564671) req@ffff8e78fe003480 x1875503571511424/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/4536 e 0 to 0 dl 1788619931 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619920.406257:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fe003480 with x1875503571511424 ext(1560576->1564671) 00002000:00010000:2.0:1788619920.406260:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fe003480 00000100:00100000:1.0:1788619920.414479:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619920.415065:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571511552 00000100:00100000:2.0:1788619920.415077:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571511552 00002000:00100000:2.0:1788619920.415129:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1564672->1568767) req@ffff8e77c4ba1880 x1875503571511552/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/0 e 0 to 0 dl 1788619931 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619920.415140:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4ba1880 with x1875503571511552 ext(1564672->1568767) 00002000:00010000:2.0:1788619920.415145:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4ba1880 00000100:00100000:2.0:1788619920.415183:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 487 00000100:00100000:2.0:1788619920.415193:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571511552:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619920.415452:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571511552:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 258us (980us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619920.415463:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 487 00002000:00100000:2.0:1788619920.415470:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1564672->1568767) req@ffff8e77c4ba1880 x1875503571511552/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/4536 e 0 to 0 dl 1788619931 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619920.415479:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4ba1880 with x1875503571511552 ext(1564672->1568767) 00002000:00010000:2.0:1788619920.415482:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4ba1880 00000100:00100000:1.0:1788619920.426655:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619920.426808:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571511680 00000100:00100000:0.0:1788619920.426819:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571511680 00002000:00100000:0.0:1788619920.426869:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1568768->1572863) req@ffff8e77c4ba0000 x1875503571511680/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/0 e 0 to 0 dl 1788619931 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619920.426880:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4ba0000 with x1875503571511680 ext(1568768->1572863) 00002000:00010000:0.0:1788619920.426886:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4ba0000 00000100:00100000:0.0:1788619920.426920:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 488 00000100:00100000:0.0:1788619920.426931:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571511680:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619920.427168:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571511680:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 238us (523us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619920.427177:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 488 00002000:00100000:0.0:1788619920.427184:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1568768->1572863) req@ffff8e77c4ba0000 x1875503571511680/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/4536 e 0 to 0 dl 1788619931 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619920.427192:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4ba0000 with x1875503571511680 ext(1568768->1572863) 00002000:00010000:0.0:1788619920.427195:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4ba0000 00000100:00100000:3.0:1788619920.431399:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619920.432945:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571511808 00000100:00100000:0.0:1788619920.432958:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571511808 00002000:00100000:0.0:1788619920.433016:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1572864->1576959) req@ffff8e78fe002680 x1875503571511808/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/0 e 0 to 0 dl 1788619931 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619920.433028:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fe002680 with x1875503571511808 ext(1572864->1576959) 00002000:00010000:0.0:1788619920.433034:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fe002680 00000100:00100000:0.0:1788619920.433078:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 489 00000100:00100000:0.0:1788619920.433088:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe002680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571511808:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619920.433460:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe002680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571511808:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 372us (2073us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619920.433467:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 489 00002000:00100000:0.0:1788619920.433474:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1572864->1576959) req@ffff8e78fe002680 x1875503571511808/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/4536 e 0 to 0 dl 1788619931 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619920.433482:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fe002680 with x1875503571511808 ext(1572864->1576959) 00002000:00010000:0.0:1788619920.433804:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fe002680 00000100:00100000:1.0:1788619920.443984:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619920.444186:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571511936 00000100:00100000:2.0:1788619920.444196:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571511936 00002000:00100000:2.0:1788619920.444246:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1576960->1581055) req@ffff8e77c4ba0380 x1875503571511936/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/0 e 0 to 0 dl 1788619931 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619920.444257:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4ba0380 with x1875503571511936 ext(1576960->1581055) 00002000:00010000:2.0:1788619920.444263:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4ba0380 00000100:00100000:2.0:1788619920.444299:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 490 00000100:00100000:2.0:1788619920.444308:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571511936:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619920.444668:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571511936:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 359us (691us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619920.444737:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 490 00002000:00100000:2.0:1788619920.444746:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1576960->1581055) req@ffff8e77c4ba0380 x1875503571511936/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/4536 e 0 to 0 dl 1788619931 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619920.444754:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4ba0380 with x1875503571511936 ext(1576960->1581055) 00002000:00010000:2.0:1788619920.444757:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4ba0380 00000100:00100000:1.0:1788619920.454002:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619920.454487:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571512064 00000100:00100000:2.0:1788619920.454499:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571512064 00002000:00100000:2.0:1788619920.454550:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1581056->1585151) req@ffff8e77c4ba0700 x1875503571512064/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/0 e 0 to 0 dl 1788619931 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619920.454569:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4ba0700 with x1875503571512064 ext(1581056->1585151) 00002000:00010000:2.0:1788619920.454574:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4ba0700 00000100:00100000:2.0:1788619920.454613:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 491 00000100:00100000:2.0:1788619920.454623:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571512064:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619920.455122:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571512064:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 499us (1138us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619920.455130:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 491 00002000:00100000:2.0:1788619920.455137:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1581056->1585151) req@ffff8e77c4ba0700 x1875503571512064/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/4536 e 0 to 0 dl 1788619931 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619920.455145:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4ba0700 with x1875503571512064 ext(1581056->1585151) 00002000:00010000:2.0:1788619920.455149:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4ba0700 00000100:00100000:3.0:1788619920.461022:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619920.461132:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571512192 00000100:00100000:0.0:1788619920.461158:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571512192 00002000:00100000:0.0:1788619920.461208:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1585152->1589247) req@ffff8e78fe001f80 x1875503571512192/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/0 e 0 to 0 dl 1788619931 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619920.461220:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fe001f80 with x1875503571512192 ext(1585152->1589247) 00002000:00010000:0.0:1788619920.461225:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fe001f80 00000100:00100000:0.0:1788619920.461255:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 492 00000100:00100000:0.0:1788619920.461265:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe001f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571512192:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619920.461566:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe001f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571512192:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 292us (544us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619920.461573:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 492 00002000:00100000:0.0:1788619920.461581:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1585152->1589247) req@ffff8e78fe001f80 x1875503571512192/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/4536 e 0 to 0 dl 1788619931 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619920.461588:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fe001f80 with x1875503571512192 ext(1585152->1589247) 00002000:00010000:0.0:1788619920.461592:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fe001f80 00000100:00100000:3.0:1788619920.479858:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619920.480856:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571512320 00000100:00100000:1.0:1788619920.480871:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571512320 00002000:00100000:1.0:1788619920.480926:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1589248->1593343) req@ffff8e78df19e300 x1875503571512320/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/0 e 0 to 0 dl 1788619931 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619920.480938:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df19e300 with x1875503571512320 ext(1589248->1593343) 00002000:00010000:1.0:1788619920.480944:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df19e300 00000100:00100000:1.0:1788619920.480982:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 493 00000100:00100000:1.0:1788619920.481002:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571512320:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619920.481285:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571512320:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 293us (1437us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619920.481293:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 493 00002000:00100000:1.0:1788619920.481300:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1589248->1593343) req@ffff8e78df19e300 x1875503571512320/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/4536 e 0 to 0 dl 1788619931 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619920.481309:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df19e300 with x1875503571512320 ext(1589248->1593343) 00002000:00010000:1.0:1788619920.481313:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df19e300 00000100:00100000:2.0:1788619920.485140:0:3319:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1788619920.485166:0:3319:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900e00e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3319:1875503589145344:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1788619920.485240:0:3319:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788619920.485491:0:6700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589145344 00000100:00100000:0.0:1788619920.485501:0:6700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589145344 00000100:00100000:0.0:1788619920.485572:0:6700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 89 00000100:00100000:0.0:1788619920.485581:0:6700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df968380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1875503589145344:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1788619920.485633:0:6700:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 1703936 granted 115802112 pending 0 free 3859808256 avail 3857711104 00000100:00080000:1.0:1788619920.485771:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1788619920.486209:0:6700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df968380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1875503589145344:12345-0@lo:13:osp-pre-0-0.0 Request processed in 627us (967us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619920.486218:0:6700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 89 00000100:00100000:2.0:1788619920.490616:0:3319:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900e00e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3319:1875503589145344:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1788619920.499987:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619920.500504:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571512448 00000100:00100000:0.0:1788619920.500515:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571512448 00002000:00100000:0.0:1788619920.500564:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1544192->1593343) req@ffff8e78e67f6d80 x1875503571512448/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/0 e 0 to 0 dl 1788619931 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619920.500589:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78e67f6d80 with x1875503571512448 ext(1544192->1593343) 00002000:00010000:0.0:1788619920.500595:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78e67f6d80 00000100:00100000:0.0:1788619920.500630:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 494 00000100:00100000:0.0:1788619920.500640:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571512448:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:1.0:1788619920.511133:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571512448:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 10483us (11144us total) trans 4294967440 rc 0/0 00000100:00100000:1.0:1788619920.511148:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 494 00002000:00100000:1.0:1788619920.511161:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1544192->1593343) req@ffff8e78e67f6d80 x1875503571512448/t4294967440(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/448 e 0 to 0 dl 1788619931 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619920.511172:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78e67f6d80 with x1875503571512448 ext(1544192->1593343) 00002000:00010000:1.0:1788619920.511180:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78e67f6d80 00000100:00100000:1.0:1788619920.516111:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619920.519761:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571512576 00000100:00100000:2.0:1788619920.519777:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571512576 00000100:00100000:2.0:1788619920.519912:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 874 00000100:00100000:2.0:1788619920.519924:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571512576:12345-192.168.201.5@tcp:16:fsx.0 00000001:00080000:3.0:1788619920.583160:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967440 is committed 00000100:00100000:0.0:1788619920.583364:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571512576:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 63434us (67252us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619920.583377:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 874 00000100:00100000:2.0:1788619920.593400:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619920.593490:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571512832 00000100:00100000:0.0:1788619920.593501:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571512832 00000100:00100000:0.0:1788619920.593571:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1093 00000100:00100000:0.0:1788619920.593581:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df968e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571512832:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:0.0:1788619920.593632:0:5846:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619920.593660:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5753400/0x1cd7fe5368735031 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c385 expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619920.593685:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619920.593691:0:5846:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788619920.593946:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe5368735206 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619920.593962:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678e00/0x1cd7fe536873503f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c38c expref: 8 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619920.593971:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe5368735206 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619920.594141:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678e00/0x1cd7fe536873503f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c38c expref: 8 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619920.594156:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678e00/0x1cd7fe536873503f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c38c expref: 8 pid: 5844 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619920.594170:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900e00380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589145472:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619920.594316:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb600 going to sleep for 11 seconds 00000100:00100000:3.0:1788619920.607457:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619920.607574:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571512960 00000100:00100000:1.0:1788619920.607585:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571512960 00010000:00010000:1.0:1788619920.607632:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678e00/0x1cd7fe536873503f lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c38c expref: 9 pid: 5844 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619920.607685:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 365 00000100:00100000:1.0:1788619920.607696:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571512960:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619920.607712:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619920.607999:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678e00/0x1cd7fe536873503f lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c38c expref: 9 pid: 5844 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.608014:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678e00/0x1cd7fe536873503f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c38c expref: 9 pid: 5844 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.608037:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619920.608043:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe5368735206 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.608053:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe5368735206 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.608074:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe5368735206 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.608094:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619920.608161:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571512960:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 464us (712us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619920.608168:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 365 00010000:00010000:0.0:1788619920.609306:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678e00/0x1cd7fe536873503f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c38c expref: 8 pid: 5844 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619920.609348:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900e00380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589145472:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619920.609373:0:5846:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe5368735206 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619920.609382:0:5846:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe5368735206 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619920.609389:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe5368735206 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788619920.609510:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967703 00000004:00010000:0.0:1788619920.609572:0:5846:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 994373 -> 994373 00000004:00010000:0.0:1788619920.609577:0:5846:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619920 -> 1788619920 00000004:00010000:0.0:1788619920.609579:0:5846:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619920 -> 1788619919 00000004:00010000:0.0:1788619920.609582:0:5846:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619920 -> 1788619920 00000004:00010000:0.0:1788619920.609585:0:5846:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1030 -> 1030 00000004:00010000:0.0:1788619920.609603:0:5846:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000fa65235a reply state 000000005f947c74 transno 4294967704 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe5368735206 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619920.609612:0:5846:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788619920.609713:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df968e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:14908:x1875503571512832:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 16132us (16322us total) trans 4294967704 rc 0/0 00000100:00100000:0.0:1788619920.609731:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1093 00000020:00080000:0.0:1788619920.609736:0:5846:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1788619920.612236:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00010000:00010000:2.0:1788619920.612353:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe5368735206 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619920.612364:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe5368735206 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619920.612372:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe5368735206 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619920.612378:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c5ed0400 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:2.0:1788619920.612383:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe5368735206 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619920.612390:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe5368735206 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619920.612407:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619920.612410:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788619920.612415:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe5368735206 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619920.612421:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe5368735206 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619920.613063:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571513088 00000100:00100000:0.0:1788619920.613077:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571513088 00002000:00010000:0.0:1788619920.613123:0:6701:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (2080454->18446744073709551615) 00002000:00100000:0.0:1788619920.613128:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7433480 with x1875503571513088 ext(2080454->18446744073709551615) 00002000:00010000:0.0:1788619920.613140:0:6701:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c7433480. 00000100:00100000:0.0:1788619920.613166:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 495 00000100:00100000:0.0:1788619920.613177:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7433480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571513088:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:2.0:1788619920.616660:0:6701:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1869263 -> 2080454 00002000:00010000:2.0:1788619920.616670:0:6701:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619919 -> 1788619920 00002000:00010000:2.0:1788619920.616673:0:6701:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619919 -> 1788619919 00002000:00010000:2.0:1788619920.616675:0:6701:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619919 -> 1788619920 00002000:00010000:2.0:1788619920.616677:0:6701:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 519 -> 647 00000100:00100000:2.0:1788619920.616831:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7433480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571513088:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 3648us (4599us total) trans 4294967441 rc 0/0 00000100:00100000:2.0:1788619920.616842:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 495 00002000:00010000:2.0:1788619920.616847:0:6701:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (2080454->18446744073709551615) 00002000:00100000:2.0:1788619920.616851:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7433480 with x1875503571513088 ext(2080454->18446744073709551615) 00002000:00010000:2.0:1788619920.616859:0:6701:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c7433480. 00000100:00100000:1.0:1788619920.634201:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619920.634302:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571513216 00000100:00100000:0.0:1788619920.634311:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571513216 00002000:00100000:0.0:1788619920.634390:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1871872->2080767) req@ffff8e78e67f7800 x1875503571513216/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/0 e 0 to 0 dl 1788619931 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619920.634402:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78e67f7800 with x1875503571513216 ext(1871872->2080767) 00002000:00010000:0.0:1788619920.634408:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78e67f7800 00000100:00100000:0.0:1788619920.634429:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 496 00000100:00100000:0.0:1788619920.634437:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571513216:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619920.638656:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571513216:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 4208us (4452us total) trans 0 rc 208896/208896 00000100:00100000:0.0:1788619920.638670:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 496 00002000:00100000:0.0:1788619920.638698:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1871872->2080767) req@ffff8e78e67f7800 x1875503571513216/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/440 e 0 to 0 dl 1788619931 ref 1 fl Complete:/600/0 rc 208896/208896 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619920.638709:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78e67f7800 with x1875503571513216 ext(1871872->2080767) 00002000:00010000:0.0:1788619920.638715:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78e67f7800 00000100:00100000:1.0:1788619920.644097:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619920.644242:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571513344 00000100:00100000:2.0:1788619920.644252:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571513344 00002000:00100000:2.0:1788619920.644303:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (2076672->2080453) req@ffff8e78e67f4a80 x1875503571513344/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 4272/0 e 0 to 0 dl 1788619931 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619920.644313:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78e67f4a80 with x1875503571513344 ext(2076672->2080453) 00002000:00010000:2.0:1788619920.644319:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78e67f4a80 00000100:00100000:2.0:1788619920.644388:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 497 00000100:00100000:2.0:1788619920.644406:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571513344:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:2.0:1788619920.644882:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2355:x1875503571513344:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 483us (793us total) trans 4294967442 rc 0/0 00000100:00100000:2.0:1788619920.644897:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 497 00002000:00100000:2.0:1788619920.644905:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (2076672->2080453) req@ffff8e78e67f4a80 x1875503571513344/t4294967442(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 4272/448 e 0 to 0 dl 1788619931 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619920.644913:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78e67f4a80 with x1875503571513344 ext(2076672->2080453) 00002000:00010000:2.0:1788619920.644917:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78e67f4a80 00000100:00100000:3.0:1788619920.648266:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619920.648664:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571513472 00000100:00100000:2.0:1788619920.648674:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571513472 00000100:00100000:2.0:1788619920.648743:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 875 00000100:00100000:2.0:1788619920.648754:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7430700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2355:x1875503571513472:12345-192.168.201.5@tcp:16:fsx.0 00000100:00100000:1.0:1788619920.770643:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7430700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2355:x1875503571513472:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 121880us (122375us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619920.770657:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 875 00000001:00080000:0.0:1788619920.770760:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967441 is committed 00000001:00080000:0.0:1788619920.770772:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967442 is committed 00000100:00100000:3.0:1788619920.775709:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619920.775934:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571513600 00000100:00100000:0.0:1788619920.775944:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571513600 00000100:00100000:0.0:1788619920.776024:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1094 00000100:00100000:0.0:1788619920.776035:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7431f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571513600:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619920.776059:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619920.776079:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0a00/0x1cd7fe536873520d lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c553 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619920.776169:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe5368735214 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619920.776184:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe5368735214 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619920.778085:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe5368735214 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619920.778141:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 2080454 00000004:00200000:0.0:1788619920.778145:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619920.778168:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe5368735214 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619920.778626:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0a00/0x1cd7fe536873520d lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c553 expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619920.778663:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe5368735214 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c553 expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619920.778673:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619920.778676:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619920.778680:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed0400, rc 0) 00000100:00100000:0.0:1788619920.778757:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7431f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:14908:x1875503571513600:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2723us (3053us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619920.778766:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1094 00000100:00100000:2.0:1788619920.791837:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619920.791928:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571513728 00000100:00100000:0.0:1788619920.791939:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571513728 00000100:00100000:0.0:1788619920.792007:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1095 00000100:00100000:0.0:1788619920.792018:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df96bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+8:14908:x1875503571513728:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619920.792039:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619920.792059:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0a00/0x1cd7fe536873521b lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c55a expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619920.792090:0:11397:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe53687351b9 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c506 expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619920.792101:0:11397:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0a00/0x1cd7fe536873521b lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c55a expref: 9 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619920.792134:0:11397:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe53687351b9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c506 expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619920.792166:0:11397:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe53687351b9 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c506 expref: 14 pid: 9417 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619920.792181:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900e00380 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589145600:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619920.792286:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb540 going to sleep for 11 seconds 00000100:00100000:0.0:1788619920.797042:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900e00380 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589145600:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619920.797083:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0a00/0x1cd7fe536873521b lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c55a expref: 9 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619920.797105:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619920.797108:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619920.797112:0:11397:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0a00/0x1cd7fe536873521b lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c55a expref: 9 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619920.797122:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed0a00, rc 0) 00000100:00100000:0.0:1788619920.797193:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df96bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571513728:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5175us (5363us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619920.797213:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1095 00000100:00100000:3.0:1788619920.812662:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619920.813049:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571513856 00000100:00100000:0.0:1788619920.813060:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571513856 00010000:00010000:0.0:1788619920.813114:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe53687351b9 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c506 expref: 15 pid: 9417 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619920.813162:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 366 00000100:00100000:0.0:1788619920.813172:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7432300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571513856:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619920.813203:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619920.813230:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe53687351b9 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c506 expref: 15 pid: 9417 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619920.813245:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe53687351b9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c506 expref: 15 pid: 9417 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619920.813268:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe53687351b9 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c506 expref: 15 pid: 9417 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619920.813287:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619920.813304:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0a00/0x1cd7fe536873521b lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c55a expref: 8 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619920.813316:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0a00/0x1cd7fe536873521b lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c55a expref: 8 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619920.813375:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0a00/0x1cd7fe536873521b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c55a expref: 8 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619920.813422:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0a00/0x1cd7fe536873521b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c55a expref: 8 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619920.813436:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900e00380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589145728:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619920.813518:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb540 going to sleep for 11 seconds 00000100:00100000:2.0:1788619920.821264:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619920.821396:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900e00380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589145728:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619920.821434:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619920.821492:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7432300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571513856:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 8321us (8838us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619920.821504:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 366 00000100:00100000:3.0:1788619920.822209:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571513984 00000100:00100000:3.0:1788619920.822219:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571513984 00002000:00100000:3.0:1788619920.822265:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df968700 x1875503571513984/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:16/0 lens 488/0 e 0 to 0 dl 1788619931 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619920.822274:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df968700 with x1875503571513984 00000100:00100000:3.0:1788619920.822310:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 706 00000100:00100000:3.0:1788619920.822318:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df968700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571513984:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619920.822535:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df968700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571513984:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 217us (1280us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619920.822713:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 706 00002000:00100000:3.0:1788619920.822730:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df968700 x1875503571513984/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:16/0 lens 488/4536 e 0 to 0 dl 1788619931 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619920.822746:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df968700 with x1875503571513984 00000100:00100000:2.0:1788619920.833265:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619920.833466:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571514112 00000100:00100000:3.0:1788619920.833476:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571514112 00002000:00100000:3.0:1788619920.833533:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df96ad80 x1875503571514112/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:16/0 lens 488/0 e 0 to 0 dl 1788619931 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619920.833543:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df96ad80 with x1875503571514112 00000100:00100000:3.0:1788619920.833613:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 707 00000100:00100000:3.0:1788619920.833622:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df96ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571514112:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619920.833869:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df96ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571514112:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 246us (613us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619920.833877:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 707 00002000:00100000:3.0:1788619920.833884:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df96ad80 x1875503571514112/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:16/0 lens 488/4536 e 0 to 0 dl 1788619931 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619920.833892:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df96ad80 with x1875503571514112 00000100:00100000:0.0:1788619920.840637:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619920.840777:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571514240 00000100:00100000:3.0:1788619920.840788:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571514240 00000100:00100000:3.0:1788619920.840838:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1096 00000100:00100000:3.0:1788619920.840847:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e00380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2352:x1875503571514240:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619920.840868:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619920.840898:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368735222 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c561 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619920.840934:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0a00/0x1cd7fe536873521b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c55a expref: 8 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619920.841007:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fe062300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589145856:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619920.841090:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c000 going to sleep for 11 seconds 00000100:00100000:3.0:1788619920.843189:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe062300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589145856:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619920.843217:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:3.0:1788619920.843233:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368735222 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c561 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619920.843249:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619920.843251:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619920.843255:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368735222 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c561 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619920.843266:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c800, rc 0) 00000100:00100000:3.0:1788619920.843312:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e00380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2352:x1875503571514240:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2465us (2684us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619920.843320:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1096 00000100:00100000:2.0:1788619920.845979:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619920.846146:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571514368 00000100:00100000:0.0:1788619920.846156:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571514368 00000100:00100000:0.0:1788619920.846252:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1097 00000100:00100000:0.0:1788619920.846263:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df96b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2352:x1875503571514368:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619920.846283:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619920.846303:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe5368735229 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c568 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619920.846734:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0a00/0x1cd7fe536873521b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c55a expref: 8 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619920.846778:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900e00a80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589145984:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619920.847279:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb540 going to sleep for 11 seconds 00000100:00100000:1.0:1788619920.849049:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900e00a80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589145984:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619920.849072:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:1.0:1788619920.849088:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe5368735229 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c568 expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.849106:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619920.849108:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619920.849112:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe5368735229 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c568 expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.849123:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed1c00, rc 0) 00000100:00100000:1.0:1788619920.849940:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df96b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2352:x1875503571514368:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3674us (3963us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619920.849951:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1097 00000100:00100000:2.0:1788619920.855128:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619920.855252:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571514496 00000100:00100000:0.0:1788619920.855262:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571514496 00000100:00100000:0.0:1788619920.855559:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1098 00000100:00100000:0.0:1788619920.855571:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571514496:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619920.855591:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619920.855612:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe5368735230 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c56f expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619920.855641:0:11397:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0a00/0x1cd7fe536873521b lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c55a expref: 8 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619920.855664:0:11397:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe5368735230 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c56f expref: 15 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619920.855697:0:11397:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0a00/0x1cd7fe536873521b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c55a expref: 8 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619920.855711:0:11397:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0a00/0x1cd7fe536873521b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c55a expref: 8 pid: 11397 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619920.855737:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900e00700 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589146112:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619920.855822:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619920.858994:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900e00700 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589146112:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619920.859033:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe5368735230 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c56f expref: 15 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619920.859054:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619920.859057:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619920.859062:0:11397:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe5368735230 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c56f expref: 15 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619920.859074:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed1400, rc 0) 00000100:00100000:0.0:1788619920.859131:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+15:14908:x1875503571514496:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3562us (5661us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619920.859141:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1098 00000100:00100000:2.0:1788619920.860388:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619920.860951:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571514624 00000100:00100000:0.0:1788619920.860970:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571514624 00002000:00100000:0.0:1788619920.861017:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c50c6300 x1875503571514624/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:16/0 lens 8680/0 e 0 to 0 dl 1788619931 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619920.861029:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c50c6300 with x1875503571514624 00010000:00010000:0.0:1788619920.861037:0:5859:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0a00/0x1cd7fe536873521b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c55a expref: 9 pid: 11397 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619920.861049:0:5859:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0a00/0x1cd7fe536873521b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c55a expref: 9 pid: 11397 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788619920.861056:0:5859:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e77c50c6300 00000100:00100000:0.0:1788619920.861094:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 26 00000100:00100000:0.0:1788619920.861104:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571514624:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:0.0:1788619920.861731:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 994373 -> 994373 00000004:00010000:0.0:1788619920.861737:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619920 -> 1788619920 00000004:00010000:0.0:1788619920.861740:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619919 -> 1788619919 00000004:00010000:0.0:1788619920.861742:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619920 -> 1788619920 00000004:00010000:0.0:1788619920.861745:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1030 -> 1030 00000100:00100000:0.0:1788619920.861813:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571514624:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 710us (1431us total) trans 4294967705 rc 0/0 00000100:00100000:0.0:1788619920.861822:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 26 00002000:00100000:0.0:1788619920.861829:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c50c6300 x1875503571514624/t4294967705(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:16/0 lens 8680/448 e 0 to 0 dl 1788619931 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619920.861837:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c50c6300 with x1875503571514624 00010000:00010000:0.0:1788619920.861843:0:5859:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0a00/0x1cd7fe536873521b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c55a expref: 11 pid: 11397 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619920.861868:0:5859:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0a00/0x1cd7fe536873521b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c55a expref: 11 pid: 11397 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788619920.861884:0:5859:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e77c50c6300 00000100:00100000:1.0:1788619920.867691:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619920.868075:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571514752 00000100:00100000:2.0:1788619920.868085:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571514752 00010000:00010000:2.0:1788619920.868131:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0a00/0x1cd7fe536873521b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c55a expref: 11 pid: 11397 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619920.868218:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 367 00000100:00100000:2.0:1788619920.868228:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f5f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571514752:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619920.868243:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619920.868269:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0a00/0x1cd7fe536873521b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c55a expref: 11 pid: 11397 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619920.868280:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0a00/0x1cd7fe536873521b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c55a expref: 11 pid: 11397 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619920.868301:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0a00/0x1cd7fe536873521b lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c55a expref: 11 pid: 11397 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619920.868318:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619920.868378:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe5368735230 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c56f expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619920.868391:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe5368735230 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c56f expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619920.868409:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe5368735230 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c56f expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619920.868454:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe5368735230 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c56f expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619920.868467:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c7480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589146240:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619920.868551:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc300 going to sleep for 11 seconds 00000100:00100000:2.0:1788619920.871801:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c7480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589146240:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619920.871827:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619920.871882:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f5f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571514752:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3655us (4200us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619920.871893:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 367 00000100:00100000:0.0:1788619920.872880:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619920.872956:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571514880 00000100:00100000:3.0:1788619920.872964:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571514880 00002000:00100000:3.0:1788619920.873021:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900e00700 x1875503571514880/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/0 e 0 to 0 dl 1788619931 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619920.873032:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900e00700 with x1875503571514880 00000100:00100000:3.0:1788619920.873072:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 708 00000100:00100000:3.0:1788619920.873081:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e00700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571514880:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619920.873281:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e00700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571514880:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 200us (404us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619920.873302:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 708 00002000:00100000:3.0:1788619920.873310:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900e00700 x1875503571514880/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/4536 e 0 to 0 dl 1788619931 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619920.873318:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900e00700 with x1875503571514880 00000100:00100000:0.0:1788619920.878814:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619920.879088:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571515008 00000100:00100000:1.0:1788619920.879098:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571515008 00002000:00100000:1.0:1788619920.879145:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900e01180 x1875503571515008/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/0 e 0 to 0 dl 1788619931 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619920.879157:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900e01180 with x1875503571515008 00000100:00100000:1.0:1788619920.879192:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 709 00000100:00100000:1.0:1788619920.879210:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e01180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2353:x1875503571515008:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619920.879443:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e01180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2353:x1875503571515008:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 234us (639us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619920.879452:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 709 00002000:00100000:1.0:1788619920.879460:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900e01180 x1875503571515008/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/4536 e 0 to 0 dl 1788619931 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619920.879469:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900e01180 with x1875503571515008 00000100:00100000:3.0:1788619920.889651:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619920.889769:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571515136 00000100:00100000:1.0:1788619920.889780:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571515136 00000100:00100000:1.0:1788619920.889841:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1099 00000100:00100000:1.0:1788619920.889852:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe062300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:14908:x1875503571515136:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619920.889948:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619920.889970:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06200/0x1cd7fe5368735237 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c576 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.890054:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07200/0x1cd7fe536873523e lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.890069:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07200/0x1cd7fe536873523e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.890099:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07200/0x1cd7fe536873523e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788619920.890128:0:5846:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 2080454 00000004:00200000:1.0:1788619920.890132:0:5846:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788619920.890150:0:5846:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07200/0x1cd7fe536873523e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.890177:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06200/0x1cd7fe5368735237 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c576 expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.890816:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07200/0x1cd7fe536873523e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c576 expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.890828:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619920.890831:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788619920.890835:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa07200, rc 0) 00000100:00100000:1.0:1788619920.891567:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe062300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571515136:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 1711us (1923us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619920.891576:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1099 00000100:00100000:0.0:1788619920.903831:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619920.903881:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619920.903979:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571515264 00000100:00100000:1.0:1788619920.903998:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571515264 00000100:00100000:1.0:1788619920.904062:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1100 00000100:00100000:1.0:1788619920.904072:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e01f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571515264:12345-192.168.201.5@tcp:101:fsx.0 00000100:00100000:2.0:1788619920.904076:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571515392 00000100:00100000:2.0:1788619920.904080:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571515392 00010000:00010000:1.0:1788619920.904092:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619920.904113:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06200/0x1cd7fe5368735245 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c57d expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619920.904117:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 876 00000100:00100000:2.0:1788619920.904124:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e01500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571515392:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619920.904131:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619920.904141:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ce5e199e/0x1cd7fe536873524c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c584 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.904168:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe5368735230 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c56f expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619920.904189:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589146368:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619920.904189:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f4000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589146496:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619920.904432:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7600 going to sleep for 11 seconds 00000100:00100000:2.0:1788619920.909266:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc300 going to sleep for 11 seconds 00002000:00010000:2.0:1788619920.911935:0:7654:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619919 -> 1788619920 00002000:00010000:2.0:1788619920.911944:0:7654:0:(ofd_lvb.c:284:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks: 647 -> 775 00000100:00100000:2.0:1788619920.911970:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589146368:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619920.912072:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ce5e199e/0x1cd7fe536873524c lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c584 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619920.912096:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ce5e199e/0x1cd7fe536873524c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c584 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619920.912109:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c7679200, rc 0) 00000100:00100000:2.0:1788619920.912170:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e01500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571515392:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8046us (8287us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619920.912180:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 876 00000004:00010000:1.0:1788619920.917308:0:5845:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619919 -> 1788619920 00000100:00100000:1.0:1788619920.917355:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f4000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589146496:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619920.917382:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:1.0:1788619920.917400:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06200/0x1cd7fe5368735245 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c57d expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.917420:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619920.917424:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619920.917430:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06200/0x1cd7fe5368735245 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c57d expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.917445:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa06200, rc 0) 00000100:00100000:1.0:1788619920.917514:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e01f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571515264:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 13443us (13692us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619920.917522:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1100 00000100:00100000:0.0:1788619920.925679:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619920.925773:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619920.926654:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571515648 00000100:00100000:3.0:1788619920.926666:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571515648 00000100:00100000:3.0:1788619920.926740:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 877 00000100:00100000:3.0:1788619920.926751:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e02680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571515648:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619920.926768:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619920.926786:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062c34eb6/0x1cd7fe5368735253 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c592 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619920.926844:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fe061f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589146624:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619920.926926:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c000 going to sleep for 11 seconds 00000100:00100000:3.0:1788619920.930277:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe061f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589146624:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619920.930319:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000062c34eb6/0x1cd7fe5368735253 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c592 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619920.930389:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062c34eb6/0x1cd7fe5368735253 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c592 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619920.930403:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c800, rc 0) 00000100:00100000:3.0:1788619920.930451:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e02680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571515648:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3702us (4677us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619920.930475:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 877 00000100:00100000:1.0:1788619920.932742:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571515520 00000100:00100000:1.0:1788619920.932756:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571515520 00000100:00100000:1.0:1788619920.932824:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1101 00000100:00100000:1.0:1788619920.932839:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e00000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571515520:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619920.932861:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619920.932894:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07800/0x1cd7fe536873525a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c58b expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.932931:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe5368735230 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c56f expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619920.932972:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f4000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589146752:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619920.933060:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7600 going to sleep for 11 seconds 00000100:00100000:1.0:1788619920.937050:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f4000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589146752:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619920.937076:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:1.0:1788619920.937092:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07800/0x1cd7fe536873525a lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c58b expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.937113:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619920.937116:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619920.937121:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07800/0x1cd7fe536873525a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c58b expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.937135:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa07800, rc 0) 00000100:00100000:1.0:1788619920.937183:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e00000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571515520:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4349us (11513us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619920.937191:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1101 00000100:00100000:0.0:1788619920.941434:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619920.941532:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571515776 00000100:00100000:2.0:1788619920.941549:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571515776 00000100:00100000:2.0:1788619920.941604:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 878 00000100:00100000:2.0:1788619920.941613:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e02300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:14908:x1875503571515776:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619920.941633:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619920.941652:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ce5e199e/0x1cd7fe5368735261 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c599 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619920.941684:0:6698:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000000a9a342/0x1cd7fe53687351ff lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1544192->1548287) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c54c expref: 5 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619920.941693:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 7) 00010000:00010000:2.0:1788619920.941697:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(3 7) 00010000:00010000:2.0:1788619920.941702:0:6698:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ce5e199e/0x1cd7fe5368735261 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1056768->1126399] (req 1056768->1126399) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c599 expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619920.941744:0:6698:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000000a9a342/0x1cd7fe53687351ff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1544192->1548287) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c54c expref: 5 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619920.941757:0:6698:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000000a9a342/0x1cd7fe53687351ff lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1544192->1548287) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c54c expref: 5 pid: 6696 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619920.941769:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589146880:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619920.941849:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc300 going to sleep for 11 seconds 00000100:00100000:2.0:1788619920.945785:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589146880:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619920.945825:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ce5e199e/0x1cd7fe5368735261 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1056768->1126399] (req 1056768->1126399) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c599 expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619920.945848:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c7679200, rc 0) 00000100:00100000:2.0:1788619920.945910:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e02300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:14908:x1875503571515776:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4298us (4486us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619920.945919:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 878 00000100:00100000:0.0:1788619920.952448:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619920.952566:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571515904 00000100:00100000:1.0:1788619920.952578:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571515904 00010000:00010000:1.0:1788619920.952625:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000000a9a342/0x1cd7fe53687351ff lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1544192->1548287) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c54c expref: 6 pid: 6696 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619920.952672:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 368 00000100:00100000:1.0:1788619920.952684:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e00a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571515904:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619920.952701:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619920.952736:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000000a9a342/0x1cd7fe53687351ff lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1544192->1548287) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c54c expref: 6 pid: 6696 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.952752:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000000a9a342/0x1cd7fe53687351ff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1544192->1548287) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c54c expref: 6 pid: 6696 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.952774:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000a9a342/0x1cd7fe53687351ff lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1544192->1548287) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c54c expref: 6 pid: 6696 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.952795:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(3 7) 00010000:00010000:1.0:1788619920.952801:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(3 7) 00010000:00010000:1.0:1788619920.952809:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000ce5e199e/0x1cd7fe5368735261 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1056768->1126399) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c599 expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.952818:0:13330:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ce5e199e/0x1cd7fe5368735261 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1056768->1126399) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c599 expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.952856:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000ce5e199e/0x1cd7fe5368735261 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1056768->1126399) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c599 expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619920.952870:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f4000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589147008:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619920.952970:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7600 going to sleep for 11 seconds 00000100:00100000:0.0:1788619920.956171:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619920.956439:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571516032 00000100:00100000:2.0:1788619920.956449:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571516032 00000100:00100000:2.0:1788619920.956521:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 498 00000100:00100000:2.0:1788619920.956532:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e00e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571516032:12345-192.168.201.5@tcp:22:fsx.0 00002000:00010000:2.0:1788619920.956836:0:6703:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 2080454 -> 2080454 00002000:00010000:2.0:1788619920.956839:0:6703:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619920 -> 1788619920 00002000:00010000:2.0:1788619920.956842:0:6703:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619920 -> 1788619919 00002000:00010000:2.0:1788619920.956843:0:6703:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619920 -> 1788619920 00002000:00010000:2.0:1788619920.956845:0:6703:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 775 -> 775 00000100:00100000:2.0:1788619920.956905:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e00e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571516032:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 374us (743us total) trans 4294967443 rc 0/0 00000100:00100000:2.0:1788619920.956914:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 498 00000100:00100000:1.0:1788619920.957953:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f4000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589147008:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619920.957980:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619920.958035:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e00a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571515904:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 5353us (5598us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619920.958045:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 368 00000100:00100000:0.0:1788619920.961147:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619920.961381:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571516160 00000100:00100000:1.0:1788619920.961391:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571516160 00000100:00100000:1.0:1788619920.961446:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 879 00000100:00100000:1.0:1788619920.961455:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74a8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571516160:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619920.961471:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619920.961488:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ce26d4f7/0x1cd7fe5368735268 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c5a0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619920.961555:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f4000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589147136:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619920.961624:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7600 going to sleep for 11 seconds 00002000:00010000:1.0:1788619920.963897:0:6697:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619919 -> 1788619920 00000100:00100000:1.0:1788619920.963937:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f4000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589147136:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619920.963973:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ce26d4f7/0x1cd7fe5368735268 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5a0 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.964007:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ce26d4f7/0x1cd7fe5368735268 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5a0 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.964020:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa07800, rc 0) 00000100:00100000:1.0:1788619920.964079:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74a8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571516160:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2625us (2943us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619920.964091:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 879 00000100:00100000:0.0:1788619920.966442:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619920.966549:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571516288 00000100:00100000:1.0:1788619920.966558:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571516288 00000100:00100000:1.0:1788619920.966605:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 880 00000100:00100000:1.0:1788619920.966614:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74a9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571516288:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619920.966631:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619920.966648:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cb4402d/0x1cd7fe536873526f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c5a7 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619920.966702:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f4000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589147264:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619920.966789:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7600 going to sleep for 11 seconds 00000100:00100000:1.0:1788619920.968845:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f4000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589147264:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619920.968885:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005cb4402d/0x1cd7fe536873526f lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5a7 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.968918:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cb4402d/0x1cd7fe536873526f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5a7 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.968932:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa06200, rc 0) 00000100:00100000:1.0:1788619920.969026:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74a9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571516288:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2412us (2594us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619920.969036:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 880 00000100:00100000:0.0:1788619920.972102:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619920.972199:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571516416 00000100:00100000:1.0:1788619920.972209:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571516416 00000100:00100000:1.0:1788619920.972271:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1102 00000100:00100000:1.0:1788619920.972281:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74a8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571516416:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619920.972302:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619920.972648:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe5368735276 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c5ae expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619920.972734:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe5368735230 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c56f expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619920.972745:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe5368735276 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5ae expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619920.972781:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe5368735230 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c56f expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619920.972795:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe5368735230 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c56f expref: 14 pid: 11397 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619920.972809:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c7480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589147392:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619920.972930:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc300 going to sleep for 11 seconds 00000100:00100000:0.0:1788619920.976759:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619920.977039:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c7480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589147392:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619920.977075:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe5368735276 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5ae expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.977093:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619920.977096:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619920.977100:0:5846:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe5368735276 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5ae expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.977110:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa06800, rc 0) 00000100:00100000:1.0:1788619920.977174:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74a8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:14908:x1875503571516416:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4894us (5081us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619920.977183:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1102 00000100:00100000:3.0:1788619920.977584:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571516544 00000100:00100000:3.0:1788619920.977592:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571516544 00002000:00100000:3.0:1788619920.977635:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c74aa680 x1875503571516544/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/0 e 0 to 0 dl 1788619931 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619920.977646:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c74aa680 with x1875503571516544 00010000:00010000:3.0:1788619920.977653:0:5857:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe5368735230 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c56f expref: 15 pid: 11397 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619920.977667:0:5857:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe5368735230 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c56f expref: 15 pid: 11397 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788619920.977674:0:5857:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e77c74aa680 00000100:00100000:3.0:1788619920.977704:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 27 00000100:00100000:3.0:1788619920.977725:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74aa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571516544:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:1.0:1788619920.981817:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 994373 -> 994373 00000004:00010000:1.0:1788619920.981826:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619920 -> 1788619920 00000004:00010000:1.0:1788619920.981829:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619920 -> 1788619919 00000004:00010000:1.0:1788619920.981831:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619920 -> 1788619920 00000004:00010000:1.0:1788619920.981833:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1030 -> 1030 00000100:00100000:1.0:1788619920.981913:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74aa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2355:x1875503571516544:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 4195us (5158us total) trans 4294967706 rc 0/0 00000100:00100000:1.0:1788619920.981928:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 27 00002000:00100000:1.0:1788619920.981940:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c74aa680 x1875503571516544/t4294967706(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:16/0 lens 488/448 e 0 to 0 dl 1788619931 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619920.981951:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c74aa680 with x1875503571516544 00010000:00010000:1.0:1788619920.981960:0:5857:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe5368735230 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c56f expref: 17 pid: 11397 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.981977:0:5857:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe5368735230 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c56f expref: 17 pid: 11397 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788619920.981995:0:5857:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e77c74aa680 00000100:00100000:3.0:1788619920.984496:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619920.985360:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571516672 00000100:00100000:1.0:1788619920.985370:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571516672 00010000:00010000:1.0:1788619920.985412:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe5368735230 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c56f expref: 17 pid: 11397 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619920.985455:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 369 00000100:00100000:1.0:1788619920.985466:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c855a450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2354:x1875503571516672:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619920.985480:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619920.985506:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe5368735230 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c56f expref: 17 pid: 11397 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.985519:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe5368735230 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c56f expref: 17 pid: 11397 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.985552:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe5368735230 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c56f expref: 17 pid: 11397 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.985572:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619920.985578:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe5368735276 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5ae expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.985590:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe5368735276 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5ae expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.985599:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe5368735276 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5ae expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.985638:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe5368735276 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5ae expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619920.985652:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f4000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589147520:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619920.985737:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7600 going to sleep for 11 seconds 00000100:00100000:2.0:1788619920.987560:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f4000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589147520:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619920.987587:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619920.987632:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c855a450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571516672:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 2168us (3145us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619920.987641:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 369 00000100:00100000:3.0:1788619920.989600:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619920.989673:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571516800 00000100:00100000:1.0:1788619920.989681:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571516800 00000100:00100000:1.0:1788619920.989756:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 710 00000100:00100000:1.0:1788619920.989766:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe061f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571516800:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:1.0:1788619920.990009:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 994373 -> 994373 00000004:00010000:1.0:1788619920.990013:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619920 -> 1788619920 00000004:00010000:1.0:1788619920.990016:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619919 -> 1788619919 00000004:00010000:1.0:1788619920.990018:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619920 -> 1788619920 00000004:00010000:1.0:1788619920.990020:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1030 -> 1030 00000100:00100000:1.0:1788619920.990086:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe061f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571516800:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 322us (491us total) trans 4294967707 rc 0/0 00000100:00100000:1.0:1788619920.990093:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 710 00000100:00100000:0.0:1788619920.995027:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619920.995071:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619920.995165:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571516928 00000100:00100000:1.0:1788619920.995175:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571516928 00000100:00100000:1.0:1788619920.995235:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1103 00000100:00100000:1.0:1788619920.995244:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74a8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571516928:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619920.995268:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619920.995288:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06200/0x1cd7fe536873527d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c5b5 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619920.995359:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe5368735276 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5ae expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619920.995399:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f5500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589147648:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619920.995472:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7780 going to sleep for 11 seconds 00000100:00100000:3.0:1788619920.995785:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571517056 00000100:00100000:3.0:1788619920.995790:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571517056 00000100:00100000:3.0:1788619920.995833:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 881 00000100:00100000:3.0:1788619920.995841:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74ab800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571517056:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619920.995849:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619920.995860:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe5368735284 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c5bc expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619920.995897:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fe063100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589147776:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619920.995952:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c000 going to sleep for 11 seconds 00000100:00100000:3.0:1788619920.998693:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe063100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589147776:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619920.998741:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe5368735284 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5bc expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619920.998765:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe5368735284 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5bc expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619920.998777:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683ca00, rc 0) 00000100:00100000:3.0:1788619920.998831:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74ab800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571517056:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2990us (3814us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619920.998838:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 881 00000004:00010000:1.0:1788619921.001836:0:5845:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619919 -> 1788619920 00000100:00100000:1.0:1788619921.001864:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f5500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589147648:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619921.001889:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:1.0:1788619921.001919:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06200/0x1cd7fe536873527d lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5b5 expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.001939:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619921.001942:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619921.001947:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06200/0x1cd7fe536873527d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5b5 expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.001968:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa06200, rc 0) 00000100:00100000:1.0:1788619921.002036:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74a8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571516928:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6791us (6962us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619921.002044:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1103 00000100:00100000:0.0:1788619921.005631:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619921.005666:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.005798:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571517312 00000100:00100000:2.0:1788619921.005807:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571517312 00000100:00100000:2.0:1788619921.005868:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 882 00000100:00100000:2.0:1788619921.005878:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74aa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571517312:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619921.005895:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619921.005913:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002a87657c/0x1cd7fe536873528b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c5ca expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619921.005971:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589147904:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619921.006071:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc600 going to sleep for 11 seconds 00000100:00100000:1.0:1788619921.006453:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571517184 00000100:00100000:1.0:1788619921.006458:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571517184 00000100:00100000:1.0:1788619921.006496:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1104 00000100:00100000:1.0:1788619921.006503:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74aad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2352:x1875503571517184:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619921.006515:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619921.006529:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07800/0x1cd7fe5368735292 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c5c3 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.006562:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe5368735276 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5ae expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619921.006587:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f5500 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589148032:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619921.006622:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7780 going to sleep for 11 seconds 00000100:00100000:2.0:1788619921.010371:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589147904:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619921.010410:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002a87657c/0x1cd7fe536873528b lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5ca expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.010435:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002a87657c/0x1cd7fe536873528b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5ca expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.010448:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c7678400, rc 0) 00000100:00100000:2.0:1788619921.010498:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74aa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571517312:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4621us (4829us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619921.010506:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 882 00000100:00100000:1.0:1788619921.010629:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f5500 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589148032:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619921.010644:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:1.0:1788619921.010656:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07800/0x1cd7fe5368735292 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5c3 expref: 17 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.010669:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619921.010672:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619921.010675:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07800/0x1cd7fe5368735292 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5c3 expref: 17 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.010687:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa07800, rc 0) 00000100:00100000:1.0:1788619921.010709:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74aad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2352:x1875503571517184:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4207us (5088us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619921.010726:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1104 00000100:00100000:0.0:1788619921.014484:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619921.014591:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571517440 00000100:00100000:1.0:1788619921.014600:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571517440 00000100:00100000:1.0:1788619921.014655:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 883 00000100:00100000:1.0:1788619921.014665:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74a8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:14908:x1875503571517440:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619921.014682:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619921.014699:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cb4402d/0x1cd7fe5368735299 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c5d1 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.014761:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000ce5e199e/0x1cd7fe5368735261 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1056768->1126399) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c599 expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.014770:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(3 7) 00010000:00010000:1.0:1788619921.014775:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(4 7) 00010000:00010000:1.0:1788619921.014779:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cb4402d/0x1cd7fe5368735299 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1277952->1327103] (req 1277952->1327103) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5d1 expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.014814:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000ce5e199e/0x1cd7fe5368735261 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1056768->1126399) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c599 expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.014827:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000ce5e199e/0x1cd7fe5368735261 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1056768->1126399) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c599 expref: 6 pid: 6698 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619921.014845:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f5500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589148160:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619921.015068:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7780 going to sleep for 11 seconds 00000100:00100000:0.0:1788619921.020371:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619921.020474:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f5500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589148160:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619921.020509:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005cb4402d/0x1cd7fe5368735299 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1277952->1327103] (req 1277952->1327103) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5d1 expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.020532:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa06200, rc 0) 00000100:00100000:1.0:1788619921.020599:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74a8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:14908:x1875503571517440:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5935us (6123us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619921.020608:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 883 00000100:00100000:3.0:1788619921.020891:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571517568 00000100:00100000:3.0:1788619921.020900:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571517568 00010000:00010000:3.0:1788619921.020941:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000ce5e199e/0x1cd7fe5368735261 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1056768->1126399) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c599 expref: 7 pid: 6698 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619921.020974:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 370 00000100:00100000:3.0:1788619921.020991:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74aaa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2353:x1875503571517568:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619921.021004:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619921.021026:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000ce5e199e/0x1cd7fe5368735261 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1056768->1126399) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c599 expref: 7 pid: 6698 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619921.021038:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000ce5e199e/0x1cd7fe5368735261 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1056768->1126399) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c599 expref: 7 pid: 6698 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619921.021082:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ce5e199e/0x1cd7fe5368735261 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1056768->1126399) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c599 expref: 7 pid: 6698 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619921.021094:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(4 7) 00010000:00010000:3.0:1788619921.021098:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(4 7) 00010000:00010000:3.0:1788619921.021106:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000005cb4402d/0x1cd7fe5368735299 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1277952->1327103) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5d1 expref: 5 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619921.021116:0:5829:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cb4402d/0x1cd7fe5368735299 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1277952->1327103) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5d1 expref: 5 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619921.021152:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000005cb4402d/0x1cd7fe5368735299 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1277952->1327103) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5d1 expref: 5 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619921.021166:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fe063100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1875503589148288:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619921.021218:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c000 going to sleep for 11 seconds 00000100:00100000:3.0:1788619921.024805:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe063100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1875503589148288:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619921.024831:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619921.024930:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74aaa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571517568:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3939us (4568us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619921.024943:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 370 00000100:00100000:0.0:1788619921.027953:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.028072:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571517696 00000100:00100000:2.0:1788619921.028082:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571517696 00000100:00100000:2.0:1788619921.028150:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 499 00000100:00100000:2.0:1788619921.028162:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74a8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571517696:12345-192.168.201.5@tcp:22:fsx.0 00002000:00010000:2.0:1788619921.028432:0:6702:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 2080454 -> 2080454 00002000:00010000:2.0:1788619921.028436:0:6702:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619920 -> 1788619920 00002000:00010000:2.0:1788619921.028440:0:6702:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619920 -> 1788619919 00002000:00010000:2.0:1788619921.028442:0:6702:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619920 -> 1788619920 00002000:00010000:2.0:1788619921.028443:0:6702:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 775 -> 775 00000100:00100000:2.0:1788619921.028494:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74a8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571517696:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 335us (551us total) trans 4294967444 rc 0/0 00000100:00100000:2.0:1788619921.028500:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 499 00000100:00100000:0.0:1788619921.036411:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619921.036504:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571517824 00000100:00100000:1.0:1788619921.036514:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571517824 00000100:00100000:1.0:1788619921.036675:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1105 00000100:00100000:1.0:1788619921.036686:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74ab100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2353:x1875503571517824:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619921.036707:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619921.036739:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07800/0x1cd7fe53687352a0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c5d8 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.036778:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe5368735276 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5ae expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619921.036821:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f5500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589148416:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619921.036948:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7780 going to sleep for 11 seconds 00000100:00100000:1.0:1788619921.039277:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f5500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589148416:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619921.039303:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:1.0:1788619921.039323:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07800/0x1cd7fe53687352a0 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5d8 expref: 17 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.039407:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619921.039410:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619921.039416:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07800/0x1cd7fe53687352a0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5d8 expref: 17 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.039430:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa07800, rc 0) 00000100:00100000:1.0:1788619921.039523:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74ab100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2353:x1875503571517824:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2837us (3121us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619921.039533:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1105 00000100:00100000:0.0:1788619921.042408:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.042506:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571517952 00000100:00100000:2.0:1788619921.042516:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571517952 00000100:00100000:2.0:1788619921.042583:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1106 00000100:00100000:2.0:1788619921.042593:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74a9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2352:x1875503571517952:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619921.042613:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619921.042633:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678400/0x1cd7fe53687352a7 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c5df expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.042668:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe5368735276 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5ae expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619921.042706:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c7480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589148544:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619921.042794:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc600 going to sleep for 11 seconds 00000100:00100000:2.0:1788619921.044639:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c7480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589148544:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619921.044661:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:2.0:1788619921.044678:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678400/0x1cd7fe53687352a7 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5df expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.044696:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619921.044699:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619921.044704:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678400/0x1cd7fe53687352a7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5df expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.044729:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c7678400, rc 0) 00000100:00100000:2.0:1788619921.044783:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74a9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2352:x1875503571517952:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2191us (2384us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619921.044792:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1106 00000100:00100000:0.0:1788619921.048097:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619921.048190:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571518080 00000100:00100000:1.0:1788619921.048199:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571518080 00000100:00100000:1.0:1788619921.048286:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1107 00000100:00100000:1.0:1788619921.048296:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74abb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2353:x1875503571518080:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619921.048317:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619921.048360:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe53687352ae lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c5e6 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.048398:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe5368735276 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5ae expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619921.048435:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f5500 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589148672:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619921.048505:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7780 going to sleep for 11 seconds 00000100:00100000:0.0:1788619921.050381:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f5500 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589148672:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619921.050407:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:0.0:1788619921.050424:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe53687352ae lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5e6 expref: 17 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.050442:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619921.050445:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619921.050451:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe53687352ae lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5e6 expref: 17 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.050466:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa07a00, rc 0) 00000100:00100000:0.0:1788619921.050516:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74abb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2353:x1875503571518080:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2222us (2428us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619921.050525:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1107 00000100:00100000:3.0:1788619921.053048:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.053536:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571518208 00000100:00100000:2.0:1788619921.053554:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571518208 00000100:00100000:2.0:1788619921.053612:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1108 00000100:00100000:2.0:1788619921.053622:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe063100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2352:x1875503571518208:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619921.053641:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619921.053659:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679800/0x1cd7fe53687352b5 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c5ed expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.053695:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe5368735276 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5ae expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619921.053750:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c7480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589148800:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619921.053827:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc600 going to sleep for 11 seconds 00000100:00100000:2.0:1788619921.056467:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c7480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589148800:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619921.056496:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:2.0:1788619921.056515:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679800/0x1cd7fe53687352b5 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5ed expref: 17 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.056636:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619921.056640:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619921.056647:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679800/0x1cd7fe53687352b5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5ed expref: 17 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.056662:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c7679800, rc 0) 00000100:00100000:2.0:1788619921.056768:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe063100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2352:x1875503571518208:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3147us (3729us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619921.056777:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1108 00000100:00100000:1.0:1788619921.059886:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.059974:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571518336 00000100:00100000:2.0:1788619921.059996:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571518336 00000100:00100000:2.0:1788619921.060054:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1109 00000100:00100000:2.0:1788619921.060065:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:14908:x1875503571518336:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619921.060085:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619921.060107:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678400/0x1cd7fe53687352bc lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c5f4 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.060138:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe5368735276 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5ae expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.060149:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678400/0x1cd7fe53687352bc lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5f4 expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.060184:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe5368735276 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5ae expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.060197:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe5368735276 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5ae expref: 12 pid: 5846 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619921.060210:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c7480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589148928:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619921.060291:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc600 going to sleep for 11 seconds 00000100:00100000:1.0:1788619921.062883:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c7480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589148928:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619921.062931:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678400/0x1cd7fe53687352bc lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5f4 expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.062948:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619921.062951:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619921.062956:0:5845:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678400/0x1cd7fe53687352bc lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5f4 expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.062966:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c7678400, rc 0) 00000100:00100000:1.0:1788619921.063040:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+17:14908:x1875503571518336:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2977us (3164us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619921.063048:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1109 00000100:00100000:1.0:1788619921.070219:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.070306:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571518464 00000100:00100000:2.0:1788619921.070317:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571518464 00010000:00010000:2.0:1788619921.070396:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe5368735276 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5ae expref: 13 pid: 5846 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619921.070444:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 371 00000100:00100000:2.0:1788619921.070454:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571518464:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619921.070468:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619921.070495:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe5368735276 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5ae expref: 13 pid: 5846 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.070508:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe5368735276 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5ae expref: 13 pid: 5846 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.070530:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe5368735276 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5ae expref: 13 pid: 5846 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.070554:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619921.070561:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678400/0x1cd7fe53687352bc lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5f4 expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.070574:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678400/0x1cd7fe53687352bc lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5f4 expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.070583:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678400/0x1cd7fe53687352bc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5f4 expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.070621:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678400/0x1cd7fe53687352bc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5f4 expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619921.070639:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c7480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589149056:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619921.070732:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccd80 going to sleep for 11 seconds 00000100:00100000:3.0:1788619921.073302:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c7480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589149056:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619921.073364:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619921.073436:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571518464:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 2982us (3225us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619921.073447:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 371 00000100:00100000:3.0:1788619921.075918:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619921.076008:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571518592 00000100:00100000:1.0:1788619921.076018:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571518592 00002000:00100000:1.0:1788619921.076066:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fe061180 x1875503571518592/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:17/0 lens 488/0 e 0 to 0 dl 1788619932 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619921.076077:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fe061180 with x1875503571518592 00000100:00100000:1.0:1788619921.076114:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 711 00000100:00100000:1.0:1788619921.076123:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe061180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2354:x1875503571518592:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619921.076436:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe061180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2354:x1875503571518592:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 312us (525us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619921.076444:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 711 00002000:00100000:1.0:1788619921.076450:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fe061180 x1875503571518592/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:17/0 lens 488/4536 e 0 to 0 dl 1788619932 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619921.076463:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fe061180 with x1875503571518592 00000100:00100000:1.0:1788619921.079462:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.079564:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571518720 00000100:00100000:2.0:1788619921.079573:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571518720 00002000:00100000:2.0:1788619921.079622:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e67f6680 x1875503571518720/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:17/0 lens 488/0 e 0 to 0 dl 1788619932 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619921.079632:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e67f6680 with x1875503571518720 00000100:00100000:2.0:1788619921.079671:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 712 00000100:00100000:2.0:1788619921.079682:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2355:x1875503571518720:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619921.079950:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2355:x1875503571518720:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 270us (498us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619921.079957:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 712 00002000:00100000:2.0:1788619921.079964:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e67f6680 x1875503571518720/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:17/0 lens 488/4536 e 0 to 0 dl 1788619932 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619921.079971:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e67f6680 with x1875503571518720 00000100:00100000:2.0:1788619921.084063:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619921.084200:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571518976 00000100:00100000:1.0:1788619921.084210:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571518976 00000100:00100000:1.0:1788619921.084270:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1110 00000100:00100000:1.0:1788619921.084280:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+17:14908:x1875503571518976:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:1.0:1788619921.084370:0:5846:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619921.084397:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe5368735214 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c553 expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.084415:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619921.084420:0:5846:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788619921.084639:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06a00/0x1cd7fe53687352c3 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.084654:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07200/0x1cd7fe536873523e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c576 expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.084662:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06a00/0x1cd7fe53687352c3 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.084691:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07200/0x1cd7fe536873523e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c576 expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.084704:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07200/0x1cd7fe536873523e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c576 expref: 11 pid: 5846 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619921.084734:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f5c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589149184:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619921.084803:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e70c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619921.087590:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619921.087780:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f5c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589149184:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619921.087807:0:5846:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06a00/0x1cd7fe53687352c3 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619921.087880:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571519104 00000100:00100000:0.0:1788619921.087888:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571519104 00010000:00010000:0.0:1788619921.087936:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07200/0x1cd7fe536873523e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c576 expref: 12 pid: 5846 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619921.087973:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 372 00000100:00100000:0.0:1788619921.087992:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe060380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571519104:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619921.088012:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619921.088020:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07200/0x1cd7fe536873523e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c576 expref: 12 pid: 5846 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.088033:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07200/0x1cd7fe536873523e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c576 expref: 12 pid: 5846 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.088073:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07200/0x1cd7fe536873523e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c576 expref: 12 pid: 5846 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.088089:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619921.088095:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06a00/0x1cd7fe53687352c3 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.088107:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06a00/0x1cd7fe53687352c3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.088115:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06a00/0x1cd7fe53687352c3 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.088144:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619921.088193:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe060380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571519104:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 203us (611us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619921.088201:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 372 00010000:00010000:1.0:1788619921.088250:0:5846:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06a00/0x1cd7fe53687352c3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.088257:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06a00/0x1cd7fe53687352c3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788619921.088405:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967708 00000004:00010000:1.0:1788619921.088454:0:5846:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 994373 -> 994373 00000004:00010000:1.0:1788619921.088458:0:5846:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619920 -> 1788619920 00000004:00010000:1.0:1788619921.088461:0:5846:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619920 -> 1788619919 00000004:00010000:1.0:1788619921.088463:0:5846:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619920 -> 1788619920 00000004:00010000:1.0:1788619921.088466:0:5846:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1030 -> 1030 00000004:00010000:1.0:1788619921.088473:0:5846:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000097e15dde reply state 00000000e5422c7d transno 4294967709 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06a00/0x1cd7fe53687352c3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619921.088481:0:5846:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788619921.088559:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+19:14908:x1875503571518976:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 4280us (4505us total) trans 4294967709 rc 0/0 00000100:00100000:1.0:1788619921.088566:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1110 00000020:00080000:1.0:1788619921.088570:0:5846:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:2.0:1788619921.091095:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06a00/0x1cd7fe53687352c3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.091113:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06a00/0x1cd7fe53687352c3 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.091121:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06a00/0x1cd7fe53687352c3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.091127:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78dfa06a00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:2.0:1788619921.091132:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06a00/0x1cd7fe53687352c3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.091139:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06a00/0x1cd7fe53687352c3 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.091159:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619921.091163:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788619921.091168:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06a00/0x1cd7fe53687352c3 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.091174:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06a00/0x1cd7fe53687352c3 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619921.092270:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.092559:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571519232 00000100:00100000:2.0:1788619921.092569:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571519232 00002000:00010000:2.0:1788619921.092606:0:6701:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1226248->18446744073709551615) 00002000:00100000:2.0:1788619921.092612:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fe063b80 with x1875503571519232 ext(1226248->18446744073709551615) 00002000:00010000:2.0:1788619921.092624:0:6701:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fe063b80. 00000100:00100000:2.0:1788619921.092663:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 500 00000100:00100000:2.0:1788619921.092673:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe063b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2353:x1875503571519232:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:2.0:1788619921.093007:0:6701:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 2080454 -> 1226248 00002000:00010000:2.0:1788619921.093010:0:6701:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619920 -> 1788619920 00002000:00010000:2.0:1788619921.093012:0:6701:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619919 -> 1788619919 00002000:00010000:2.0:1788619921.093015:0:6701:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619920 -> 1788619920 00002000:00010000:2.0:1788619921.093017:0:6701:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 775 -> 775 00000100:00100000:2.0:1788619921.093077:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe063b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571519232:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 404us (813us total) trans 4294967445 rc 0/0 00000100:00100000:2.0:1788619921.093085:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 500 00002000:00010000:2.0:1788619921.093089:0:6701:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1226248->18446744073709551615) 00002000:00100000:2.0:1788619921.093091:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fe063b80 with x1875503571519232 ext(1226248->18446744073709551615) 00002000:00010000:2.0:1788619921.093096:0:6701:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fe063b80. 00000100:00100000:0.0:1788619921.096074:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.096154:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571519360 00000100:00100000:2.0:1788619921.096164:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571519360 00000100:00100000:2.0:1788619921.096219:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1111 00000100:00100000:2.0:1788619921.096229:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74ab480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+18:14908:x1875503571519360:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619921.096252:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619921.096270:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678a00/0x1cd7fe53687352ca lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c5fb expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.096381:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679e00/0x1cd7fe53687352d1 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.096396:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679e00/0x1cd7fe53687352d1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.096408:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679e00/0x1cd7fe53687352d1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788619921.096434:0:9417:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1226248 00000004:00200000:2.0:1788619921.096438:0:9417:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788619921.096455:0:9417:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679e00/0x1cd7fe53687352d1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.096482:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678a00/0x1cd7fe53687352ca lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5fb expref: 20 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.096499:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679e00/0x1cd7fe53687352d1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5fb expref: 19 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.096513:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619921.096516:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788619921.096520:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c7679e00, rc 0) 00000100:00100000:2.0:1788619921.096583:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74ab480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+19:14908:x1875503571519360:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 354us (518us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619921.096589:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1111 00000100:00100000:3.0:1788619921.100042:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619921.100184:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571519488 00000100:00100000:1.0:1788619921.100193:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571519488 00000100:00100000:1.0:1788619921.100250:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1112 00000100:00100000:1.0:1788619921.100259:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe061c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571519488:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619921.100280:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619921.100305:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06e00/0x1cd7fe53687352d8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c602 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.100420:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07c00/0x1cd7fe53687352df lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.100436:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07c00/0x1cd7fe53687352df lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.100448:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07c00/0x1cd7fe53687352df lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788619921.100473:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1226248 00000004:00200000:1.0:1788619921.100477:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788619921.100494:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07c00/0x1cd7fe53687352df lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.100519:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06e00/0x1cd7fe53687352d8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c602 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.100535:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07c00/0x1cd7fe53687352df lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c602 expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.100554:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619921.100557:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788619921.100561:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa07c00, rc 0) 00000100:00100000:1.0:1788619921.100610:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe061c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:14908:x1875503571519488:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 352us (577us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619921.100616:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1112 00000100:00100000:0.0:1788619921.105103:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619921.105148:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619921.105263:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571519744 00000100:00100000:3.0:1788619921.105273:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571519744 00000100:00100000:3.0:1788619921.105357:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 884 00000100:00100000:3.0:1788619921.105367:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571519744:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619921.105387:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619921.105404:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe53687352e6 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c610 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619921.105461:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fe062d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589149312:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619921.105537:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c000 going to sleep for 11 seconds 00000100:00100000:1.0:1788619921.105633:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571519616 00000100:00100000:1.0:1788619921.105639:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571519616 00000100:00100000:1.0:1788619921.105667:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1113 00000100:00100000:1.0:1788619921.105674:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74a9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571519616:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619921.105684:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619921.105697:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07600/0x1cd7fe53687352ed lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c609 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.105735:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678400/0x1cd7fe53687352bc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5f4 expref: 18 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619921.105761:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f5c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589149440:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619921.105791:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e70c0 going to sleep for 11 seconds 00000004:00010000:1.0:1788619921.108357:0:5846:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619919 -> 1788619920 00000100:00100000:1.0:1788619921.108395:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f5c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589149440:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619921.108419:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:1.0:1788619921.108436:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07600/0x1cd7fe53687352ed lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c609 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.108455:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619921.108459:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619921.108464:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07600/0x1cd7fe53687352ed lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c609 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.108483:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa07600, rc 0) 00000100:00100000:1.0:1788619921.108531:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74a9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571519616:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2857us (3440us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619921.108540:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1113 00002000:00010000:3.0:1788619921.108622:0:7654:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619919 -> 1788619920 00000100:00100000:3.0:1788619921.108638:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe062d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589149312:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619921.108657:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe53687352e6 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c610 expref: 7 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619921.108673:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe53687352e6 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c610 expref: 7 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619921.108682:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683ca00, rc 0) 00000100:00100000:3.0:1788619921.108735:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571519744:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3370us (3586us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619921.108741:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 884 00000100:00100000:0.0:1788619921.112432:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619921.112469:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619921.112574:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571520000 00000100:00100000:1.0:1788619921.112584:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571520000 00000100:00100000:1.0:1788619921.112633:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 885 00000100:00100000:1.0:1788619921.112643:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571520000:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619921.112660:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619921.112678:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000833280d1/0x1cd7fe53687352f4 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c61e expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619921.112751:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589149568:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619921.112823:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e70c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619921.113473:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571519872 00000100:00100000:3.0:1788619921.113479:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571519872 00000100:00100000:3.0:1788619921.113528:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1114 00000100:00100000:3.0:1788619921.113545:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2352:x1875503571519872:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619921.113557:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619921.113572:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe53687352fb lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c617 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619921.113602:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678400/0x1cd7fe53687352bc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5f4 expref: 18 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619921.113630:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fe062d80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589149696:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619921.113676:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c000 going to sleep for 11 seconds 00000100:00100000:1.0:1788619921.117382:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589149568:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619921.117424:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000833280d1/0x1cd7fe53687352f4 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c61e expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.117448:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000833280d1/0x1cd7fe53687352f4 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c61e expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.117461:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa06e00, rc 0) 00000100:00100000:1.0:1788619921.117514:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571520000:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4872us (5044us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619921.117527:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 885 00000100:00100000:3.0:1788619921.118367:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe062d80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589149696:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619921.118386:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:3.0:1788619921.118400:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe53687352fb lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c617 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619921.118414:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619921.118417:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619921.118422:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe53687352fb lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c617 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619921.118435:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c800, rc 0) 00000100:00100000:3.0:1788619921.118497:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2352:x1875503571519872:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4961us (6073us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619921.118504:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1114 00000100:00100000:0.0:1788619921.122165:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.122275:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571520128 00000100:00100000:2.0:1788619921.122285:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571520128 00000100:00100000:2.0:1788619921.122473:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1115 00000100:00100000:2.0:1788619921.122486:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+12:14908:x1875503571520128:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619921.122510:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619921.122532:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678a00/0x1cd7fe5368735302 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c625 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.122563:0:11397:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678400/0x1cd7fe53687352bc lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5f4 expref: 18 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.122574:0:11397:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678a00/0x1cd7fe5368735302 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c625 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.122607:0:11397:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678400/0x1cd7fe53687352bc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5f4 expref: 18 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.122620:0:11397:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678400/0x1cd7fe53687352bc lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5f4 expref: 18 pid: 5845 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619921.122637:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fe000700 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589149824:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619921.122798:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc240 going to sleep for 11 seconds 00000100:00100000:0.0:1788619921.126241:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619921.126317:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571520256 00000100:00100000:3.0:1788619921.126358:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571520256 00002000:00100000:3.0:1788619921.126407:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fefd5500 x1875503571520256/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:17/0 lens 488/0 e 0 to 0 dl 1788619932 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619921.126418:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fefd5500 with x1875503571520256 00010000:00010000:3.0:1788619921.126426:0:5858:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678400/0x1cd7fe53687352bc lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5f4 expref: 19 pid: 5845 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619921.126442:0:5858:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678400/0x1cd7fe53687352bc lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5f4 expref: 19 pid: 5845 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788619921.126449:0:5858:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e78fefd5500 00000100:00100000:3.0:1788619921.126482:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 28 00000100:00100000:3.0:1788619921.126491:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2354:x1875503571520256:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:2.0:1788619921.126980:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe000700 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589149824:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619921.127008:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678a00/0x1cd7fe5368735302 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c625 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.127023:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619921.127026:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619921.127030:0:11397:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678a00/0x1cd7fe5368735302 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c625 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.127040:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c7678a00, rc 0) 00000100:00100000:2.0:1788619921.127084:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+13:14908:x1875503571520128:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4602us (4930us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619921.127091:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1115 00000004:00010000:1.0:1788619921.131671:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 994373 -> 994373 00000004:00010000:1.0:1788619921.131679:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619920 -> 1788619920 00000004:00010000:1.0:1788619921.131682:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619920 -> 1788619919 00000004:00010000:1.0:1788619921.131684:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619920 -> 1788619920 00000004:00010000:1.0:1788619921.131686:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1030 -> 1030 00000100:00100000:1.0:1788619921.131784:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2354:x1875503571520256:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 5289us (5547us total) trans 4294967710 rc 0/0 00000100:00100000:1.0:1788619921.131795:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 28 00002000:00100000:1.0:1788619921.131808:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fefd5500 x1875503571520256/t4294967710(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:17/0 lens 488/448 e 0 to 0 dl 1788619932 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619921.131817:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fefd5500 with x1875503571520256 00010000:00010000:1.0:1788619921.131825:0:5858:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678400/0x1cd7fe53687352bc lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5f4 expref: 21 pid: 5845 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.131841:0:5858:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678400/0x1cd7fe53687352bc lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5f4 expref: 21 pid: 5845 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788619921.131848:0:5858:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e78fefd5500 00000100:00100000:1.0:1788619921.134320:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.134474:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571520384 00000100:00100000:2.0:1788619921.134484:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571520384 00010000:00010000:2.0:1788619921.134524:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678400/0x1cd7fe53687352bc lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5f4 expref: 21 pid: 5845 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619921.134570:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 373 00000100:00100000:2.0:1788619921.134580:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2355:x1875503571520384:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619921.134595:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619921.134617:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678400/0x1cd7fe53687352bc lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5f4 expref: 21 pid: 5845 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.134630:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678400/0x1cd7fe53687352bc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5f4 expref: 21 pid: 5845 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.134650:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678400/0x1cd7fe53687352bc lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5f4 expref: 21 pid: 5845 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.134664:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619921.134670:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678a00/0x1cd7fe5368735302 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c625 expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.134681:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678a00/0x1cd7fe5368735302 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c625 expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.134691:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678a00/0x1cd7fe5368735302 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c625 expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.134734:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678a00/0x1cd7fe5368735302 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c625 expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619921.134746:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fe000700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589149952:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619921.134816:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc240 going to sleep for 11 seconds 00000100:00100000:2.0:1788619921.137889:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe000700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589149952:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619921.137915:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619921.137982:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2355:x1875503571520384:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3397us (3666us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619921.137997:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 373 00000100:00100000:1.0:1788619921.140695:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619921.141079:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571520512 00000100:00100000:3.0:1788619921.141090:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571520512 00000100:00100000:3.0:1788619921.141167:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 713 00000100:00100000:3.0:1788619921.141177:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+13:2355:x1875503571520512:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:3.0:1788619921.141482:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 994373 -> 994373 00000004:00010000:3.0:1788619921.141487:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619920 -> 1788619920 00000004:00010000:3.0:1788619921.141490:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619919 -> 1788619919 00000004:00010000:3.0:1788619921.141492:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619920 -> 1788619920 00000004:00010000:3.0:1788619921.141494:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1030 -> 1030 00000100:00100000:3.0:1788619921.141581:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+14:2355:x1875503571520512:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 405us (896us total) trans 4294967711 rc 0/0 00000100:00100000:3.0:1788619921.141589:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 713 00000100:00100000:1.0:1788619921.145947:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.146101:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571520640 00000100:00100000:2.0:1788619921.146111:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571520640 00000100:00100000:2.0:1788619921.146174:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 886 00000100:00100000:2.0:1788619921.146184:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571520640:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619921.146201:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619921.146218:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002a87657c/0x1cd7fe5368735309 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c62c expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619921.146275:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fe000700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589150080:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619921.146387:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc240 going to sleep for 11 seconds 00000100:00100000:2.0:1788619921.148609:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe000700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589150080:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619921.148648:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002a87657c/0x1cd7fe5368735309 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c62c expref: 7 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.148673:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002a87657c/0x1cd7fe5368735309 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c62c expref: 7 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.148688:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c7678400, rc 0) 00000100:00100000:2.0:1788619921.148759:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571520640:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2575us (2819us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619921.148768:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 886 00000100:00100000:1.0:1788619921.151272:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.151373:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571520768 00000100:00100000:2.0:1788619921.151382:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571520768 00000100:00100000:2.0:1788619921.151427:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 887 00000100:00100000:2.0:1788619921.151437:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571520768:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619921.151455:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619921.151471:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cc4d367f/0x1cd7fe5368735310 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c633 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619921.151529:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fe000700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589150208:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619921.151699:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc240 going to sleep for 11 seconds 00000100:00100000:2.0:1788619921.154002:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe000700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589150208:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619921.154039:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000cc4d367f/0x1cd7fe5368735310 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c633 expref: 7 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.154062:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc4d367f/0x1cd7fe5368735310 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c633 expref: 7 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.154075:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c7678800, rc 0) 00000100:00100000:2.0:1788619921.154135:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571520768:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2699us (2870us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619921.154143:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 887 00000100:00100000:1.0:1788619921.156796:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.156899:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571521024 00000100:00100000:2.0:1788619921.156918:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571521024 00000100:00100000:2.0:1788619921.156973:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1116 00000100:00100000:2.0:1788619921.156994:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+20:14908:x1875503571521024:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:2.0:1788619921.157061:0:5846:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619921.157093:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679e00/0x1cd7fe53687352d1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5fb expref: 20 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.157113:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619921.157119:0:5846:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788619921.157632:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679400/0x1cd7fe5368735317 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.157648:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07c00/0x1cd7fe53687352df lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c602 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.157656:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679400/0x1cd7fe5368735317 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.157688:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07c00/0x1cd7fe53687352df lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c602 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.157702:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07c00/0x1cd7fe53687352df lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c602 expref: 13 pid: 5845 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619921.157725:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fe000700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589150336:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619921.157850:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc240 going to sleep for 11 seconds 00000100:00100000:0.0:1788619921.162460:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619921.162546:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571521152 00000100:00100000:1.0:1788619921.162556:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571521152 00010000:00010000:1.0:1788619921.162598:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07c00/0x1cd7fe53687352df lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c602 expref: 14 pid: 5845 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619921.162634:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 374 00000100:00100000:1.0:1788619921.162644:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd5f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+14:2354:x1875503571521152:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619921.162658:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619921.162664:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07c00/0x1cd7fe53687352df lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c602 expref: 14 pid: 5845 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.162676:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07c00/0x1cd7fe53687352df lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c602 expref: 14 pid: 5845 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.162700:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619921.162706:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679400/0x1cd7fe5368735317 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.162727:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679400/0x1cd7fe5368735317 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.162736:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679400/0x1cd7fe5368735317 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.162758:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619921.162824:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd5f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+14:2354:x1875503571521152:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 182us (373us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619921.162832:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 374 00010000:00010000:2.0:1788619921.163218:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07c00/0x1cd7fe53687352df lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c602 expref: 13 pid: 5845 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619921.163235:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe000700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589150336:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619921.163256:0:5846:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679400/0x1cd7fe5368735317 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.163264:0:5846:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679400/0x1cd7fe5368735317 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.163270:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679400/0x1cd7fe5368735317 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788619921.163405:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967712 00000004:00010000:2.0:1788619921.163453:0:5846:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 994373 -> 994373 00000004:00010000:2.0:1788619921.163457:0:5846:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619920 -> 1788619921 00000004:00010000:2.0:1788619921.163460:0:5846:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619919 -> 1788619919 00000004:00010000:2.0:1788619921.163462:0:5846:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619920 -> 1788619921 00000004:00010000:2.0:1788619921.163465:0:5846:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1030 -> 1030 00000004:00010000:2.0:1788619921.163472:0:5846:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000005b25309d reply state 0000000086ae40dc transno 4294967713 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679400/0x1cd7fe5368735317 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619921.163481:0:5846:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788619921.163564:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+22:14908:x1875503571521024:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 6583us (6777us total) trans 4294967713 rc 0/0 00000100:00100000:2.0:1788619921.163571:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1116 00000020:00080000:2.0:1788619921.163575:0:5846:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:1.0:1788619921.168534:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619921.168660:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571521280 00000100:00100000:0.0:1788619921.168670:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571521280 00000100:00100000:0.0:1788619921.168735:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1117 00000100:00100000:0.0:1788619921.168763:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+22:14908:x1875503571521280:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619921.168789:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619921.168809:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe536873531e lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c63a expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.168838:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678a00/0x1cd7fe5368735302 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c625 expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.168848:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe536873531e lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c63a expref: 23 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.168883:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678a00/0x1cd7fe5368735302 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c625 expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.168896:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678a00/0x1cd7fe5368735302 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c625 expref: 12 pid: 11397 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619921.168910:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fefd6d80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589150464:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619921.169044:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb0c0 going to sleep for 11 seconds 00010000:00010000:2.0:1788619921.169173:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679400/0x1cd7fe5368735317 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.169182:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679400/0x1cd7fe5368735317 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.169189:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679400/0x1cd7fe5368735317 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.169194:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c7679400 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:2.0:1788619921.169199:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679400/0x1cd7fe5368735317 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.169213:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679400/0x1cd7fe5368735317 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.169229:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619921.169233:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788619921.169237:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679400/0x1cd7fe5368735317 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.169244:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679400/0x1cd7fe5368735317 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619921.173162:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619921.173341:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fefd6d80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589150464:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619921.173376:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe536873531e lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c63a expref: 22 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619921.173395:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619921.173398:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619921.173402:0:5844:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe536873531e lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c63a expref: 22 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619921.173411:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed1400, rc 0) 00000100:00100000:3.0:1788619921.173522:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+22:14908:x1875503571521280:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4760us (4996us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619921.173531:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1117 00000100:00100000:2.0:1788619921.173738:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571521408 00000100:00100000:2.0:1788619921.173745:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571521408 00010000:00010000:2.0:1788619921.173774:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678a00/0x1cd7fe5368735302 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c625 expref: 13 pid: 11397 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619921.173885:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 375 00000100:00100000:2.0:1788619921.173894:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+13:2355:x1875503571521408:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619921.173918:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619921.173944:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678a00/0x1cd7fe5368735302 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c625 expref: 13 pid: 11397 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.173957:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678a00/0x1cd7fe5368735302 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c625 expref: 13 pid: 11397 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.173994:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678a00/0x1cd7fe5368735302 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c625 expref: 13 pid: 11397 timeout: 513 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.174011:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619921.174016:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe536873531e lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c63a expref: 21 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.174025:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe536873531e lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c63a expref: 21 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.174033:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe536873531e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c63a expref: 21 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.174066:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe536873531e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c63a expref: 21 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619921.174080:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fe000700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589150592:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619921.174133:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc240 going to sleep for 11 seconds 00000100:00100000:3.0:1788619921.177265:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619921.177298:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571521536 00000100:00100000:3.0:1788619921.177307:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571521536 00002000:00010000:3.0:1788619921.177374:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1536430->18446744073709551615) 00002000:00100000:3.0:1788619921.177379:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fe062d80 with x1875503571521536 ext(1536430->18446744073709551615) 00002000:00010000:3.0:1788619921.177390:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fe062d80. 00000100:00100000:3.0:1788619921.177415:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 501 00000100:00100000:3.0:1788619921.177425:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe062d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571521536:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:3.0:1788619921.177650:0:6703:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1226248 -> 1536430 00002000:00010000:3.0:1788619921.177653:0:6703:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619920 -> 1788619921 00002000:00010000:3.0:1788619921.177667:0:6703:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619920 -> 1788619919 00002000:00010000:3.0:1788619921.177669:0:6703:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619920 -> 1788619921 00002000:00010000:3.0:1788619921.177671:0:6703:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 775 -> 775 00000100:00100000:3.0:1788619921.177731:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe062d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571521536:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 307us (475us total) trans 4294967446 rc 0/0 00000100:00100000:3.0:1788619921.177738:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 501 00002000:00010000:3.0:1788619921.177742:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1536430->18446744073709551615) 00002000:00100000:3.0:1788619921.177744:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fe062d80 with x1875503571521536 ext(1536430->18446744073709551615) 00002000:00010000:3.0:1788619921.177749:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fe062d80. 00000100:00100000:2.0:1788619921.180892:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe000700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589150592:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619921.180920:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619921.180996:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571521408:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 7101us (7842us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619921.181006:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 375 00000100:00100000:3.0:1788619921.183692:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619921.183806:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571521664 00000100:00100000:1.0:1788619921.183817:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571521664 00002000:00100000:1.0:1788619921.183868:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1511424->1515519) req@ffff8e78fe060e00 x1875503571521664/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:17/0 lens 488/0 e 0 to 0 dl 1788619932 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619921.183880:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fe060e00 with x1875503571521664 ext(1511424->1515519) 00002000:00010000:1.0:1788619921.183886:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fe060e00 00000100:00100000:1.0:1788619921.183919:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 502 00000100:00100000:1.0:1788619921.183929:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe060e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2352:x1875503571521664:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619921.184212:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe060e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2352:x1875503571521664:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 284us (530us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619921.184220:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 502 00002000:00100000:1.0:1788619921.184227:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1511424->1515519) req@ffff8e78fe060e00 x1875503571521664/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:17/0 lens 488/4536 e 0 to 0 dl 1788619932 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619921.184235:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fe060e00 with x1875503571521664 ext(1511424->1515519) 00002000:00010000:1.0:1788619921.184239:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fe060e00 00010000:00010000:1.0:1788619921.186802:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788619921.188271:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.188784:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571521792 00000100:00100000:2.0:1788619921.188794:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571521792 00002000:00100000:2.0:1788619921.188841:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1515520->1519615) req@ffff8e78c75f7480 x1875503571521792/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:17/0 lens 488/0 e 0 to 0 dl 1788619932 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619921.188852:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c75f7480 with x1875503571521792 ext(1515520->1519615) 00002000:00010000:2.0:1788619921.188858:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c75f7480 00000100:00100000:2.0:1788619921.188890:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 503 00000100:00100000:2.0:1788619921.188899:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571521792:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619921.189659:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571521792:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 757us (1391us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619921.189669:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 503 00002000:00100000:2.0:1788619921.189676:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1515520->1519615) req@ffff8e78c75f7480 x1875503571521792/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:17/0 lens 488/4536 e 0 to 0 dl 1788619932 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619921.189685:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c75f7480 with x1875503571521792 ext(1515520->1519615) 00002000:00010000:2.0:1788619921.189689:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c75f7480 00000100:00100000:0.0:1788619921.193813:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619921.193887:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571521920 00000100:00100000:3.0:1788619921.193896:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571521920 00002000:00100000:3.0:1788619921.193945:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1519616->1523711) req@ffff8e78fefd5c00 x1875503571521920/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:17/0 lens 488/0 e 0 to 0 dl 1788619932 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619921.193956:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fefd5c00 with x1875503571521920 ext(1519616->1523711) 00002000:00010000:3.0:1788619921.193961:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fefd5c00 00000100:00100000:3.0:1788619921.194001:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 504 00000100:00100000:3.0:1788619921.194011:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2355:x1875503571521920:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619921.194257:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2355:x1875503571521920:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 246us (451us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619921.194264:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 504 00002000:00100000:3.0:1788619921.194272:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1519616->1523711) req@ffff8e78fefd5c00 x1875503571521920/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:17/0 lens 488/4536 e 0 to 0 dl 1788619932 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619921.194279:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fefd5c00 with x1875503571521920 ext(1519616->1523711) 00002000:00010000:3.0:1788619921.194283:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fefd5c00 00000100:00100000:3.0:1788619921.198789:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.199009:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571522048 00000100:00100000:2.0:1788619921.199020:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571522048 00002000:00100000:2.0:1788619921.199071:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1523712->1527807) req@ffff8e78fe060a80 x1875503571522048/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:17/0 lens 488/0 e 0 to 0 dl 1788619932 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619921.199082:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fe060a80 with x1875503571522048 ext(1523712->1527807) 00002000:00010000:2.0:1788619921.199087:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fe060a80 00000100:00100000:2.0:1788619921.199115:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 505 00000100:00100000:2.0:1788619921.199124:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe060a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571522048:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619921.199420:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe060a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571522048:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 296us (639us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619921.199428:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 505 00002000:00100000:2.0:1788619921.199434:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1523712->1527807) req@ffff8e78fe060a80 x1875503571522048/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:17/0 lens 488/4536 e 0 to 0 dl 1788619932 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619921.199442:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fe060a80 with x1875503571522048 ext(1523712->1527807) 00002000:00010000:2.0:1788619921.199447:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fe060a80 00000100:00100000:2.0:1788619921.203532:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619921.203635:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571522176 00000100:00100000:1.0:1788619921.203646:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571522176 00002000:00100000:1.0:1788619921.203694:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1527808->1531903) req@ffff8e78fe000700 x1875503571522176/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:17/0 lens 488/0 e 0 to 0 dl 1788619932 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619921.203704:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fe000700 with x1875503571522176 ext(1527808->1531903) 00002000:00010000:1.0:1788619921.203719:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fe000700 00000100:00100000:1.0:1788619921.203750:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 506 00000100:00100000:1.0:1788619921.203759:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe000700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2352:x1875503571522176:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619921.204066:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe000700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2352:x1875503571522176:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 306us (542us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619921.204074:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 506 00002000:00100000:1.0:1788619921.204080:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1527808->1531903) req@ffff8e78fe000700 x1875503571522176/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:17/0 lens 488/4536 e 0 to 0 dl 1788619932 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619921.204088:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fe000700 with x1875503571522176 ext(1527808->1531903) 00002000:00010000:1.0:1788619921.204091:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fe000700 00000100:00100000:0.0:1788619921.206996:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619921.207087:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571522304 00000100:00100000:3.0:1788619921.207097:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571522304 00002000:00100000:3.0:1788619921.207147:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1531904->1535999) req@ffff8e78fefd4380 x1875503571522304/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:17/0 lens 488/0 e 0 to 0 dl 1788619932 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619921.207157:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fefd4380 with x1875503571522304 ext(1531904->1535999) 00002000:00010000:3.0:1788619921.207162:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fefd4380 00000100:00100000:3.0:1788619921.207197:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 507 00000100:00100000:3.0:1788619921.207206:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571522304:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619921.207510:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571522304:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 304us (530us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619921.207518:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 507 00002000:00100000:3.0:1788619921.207526:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1531904->1535999) req@ffff8e78fefd4380 x1875503571522304/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:17/0 lens 488/4536 e 0 to 0 dl 1788619932 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619921.207535:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fefd4380 with x1875503571522304 ext(1531904->1535999) 00002000:00010000:3.0:1788619921.207538:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fefd4380 00000100:00100000:3.0:1788619921.211445:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.211556:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571522432 00000100:00100000:2.0:1788619921.211565:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571522432 00002000:00100000:2.0:1788619921.211617:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1536000->1540095) req@ffff8e78fe063480 x1875503571522432/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:17/0 lens 488/0 e 0 to 0 dl 1788619932 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619921.211628:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fe063480 with x1875503571522432 ext(1536000->1540095) 00002000:00010000:2.0:1788619921.211634:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fe063480 00000100:00100000:2.0:1788619921.211667:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 508 00000100:00100000:2.0:1788619921.211678:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe063480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2353:x1875503571522432:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619921.211938:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe063480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2353:x1875503571522432:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 261us (502us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619921.211945:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 508 00002000:00100000:2.0:1788619921.211951:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1536000->1540095) req@ffff8e78fe063480 x1875503571522432/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:17/0 lens 488/4536 e 0 to 0 dl 1788619932 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619921.211959:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fe063480 with x1875503571522432 ext(1536000->1540095) 00002000:00010000:2.0:1788619921.211962:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fe063480 00000100:00100000:2.0:1788619921.215779:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619921.215971:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571522560 00000100:00100000:1.0:1788619921.215991:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571522560 00002000:00100000:1.0:1788619921.216048:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1511424->1536429) req@ffff8e78fe000a80 x1875503571522560/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:17/0 lens 488/0 e 0 to 0 dl 1788619932 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619921.216059:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fe000a80 with x1875503571522560 ext(1511424->1536429) 00002000:00010000:1.0:1788619921.216064:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fe000a80 00000100:00100000:1.0:1788619921.216102:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 509 00000100:00100000:1.0:1788619921.216112:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe000a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571522560:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:1.0:1788619921.219783:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe000a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+11:2354:x1875503571522560:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 3660us (4002us total) trans 4294967447 rc 0/0 00000100:00100000:1.0:1788619921.219798:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 509 00002000:00100000:1.0:1788619921.219811:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1511424->1536429) req@ffff8e78fe000a80 x1875503571522560/t4294967447(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:17/0 lens 488/448 e 0 to 0 dl 1788619932 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619921.219822:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fe000a80 with x1875503571522560 ext(1511424->1536429) 00002000:00010000:1.0:1788619921.219828:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fe000a80 00000100:00100000:2.0:1788619921.223299:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619921.223420:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571522688 00000100:00100000:3.0:1788619921.223431:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571522688 00000100:00100000:3.0:1788619921.223489:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 888 00000100:00100000:3.0:1788619921.223499:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd939180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+11:2355:x1875503571522688:12345-192.168.201.5@tcp:16:fsx.0 00000100:00100000:1.0:1788619921.252876:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd939180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+11:2355:x1875503571522688:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 29368us (29576us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619921.252893:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 888 00000001:00080000:0.0:1788619921.252996:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967447 is committed 00000001:00080000:0.0:1788619921.253019:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967443 is committed 00000100:00100000:0.0:1788619921.256293:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.256381:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571522816 00000100:00100000:2.0:1788619921.256390:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571522816 00000100:00100000:2.0:1788619921.256446:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1118 00000100:00100000:2.0:1788619921.256456:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+12:14908:x1875503571522816:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619921.256479:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619921.256497:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678800/0x1cd7fe5368735325 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c641 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.256589:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678400/0x1cd7fe536873532c lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.256603:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678400/0x1cd7fe536873532c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.256614:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678400/0x1cd7fe536873532c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788619921.256638:0:11397:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1536430 00000004:00200000:2.0:1788619921.256642:0:11397:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788619921.256658:0:11397:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678400/0x1cd7fe536873532c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.256680:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678800/0x1cd7fe5368735325 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c641 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.256699:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678400/0x1cd7fe536873532c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c641 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.256719:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619921.256722:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788619921.256726:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c7678400, rc 0) 00000100:00100000:2.0:1788619921.256919:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+13:14908:x1875503571522816:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 463us (633us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619921.256927:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1118 00000100:00100000:3.0:1788619921.259403:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.259498:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571522944 00000100:00100000:2.0:1788619921.259508:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571522944 00000100:00100000:2.0:1788619921.259566:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1119 00000100:00100000:2.0:1788619921.259577:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe061880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571522944:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619921.259597:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619921.259618:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679c00/0x1cd7fe5368735333 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c648 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.259655:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe536873531e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c63a expref: 21 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619921.259695:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fd93b100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589150720:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619921.259784:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619921.262571:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000004:00010000:2.0:1788619921.262735:0:5846:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619919 -> 1788619921 00000100:00100000:2.0:1788619921.262762:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fd93b100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589150720:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619921.262787:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:2.0:1788619921.262806:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679c00/0x1cd7fe5368735333 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c648 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.262826:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619921.262829:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619921.262835:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679c00/0x1cd7fe5368735333 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c648 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.262849:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c7679c00, rc 0) 00000100:00100000:2.0:1788619921.262908:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe061880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571522944:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3333us (3513us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619921.262918:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1119 00000100:00100000:1.0:1788619921.263096:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571523072 00000100:00100000:1.0:1788619921.263102:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571523072 00000100:00100000:1.0:1788619921.263151:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 889 00000100:00100000:1.0:1788619921.263160:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe060700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571523072:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619921.263174:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619921.263190:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001b18afa8/0x1cd7fe536873533a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c64f expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619921.263244:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c75f5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589150848:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619921.263293:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7540 going to sleep for 11 seconds 00002000:00010000:1.0:1788619921.266134:0:6696:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619919 -> 1788619921 00000100:00100000:1.0:1788619921.266165:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c75f5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589150848:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619921.266199:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001b18afa8/0x1cd7fe536873533a lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c64f expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.266226:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001b18afa8/0x1cd7fe536873533a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c64f expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.266239:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa07200, rc 0) 00000100:00100000:1.0:1788619921.266297:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe060700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571523072:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3137us (3734us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619921.266304:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 889 00000100:00100000:3.0:1788619921.269426:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619921.269520:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571523200 00000100:00100000:0.0:1788619921.269530:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571523200 00000100:00100000:0.0:1788619921.269608:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1120 00000100:00100000:0.0:1788619921.269617:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe062680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571523200:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619921.269635:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619921.269655:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0800/0x1cd7fe5368735341 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c656 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.269692:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe536873531e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c63a expref: 21 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619921.269744:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fefd6d80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589150976:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619921.269828:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb600 going to sleep for 11 seconds 00000100:00100000:3.0:1788619921.273787:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619921.274012:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fefd6d80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589150976:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619921.274037:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:0.0:1788619921.274057:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0800/0x1cd7fe5368735341 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c656 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.274078:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619921.274081:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619921.274087:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0800/0x1cd7fe5368735341 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c656 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.274102:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed0800, rc 0) 00000100:00100000:0.0:1788619921.274165:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe062680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571523200:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4548us (4750us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619921.274174:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1120 00000100:00100000:2.0:1788619921.274309:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571523328 00000100:00100000:2.0:1788619921.274317:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571523328 00000100:00100000:2.0:1788619921.274409:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 890 00000100:00100000:2.0:1788619921.274418:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe062a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571523328:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619921.274432:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619921.274448:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003fefefac/0x1cd7fe5368735348 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c65d expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619921.274496:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fd93b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589151104:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619921.274547:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619921.277273:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fd93b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589151104:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619921.277315:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003fefefac/0x1cd7fe5368735348 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c65d expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.277365:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003fefefac/0x1cd7fe5368735348 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c65d expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.277378:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c7679c00, rc 0) 00000100:00100000:2.0:1788619921.277438:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe062a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571523328:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3020us (3658us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619921.277447:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 890 00000100:00100000:3.0:1788619921.280156:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.280855:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571523456 00000100:00100000:2.0:1788619921.280866:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571523456 00000100:00100000:2.0:1788619921.280942:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 714 00000100:00100000:2.0:1788619921.280952:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe061500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2355:x1875503571523456:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:2.0:1788619921.281626:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 994373 -> 994373 00000004:00010000:2.0:1788619921.281632:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619921 -> 1788619921 00000004:00010000:2.0:1788619921.281634:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619921 -> 1788619919 00000004:00010000:2.0:1788619921.281636:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619921 -> 1788619921 00000004:00010000:2.0:1788619921.281639:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1030 -> 1030 00000100:00100000:2.0:1788619921.281765:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe061500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+23:2355:x1875503571523456:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 812us (1617us total) trans 4294967714 rc 0/0 00000100:00100000:2.0:1788619921.281773:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 714 00000100:00100000:3.0:1788619921.284516:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619921.284594:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571523584 00000100:00100000:1.0:1788619921.284604:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571523584 00000100:00100000:1.0:1788619921.284662:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1121 00000100:00100000:1.0:1788619921.284672:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe063800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+23:14908:x1875503571523584:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619921.284694:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619921.284727:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07200/0x1cd7fe536873534f lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c664 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.284811:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe5368735356 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.284827:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe5368735356 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.284838:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe5368735356 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788619921.284865:0:11397:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1536430 00000004:00200000:1.0:1788619921.284870:0:11397:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788619921.284887:0:11397:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe5368735356 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.284926:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07200/0x1cd7fe536873534f lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c664 expref: 25 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.284943:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe5368735356 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c664 expref: 24 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.284952:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619921.284955:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788619921.284959:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa06800, rc 0) 00000100:00100000:1.0:1788619921.285279:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe063800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+24:14908:x1875503571523584:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 607us (770us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619921.285286:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1121 00000100:00100000:3.0:1788619921.298946:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619921.299003:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619921.299148:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571523840 00000100:00100000:1.0:1788619921.299160:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571523840 00002000:00100000:1.0:1788619921.299208:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6530a80 x1875503571523840/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:17/0 lens 488/0 e 0 to 0 dl 1788619932 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619921.299220:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6530a80 with x1875503571523840 00000100:00100000:1.0:1788619921.299257:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 715 00000100:00100000:1.0:1788619921.299265:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6530a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2352:x1875503571523840:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619921.301185:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571523712 00000100:00100000:0.0:1788619921.301197:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571523712 00002000:00100000:0.0:1788619921.301253:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1511423) req@ffff8e77c6533b80 x1875503571523712/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:17/0 lens 488/0 e 0 to 0 dl 1788619932 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619921.301264:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6533b80 with x1875503571523712 ext(1048576->1511423) 00002000:00010000:0.0:1788619921.301269:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6533b80 00000100:00100000:0.0:1788619921.301300:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 510 00000100:00100000:0.0:1788619921.301309:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6533b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571523712:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619921.316211:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6530a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2352:x1875503571523840:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 16934us (17195us total) trans 0 rc 995328/995328 00000100:00100000:1.0:1788619921.316227:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 715 00002000:00100000:1.0:1788619921.316240:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6530a80 x1875503571523840/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:17/0 lens 488/440 e 0 to 0 dl 1788619932 ref 1 fl Complete:/600/0 rc 995328/995328 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619921.316250:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6530a80 with x1875503571523840 00000100:00100000:0.0:1788619921.318374:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6533b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571523712:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 17055us (19428us total) trans 0 rc 462848/462848 00000100:00100000:0.0:1788619921.318387:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 510 00002000:00100000:0.0:1788619921.318399:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1511423) req@ffff8e77c6533b80 x1875503571523712/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:17/0 lens 488/440 e 0 to 0 dl 1788619932 ref 1 fl Complete:/600/0 rc 462848/462848 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619921.318410:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6533b80 with x1875503571523712 ext(1048576->1511423) 00002000:00010000:0.0:1788619921.318415:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6533b80 00000100:00100000:3.0:1788619921.323094:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619921.323147:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619921.323230:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571524096 00000100:00100000:1.0:1788619921.323242:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571524096 00000100:00100000:1.0:1788619921.323300:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 891 00000100:00100000:1.0:1788619921.323310:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe138050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571524096:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619921.323348:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619921.323367:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001b18afa8/0x1cd7fe536873535d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c672 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619921.323425:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c75f5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589151232:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619921.323511:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7540 going to sleep for 11 seconds 00000100:00100000:2.0:1788619921.323781:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571523968 00000100:00100000:2.0:1788619921.323786:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571523968 00000100:00100000:2.0:1788619921.323828:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1122 00000100:00100000:2.0:1788619921.323837:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6530000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+13:2353:x1875503571523968:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619921.323848:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619921.323863:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7200/0x1cd7fe5368735364 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c66b expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.323892:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe536873531e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c63a expref: 23 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619921.323917:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fd93b100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589151360:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619921.323960:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc840 going to sleep for 11 seconds 00000100:00100000:1.0:1788619921.326418:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c75f5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589151232:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619921.326455:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001b18afa8/0x1cd7fe536873535d lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c672 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.326481:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001b18afa8/0x1cd7fe536873535d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c672 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.326494:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa07200, rc 0) 00000100:00100000:1.0:1788619921.326553:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe138050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571524096:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3243us (3404us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619921.326561:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 891 00000004:00010000:2.0:1788619921.327717:0:5846:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619919 -> 1788619921 00000100:00100000:2.0:1788619921.327743:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fd93b100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589151360:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619921.327762:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:2.0:1788619921.327778:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7200/0x1cd7fe5368735364 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c66b expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.327800:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619921.327803:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619921.327808:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7200/0x1cd7fe5368735364 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c66b expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.327820:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7200, rc 0) 00000100:00100000:2.0:1788619921.327922:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6530000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+13:2353:x1875503571523968:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4085us (4837us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619921.327929:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1122 00000100:00100000:0.0:1788619921.333246:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619921.333320:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619921.333472:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571524352 00000100:00100000:1.0:1788619921.333482:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571524352 00000100:00100000:1.0:1788619921.333622:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 892 00000100:00100000:1.0:1788619921.333633:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571524352:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619921.333652:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619921.333671:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006c1b75d6/0x1cd7fe536873536b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c680 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619921.333742:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c75f5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589151488:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619921.333825:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7540 going to sleep for 11 seconds 00000100:00100000:3.0:1788619921.335991:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571524224 00000100:00100000:3.0:1788619921.336005:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571524224 00000100:00100000:3.0:1788619921.336067:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1123 00000100:00100000:3.0:1788619921.336077:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+13:2353:x1875503571524224:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619921.336096:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619921.336115:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe5368735372 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c679 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619921.336153:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe536873531e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c63a expref: 23 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619921.336193:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6531500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589151616:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619921.336266:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c240 going to sleep for 11 seconds 00000100:00100000:1.0:1788619921.339070:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c75f5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589151488:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619921.339112:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000006c1b75d6/0x1cd7fe536873536b lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c680 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.339146:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006c1b75d6/0x1cd7fe536873536b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c680 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.339161:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa07e00, rc 0) 00000100:00100000:1.0:1788619921.339230:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571524352:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5599us (5993us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619921.339238:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 892 00000100:00100000:3.0:1788619921.342013:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6531500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589151616:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619921.342041:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:3.0:1788619921.342060:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe5368735372 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c679 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619921.342080:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619921.342082:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619921.342087:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe5368735372 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c679 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619921.342122:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683da00, rc 0) 00000100:00100000:3.0:1788619921.342349:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+13:2353:x1875503571524224:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6249us (9004us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619921.342359:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1123 00000100:00100000:0.0:1788619921.346424:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619921.346470:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.346626:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571524480 00000100:00100000:2.0:1788619921.346637:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571524480 00000100:00100000:2.0:1788619921.346701:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1124 00000100:00100000:2.0:1788619921.346724:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8595880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571524480:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619921.346746:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619921.346765:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe5368735379 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c687 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.346799:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe536873531e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c63a expref: 23 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619921.346839:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fd93b100 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589151744:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619921.347122:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc840 going to sleep for 11 seconds 00000100:00100000:3.0:1788619921.347205:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571524608 00000100:00100000:3.0:1788619921.347210:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571524608 00000100:00100000:3.0:1788619921.347247:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 893 00000100:00100000:3.0:1788619921.347255:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571524608:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619921.347265:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619921.347278:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f980cda4/0x1cd7fe5368735380 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c68e expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619921.347313:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6531500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589151872:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619921.347359:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c240 going to sleep for 11 seconds 00000100:00100000:1.0:1788619921.350142:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fd93b100 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589151744:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619921.350167:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:1.0:1788619921.350185:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe5368735379 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c687 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.350204:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619921.350207:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619921.350213:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe5368735379 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c687 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.350227:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7c00, rc 0) 00000100:00100000:1.0:1788619921.350284:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8595880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571524480:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3572us (3810us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619921.350292:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1124 00000100:00100000:1.0:1788619921.351817:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6531500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589151872:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619921.351849:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f980cda4/0x1cd7fe5368735380 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c68e expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.351876:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f980cda4/0x1cd7fe5368735380 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c68e expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.351888:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d200, rc 0) 00000100:00100000:1.0:1788619921.352109:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571524608:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4853us (5693us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619921.352118:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 893 00000100:00100000:0.0:1788619921.355398:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619921.355435:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619921.355505:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571524736 00000100:00100000:1.0:1788619921.355515:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571524736 00000100:00100000:1.0:1788619921.355580:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1125 00000100:00100000:1.0:1788619921.355589:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8594e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571524736:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619921.355609:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619921.355629:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07800/0x1cd7fe5368735387 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c695 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.355662:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe536873531e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c63a expref: 23 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619921.355697:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c75f5c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589152000:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619921.355777:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7540 going to sleep for 11 seconds 00000100:00100000:3.0:1788619921.355887:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571524864 00000100:00100000:3.0:1788619921.355892:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571524864 00000100:00100000:3.0:1788619921.355937:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 894 00000100:00100000:3.0:1788619921.355945:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8595c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571524864:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619921.355953:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619921.355963:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe536873538e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c69c expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619921.356169:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6530e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589152128:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619921.356198:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c240 going to sleep for 11 seconds 00000100:00100000:3.0:1788619921.358818:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6530e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589152128:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619921.358857:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe536873538e lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c69c expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619921.358881:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe536873538e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c69c expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619921.358893:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683dc00, rc 0) 00000100:00100000:3.0:1788619921.358957:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8595c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571524864:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3012us (3568us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619921.358966:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 894 00000100:00100000:1.0:1788619921.360722:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c75f5c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589152000:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619921.360747:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:1.0:1788619921.360765:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07800/0x1cd7fe5368735387 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c695 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.360789:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619921.360792:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619921.360798:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07800/0x1cd7fe5368735387 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c695 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.360810:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa07800, rc 0) 00000100:00100000:1.0:1788619921.360950:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8594e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571524736:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5362us (5513us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619921.360959:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1125 00000100:00100000:0.0:1788619921.364660:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.364765:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571524992 00000100:00100000:2.0:1788619921.364774:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571524992 00000100:00100000:2.0:1788619921.364874:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 716 00000100:00100000:2.0:1788619921.364886:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8596a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2355:x1875503571524992:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:2.0:1788619921.365169:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 994373 -> 994373 00000004:00010000:2.0:1788619921.365173:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619921 -> 1788619921 00000004:00010000:2.0:1788619921.365176:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619921 -> 1788619919 00000004:00010000:2.0:1788619921.365179:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619921 -> 1788619921 00000004:00010000:2.0:1788619921.365181:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1030 -> 1030 00000100:00100000:2.0:1788619921.365252:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8596a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+25:2355:x1875503571524992:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 369us (602us total) trans 4294967715 rc 0/0 00000100:00100000:2.0:1788619921.365260:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 716 00000100:00100000:0.0:1788619921.368786:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619921.368860:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619921.369210:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571525248 00000100:00100000:1.0:1788619921.369219:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571525248 00000100:00100000:1.0:1788619921.369278:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 895 00000100:00100000:1.0:1788619921.369290:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8595f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571525248:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619921.369307:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619921.369323:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006c1b75d6/0x1cd7fe5368735395 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c6aa expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619921.369423:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c75f5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589152256:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619921.369502:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7540 going to sleep for 11 seconds 00000100:00100000:3.0:1788619921.371489:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571525120 00000100:00100000:3.0:1788619921.371503:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571525120 00000100:00100000:1.0:1788619921.371911:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c75f5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589152256:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619921.371944:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000006c1b75d6/0x1cd7fe5368735395 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c6aa expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.371968:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006c1b75d6/0x1cd7fe5368735395 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c6aa expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.371992:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa07e00, rc 0) 00000100:00100000:1.0:1788619921.372049:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8595f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571525248:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2761us (3273us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619921.372057:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 895 00000100:00100000:2.0:1788619921.373721:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1126 00000100:00100000:2.0:1788619921.373734:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8594700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+13:2355:x1875503571525120:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619921.373754:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619921.373775:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6800/0x1cd7fe536873539c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c6a3 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.373812:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe536873531e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c63a expref: 24 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619921.373855:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fd939880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589152384:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619921.373948:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc240 going to sleep for 11 seconds 00000004:00010000:1.0:1788619921.376295:0:5844:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619919 -> 1788619921 00000100:00100000:1.0:1788619921.376365:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fd939880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589152384:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619921.376390:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:1.0:1788619921.376409:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6800/0x1cd7fe536873539c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c6a3 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.376428:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619921.376430:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619921.376436:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6800/0x1cd7fe536873539c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c6a3 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.376450:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6800, rc 0) 00000100:00100000:1.0:1788619921.376506:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8594700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+13:2355:x1875503571525120:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2774us (7644us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619921.376515:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1126 00000100:00100000:1.0:1788619921.379152:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619921.379305:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571525376 00000100:00100000:3.0:1788619921.379314:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571525376 00000100:00100000:3.0:1788619921.379389:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1127 00000100:00100000:3.0:1788619921.379399:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571525376:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619921.379420:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619921.379439:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe53687353a3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c6b1 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619921.379475:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe536873531e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c63a expref: 24 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619921.379516:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6530e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589152512:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619921.379607:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c240 going to sleep for 11 seconds 00000100:00100000:0.0:1788619921.382004:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619921.382105:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571525504 00000100:00100000:1.0:1788619921.382113:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571525504 00000100:00100000:1.0:1788619921.382168:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 896 00000100:00100000:1.0:1788619921.382179:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8596300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571525504:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619921.382197:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619921.382216:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000034a58768/0x1cd7fe53687353aa lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c6b8 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619921.382270:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c75f5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589152640:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619921.382364:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7540 going to sleep for 11 seconds 00000100:00100000:3.0:1788619921.383317:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6530e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589152512:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619921.383366:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:3.0:1788619921.383382:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe53687353a3 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c6b1 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619921.383402:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619921.383405:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619921.383410:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe53687353a3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c6b1 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619921.383424:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683dc00, rc 0) 00000100:00100000:3.0:1788619921.383469:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571525376:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4071us (4326us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619921.383476:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1127 00000100:00100000:1.0:1788619921.389305:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c75f5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589152640:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619921.389372:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000034a58768/0x1cd7fe53687353aa lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c6b8 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.389398:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000034a58768/0x1cd7fe53687353aa lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c6b8 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.389411:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa06600, rc 0) 00000100:00100000:1.0:1788619921.389462:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8596300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571525504:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7285us (7467us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619921.389472:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 896 00000100:00100000:1.0:1788619921.394072:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.394208:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571525632 00000100:00100000:2.0:1788619921.394218:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571525632 00002000:00100000:2.0:1788619921.394268:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c75f5180 x1875503571525632/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:17/0 lens 488/0 e 0 to 0 dl 1788619932 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619921.394278:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c75f5180 with x1875503571525632 00000100:00100000:2.0:1788619921.394314:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 717 00000100:00100000:2.0:1788619921.394362:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+25:2353:x1875503571525632:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619921.399039:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+25:2353:x1875503571525632:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 4706us (4967us total) trans 0 rc 69632/69632 00000100:00100000:2.0:1788619921.399054:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 717 00002000:00100000:2.0:1788619921.399066:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c75f5180 x1875503571525632/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:17/0 lens 488/440 e 0 to 0 dl 1788619932 ref 1 fl Complete:/600/0 rc 69632/69632 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619921.399076:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c75f5180 with x1875503571525632 00000100:00100000:0.0:1788619921.406051:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619921.406152:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571525760 00000100:00100000:1.0:1788619921.406163:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571525760 00000100:00100000:1.0:1788619921.406221:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1128 00000100:00100000:1.0:1788619921.406231:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8597b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+13:2353:x1875503571525760:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619921.406251:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619921.406272:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06600/0x1cd7fe53687353b1 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c6bf expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.406305:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe536873531e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c63a expref: 24 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619921.406382:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c75f7100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589152768:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619921.406457:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7540 going to sleep for 11 seconds 00000100:00100000:0.0:1788619921.408720:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619921.408908:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c75f7100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589152768:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619921.408930:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:1.0:1788619921.408948:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06600/0x1cd7fe53687353b1 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c6bf expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.408968:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619921.408970:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619921.408976:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06600/0x1cd7fe53687353b1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c6bf expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.409003:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa06600, rc 0) 00000100:00100000:1.0:1788619921.409060:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8597b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+13:2353:x1875503571525760:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2830us (3019us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619921.409068:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1128 00000100:00100000:2.0:1788619921.409313:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571525888 00000100:00100000:2.0:1788619921.409320:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571525888 00000100:00100000:2.0:1788619921.409402:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 897 00000100:00100000:2.0:1788619921.409411:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8595180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571525888:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619921.409424:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619921.409441:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003a2d9d44/0x1cd7fe53687353b8 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c6c6 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619921.409493:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fd939c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589152896:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619921.409567:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccc00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619921.412839:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fd939c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589152896:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619921.412879:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003a2d9d44/0x1cd7fe53687353b8 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c6c6 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.412902:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a2d9d44/0x1cd7fe53687353b8 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c6c6 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.412921:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7600, rc 0) 00000100:00100000:2.0:1788619921.412988:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8595180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571525888:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3577us (4285us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619921.412997:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 897 00000100:00100000:3.0:1788619921.416092:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619921.416214:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571526016 00000100:00100000:1.0:1788619921.416225:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571526016 00000100:00100000:1.0:1788619921.416285:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1129 00000100:00100000:1.0:1788619921.416296:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6530e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571526016:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619921.416315:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619921.416364:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06600/0x1cd7fe53687353bf lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c6cd expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.416400:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe536873531e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c63a expref: 24 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619921.416439:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c75f7100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589153024:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619921.416511:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7540 going to sleep for 11 seconds 00000100:00100000:0.0:1788619921.419090:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.419206:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571526144 00000100:00100000:2.0:1788619921.419215:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571526144 00000100:00100000:2.0:1788619921.419275:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 898 00000100:00100000:2.0:1788619921.419285:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8596680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571526144:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619921.419302:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619921.419319:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fb640cb5/0x1cd7fe53687353c6 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c6d4 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619921.419415:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fd939c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589153152:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619921.419488:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccc00 going to sleep for 11 seconds 00000100:00100000:1.0:1788619921.419650:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c75f7100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589153024:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619921.419670:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:1.0:1788619921.419684:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06600/0x1cd7fe53687353bf lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c6cd expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.419701:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619921.419704:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619921.419737:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06600/0x1cd7fe53687353bf lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c6cd expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.419749:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa06600, rc 0) 00000100:00100000:1.0:1788619921.419778:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6530e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571526016:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3485us (3695us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619921.419784:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1129 00000100:00100000:3.0:1788619921.422523:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fd939c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589153152:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619921.422570:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000fb640cb5/0x1cd7fe53687353c6 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c6d4 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619921.422594:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fb640cb5/0x1cd7fe53687353c6 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c6d4 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619921.422606:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6e00, rc 0) 00000100:00100000:3.0:1788619921.422663:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8596680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571526144:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3380us (3582us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619921.422672:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 898 00000100:00100000:0.0:1788619921.430142:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619921.430251:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571526272 00000100:00100000:1.0:1788619921.430263:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571526272 00002000:00100000:1.0:1788619921.430311:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1228800->1265663) req@ffff8e78c8597480 x1875503571526272/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:17/0 lens 488/0 e 0 to 0 dl 1788619932 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619921.430322:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c8597480 with x1875503571526272 ext(1228800->1265663) 00002000:00010000:1.0:1788619921.430364:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c8597480 00000100:00100000:1.0:1788619921.430397:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 511 00000100:00100000:1.0:1788619921.430407:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8597480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571526272:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:1.0:1788619921.433883:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8597480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571526272:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 3468us (3740us total) trans 4294967448 rc 0/0 00000100:00100000:1.0:1788619921.433896:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 511 00002000:00100000:1.0:1788619921.433908:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1228800->1265663) req@ffff8e78c8597480 x1875503571526272/t4294967448(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:17/0 lens 488/448 e 0 to 0 dl 1788619932 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619921.433918:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c8597480 with x1875503571526272 ext(1228800->1265663) 00002000:00010000:1.0:1788619921.433925:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c8597480 00000100:00100000:0.0:1788619921.437433:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619921.437637:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571526400 00000100:00100000:1.0:1788619921.437648:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571526400 00000100:00100000:1.0:1788619921.437726:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 899 00000100:00100000:1.0:1788619921.437737:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8597100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571526400:12345-192.168.201.5@tcp:16:fsx.0 00000001:00080000:0.0:1788619921.473890:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967448 is committed 00000100:00100000:3.0:1788619921.474110:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8597100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571526400:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 36368us (36681us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619921.474121:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 899 00000100:00100000:0.0:1788619921.477070:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619921.477147:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619921.477219:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571526528 00000100:00100000:1.0:1788619921.477229:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571526528 00000100:00100000:1.0:1788619921.477285:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1130 00000100:00100000:1.0:1788619921.477296:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5025c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+13:2355:x1875503571526528:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619921.477317:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619921.477361:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06600/0x1cd7fe53687353cd lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c6db expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.477400:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe536873531e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c63a expref: 24 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619921.477439:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c75f7100 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589153280:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619921.477512:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7540 going to sleep for 11 seconds 00000100:00100000:3.0:1788619921.479291:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571526656 00000100:00100000:3.0:1788619921.479304:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571526656 00000100:00100000:0.0:1788619921.479431:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 900 00000100:00100000:0.0:1788619921.479441:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8597800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571526656:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619921.479456:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619921.479471:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090281211/0x1cd7fe53687353d4 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c6e2 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619921.479519:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5024e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589153408:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619921.479757:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeba80 going to sleep for 11 seconds 00000100:00100000:1.0:1788619921.481756:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5024e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589153408:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619921.481796:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000090281211/0x1cd7fe53687353d4 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c6e2 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.481823:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090281211/0x1cd7fe53687353d4 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c6e2 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.481836:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed0800, rc 0) 00000100:00100000:1.0:1788619921.481894:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8597800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571526656:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2453us (4832us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619921.481902:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 900 00000100:00100000:2.0:1788619921.483219:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c75f7100 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589153280:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619921.483241:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:2.0:1788619921.483255:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06600/0x1cd7fe53687353cd lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c6db expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.483274:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619921.483277:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619921.483281:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06600/0x1cd7fe53687353cd lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c6db expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.483293:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa06600, rc 0) 00000100:00100000:2.0:1788619921.483466:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5025c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+13:2355:x1875503571526528:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6171us (6317us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619921.483474:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1130 00000100:00100000:0.0:1788619921.486681:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619921.486763:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.486828:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571526912 00000100:00100000:2.0:1788619921.486838:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571526912 00000100:00100000:2.0:1788619921.486895:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 901 00000100:00100000:2.0:1788619921.486919:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5026d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571526912:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619921.486937:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619921.486954:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1c87370/0x1cd7fe53687353db lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c6f0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619921.487023:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fd93aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589153536:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619921.487102:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78ce7f3540 going to sleep for 11 seconds 00000100:00100000:2.0:1788619921.489461:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fd93aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589153536:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619921.489498:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b1c87370/0x1cd7fe53687353db lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c6f0 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.489524:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1c87370/0x1cd7fe53687353db lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c6f0 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.489549:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6200, rc 0) 00000100:00100000:2.0:1788619921.489604:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5026d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571526912:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2700us (2839us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619921.489612:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 901 00000100:00100000:1.0:1788619921.492715:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571526784 00000100:00100000:1.0:1788619921.492729:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571526784 00000100:00100000:0.0:1788619921.493063:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1131 00000100:00100000:0.0:1788619921.493075:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5025500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571526784:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619921.493095:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619921.493116:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1200/0x1cd7fe53687353e2 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c6e9 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.493150:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe536873531e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c63a expref: 24 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619921.493192:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5025f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589153664:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619921.493350:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb840 going to sleep for 11 seconds 00000100:00100000:2.0:1788619921.496165:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5025f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589153664:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619921.496192:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:2.0:1788619921.496211:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1200/0x1cd7fe53687353e2 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c6e9 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.496230:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619921.496233:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619921.496239:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1200/0x1cd7fe53687353e2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c6e9 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.496254:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed1200, rc 0) 00000100:00100000:2.0:1788619921.496444:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5025500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571526784:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3369us (9769us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619921.496454:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1131 00000100:00100000:2.0:1788619921.499662:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.499752:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619921.499819:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571527168 00000100:00100000:3.0:1788619921.499828:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571527168 00000100:00100000:3.0:1788619921.499885:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 902 00000100:00100000:3.0:1788619921.499895:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd93a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571527168:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619921.499923:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619921.499941:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe53687353e9 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c6fe expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619921.500008:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6533800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589153792:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619921.500097:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c600 going to sleep for 11 seconds 00000100:00100000:1.0:1788619921.501234:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571527040 00000100:00100000:1.0:1788619921.501242:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571527040 00000100:00100000:1.0:1788619921.501285:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1132 00000100:00100000:1.0:1788619921.501293:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd93aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+13:2353:x1875503571527040:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619921.501307:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619921.501343:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe53687353f0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c6f7 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.501376:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe536873531e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c63a expref: 24 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619921.501407:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c75f5880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589153920:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619921.501476:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7b40 going to sleep for 11 seconds 00000100:00100000:3.0:1788619921.504377:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6533800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589153792:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619921.504420:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe53687353e9 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c6fe expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619921.504445:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe53687353e9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c6fe expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619921.504457:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683dc00, rc 0) 00000100:00100000:3.0:1788619921.504507:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd93a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571527168:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4613us (4754us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619921.504516:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 902 00000100:00100000:1.0:1788619921.505883:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c75f5880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589153920:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619921.506217:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:1.0:1788619921.506235:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe53687353f0 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c6f7 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.506253:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619921.506256:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619921.506261:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe53687353f0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c6f7 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.506273:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa07e00, rc 0) 00000100:00100000:1.0:1788619921.506517:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd93aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+13:2353:x1875503571527040:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5223us (6862us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619921.506525:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1132 00000100:00100000:0.0:1788619921.509656:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619921.509714:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571527296 00000100:00100000:1.0:1788619921.509722:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571527296 00000100:00100000:1.0:1788619921.509774:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1133 00000100:00100000:1.0:1788619921.509783:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5026680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+13:14908:x1875503571527296:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619921.509802:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619921.509821:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07400/0x1cd7fe53687353f7 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c705 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.509847:0:11397:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe536873531e lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c63a expref: 24 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.509858:0:11397:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07400/0x1cd7fe53687353f7 lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c705 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.509890:0:11397:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe536873531e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c63a expref: 24 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.509919:0:11397:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe536873531e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c63a expref: 24 pid: 5844 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619921.509933:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c75f5880 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589154048:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619921.510366:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7b40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619921.515481:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619921.515721:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c75f5880 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589154048:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619921.515755:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07400/0x1cd7fe53687353f7 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c705 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.515775:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619921.515779:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619921.515783:0:11397:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07400/0x1cd7fe53687353f7 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c705 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.515793:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa07400, rc 0) 00000100:00100000:1.0:1788619921.515898:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5026680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+14:14908:x1875503571527296:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6115us (6249us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619921.515916:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1133 00000100:00100000:3.0:1788619921.516145:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571527424 00000100:00100000:3.0:1788619921.516152:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571527424 00010000:00010000:3.0:1788619921.516184:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe536873531e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c63a expref: 25 pid: 5844 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619921.516219:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 376 00000100:00100000:3.0:1788619921.516228:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8558850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+25:2355:x1875503571527424:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619921.516241:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619921.516268:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe536873531e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c63a expref: 25 pid: 5844 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619921.516279:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe536873531e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c63a expref: 25 pid: 5844 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619921.516302:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe536873531e lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c63a expref: 25 pid: 5844 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619921.516316:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619921.516321:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07400/0x1cd7fe53687353f7 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c705 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619921.516366:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07400/0x1cd7fe53687353f7 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c705 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619921.516375:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07400/0x1cd7fe53687353f7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c705 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619921.516408:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07400/0x1cd7fe53687353f7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c705 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619921.516436:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6533800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589154176:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619921.516489:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c600 going to sleep for 11 seconds 00000100:00100000:2.0:1788619921.519840:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619921.519937:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571527552 00000100:00100000:1.0:1788619921.519947:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571527552 00002000:00100000:1.0:1788619921.520000:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fd93b800 x1875503571527552/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:17/0 lens 488/0 e 0 to 0 dl 1788619932 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619921.520011:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fd93b800 with x1875503571527552 00000100:00100000:1.0:1788619921.520052:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 718 00000100:00100000:1.0:1788619921.520061:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd93b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+14:2354:x1875503571527552:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619921.520268:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd93b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+14:2354:x1875503571527552:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 207us (436us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619921.520276:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 718 00002000:00100000:1.0:1788619921.520283:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fd93b800 x1875503571527552/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:17/0 lens 488/4536 e 0 to 0 dl 1788619932 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619921.520290:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fd93b800 with x1875503571527552 00000100:00100000:3.0:1788619921.523054:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6533800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589154176:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619921.523080:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619921.523146:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8558850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2355:x1875503571527424:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 6918us (7674us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619921.523158:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 376 00000100:00100000:0.0:1788619921.523308:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619921.523494:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571527680 00000100:00100000:3.0:1788619921.523502:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571527680 00002000:00100000:3.0:1788619921.523550:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5025880 x1875503571527680/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:17/0 lens 488/0 e 0 to 0 dl 1788619932 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619921.523560:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5025880 with x1875503571527680 00000100:00100000:3.0:1788619921.523583:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 719 00000100:00100000:3.0:1788619921.523591:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5025880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+14:2352:x1875503571527680:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619921.523802:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5025880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+14:2352:x1875503571527680:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 211us (495us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619921.523809:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 719 00002000:00100000:3.0:1788619921.523815:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5025880 x1875503571527680/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:17/0 lens 488/4536 e 0 to 0 dl 1788619932 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619921.523822:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5025880 with x1875503571527680 00000100:00100000:0.0:1788619921.526595:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.526682:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571527808 00000100:00100000:2.0:1788619921.526690:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571527808 00002000:00100000:2.0:1788619921.526741:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5027480 x1875503571527808/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:17/0 lens 488/0 e 0 to 0 dl 1788619932 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619921.526753:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5027480 with x1875503571527808 00000100:00100000:2.0:1788619921.526784:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 720 00000100:00100000:2.0:1788619921.526793:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5027480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+14:2354:x1875503571527808:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619921.526970:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5027480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+14:2354:x1875503571527808:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 178us (382us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619921.526984:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 720 00002000:00100000:2.0:1788619921.526991:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5027480 x1875503571527808/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:17/0 lens 488/4536 e 0 to 0 dl 1788619932 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619921.526999:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5027480 with x1875503571527808 00000100:00100000:0.0:1788619921.529294:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.529777:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571527936 00000100:00100000:2.0:1788619921.529787:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571527936 00002000:00100000:2.0:1788619921.529836:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5026300 x1875503571527936/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:17/0 lens 488/0 e 0 to 0 dl 1788619932 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619921.529847:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5026300 with x1875503571527936 00000100:00100000:2.0:1788619921.529882:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 721 00000100:00100000:2.0:1788619921.529891:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5026300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+14:2352:x1875503571527936:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619921.530128:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5026300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+14:2352:x1875503571527936:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 237us (842us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619921.530135:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 721 00002000:00100000:2.0:1788619921.530141:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5026300 x1875503571527936/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:17/0 lens 488/4536 e 0 to 0 dl 1788619932 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619921.530149:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5026300 with x1875503571527936 00000100:00100000:1.0:1788619921.533502:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619921.533629:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571528064 00000100:00100000:3.0:1788619921.533639:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571528064 00002000:00100000:3.0:1788619921.533685:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c75f5880 x1875503571528064/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:17/0 lens 488/0 e 0 to 0 dl 1788619932 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619921.533724:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c75f5880 with x1875503571528064 00000100:00100000:3.0:1788619921.533763:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 722 00000100:00100000:3.0:1788619921.533773:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+14:2355:x1875503571528064:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619921.533984:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+14:2355:x1875503571528064:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 200us (479us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619921.533992:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 722 00002000:00100000:3.0:1788619921.533999:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c75f5880 x1875503571528064/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:17/0 lens 488/4536 e 0 to 0 dl 1788619932 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619921.534006:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c75f5880 with x1875503571528064 00000100:00100000:0.0:1788619921.538292:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619921.538407:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571528192 00000100:00100000:3.0:1788619921.538417:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571528192 00002000:00100000:3.0:1788619921.538466:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5024000 x1875503571528192/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:17/0 lens 488/0 e 0 to 0 dl 1788619932 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619921.538477:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5024000 with x1875503571528192 00000100:00100000:3.0:1788619921.538516:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 723 00000100:00100000:3.0:1788619921.538526:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5024000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+14:2353:x1875503571528192:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619921.538753:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5024000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+14:2353:x1875503571528192:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 227us (470us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619921.538761:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 723 00002000:00100000:3.0:1788619921.538767:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5024000 x1875503571528192/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:17/0 lens 488/4536 e 0 to 0 dl 1788619932 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619921.538775:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5024000 with x1875503571528192 00000100:00100000:0.0:1788619921.542419:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619921.542506:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571528320 00000100:00100000:1.0:1788619921.542515:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571528320 00002000:00100000:1.0:1788619921.542564:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5024380 x1875503571528320/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:17/0 lens 488/0 e 0 to 0 dl 1788619932 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619921.542576:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5024380 with x1875503571528320 00000100:00100000:1.0:1788619921.542612:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 724 00000100:00100000:1.0:1788619921.542621:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5024380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+14:2355:x1875503571528320:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619921.542836:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5024380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+14:2355:x1875503571528320:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 215us (426us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619921.542844:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 724 00002000:00100000:1.0:1788619921.542851:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5024380 x1875503571528320/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:17/0 lens 488/4536 e 0 to 0 dl 1788619932 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619921.542860:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5024380 with x1875503571528320 00000100:00100000:1.0:1788619921.545804:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.545895:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571528448 00000100:00100000:2.0:1788619921.545905:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571528448 00002000:00100000:2.0:1788619921.545956:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c75f5f80 x1875503571528448/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:17/0 lens 488/0 e 0 to 0 dl 1788619932 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619921.545967:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c75f5f80 with x1875503571528448 00000100:00100000:2.0:1788619921.546009:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 725 00000100:00100000:2.0:1788619921.546018:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+14:2355:x1875503571528448:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619921.546209:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+14:2355:x1875503571528448:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 191us (412us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619921.546221:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 725 00002000:00100000:2.0:1788619921.546228:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c75f5f80 x1875503571528448/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:17/0 lens 488/4536 e 0 to 0 dl 1788619932 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619921.546235:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c75f5f80 with x1875503571528448 00000100:00100000:0.0:1788619921.549431:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619921.549512:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571528576 00000100:00100000:3.0:1788619921.549544:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571528576 00002000:00100000:3.0:1788619921.549593:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5024700 x1875503571528576/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:17/0 lens 488/0 e 0 to 0 dl 1788619932 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619921.549603:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5024700 with x1875503571528576 00000100:00100000:3.0:1788619921.549640:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 726 00000100:00100000:3.0:1788619921.549649:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5024700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+14:2354:x1875503571528576:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619921.549876:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5024700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+14:2354:x1875503571528576:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 226us (454us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619921.549883:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 726 00002000:00100000:3.0:1788619921.549891:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5024700 x1875503571528576/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:17/0 lens 488/4536 e 0 to 0 dl 1788619932 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619921.549898:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5024700 with x1875503571528576 00000100:00100000:0.0:1788619921.552847:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619921.553141:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571528704 00000100:00100000:1.0:1788619921.553150:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571528704 00002000:00100000:1.0:1788619921.553195:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5027b80 x1875503571528704/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:17/0 lens 488/0 e 0 to 0 dl 1788619932 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619921.553213:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5027b80 with x1875503571528704 00000100:00100000:1.0:1788619921.553282:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 727 00000100:00100000:1.0:1788619921.553291:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5027b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+14:2355:x1875503571528704:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619921.553521:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5027b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+14:2355:x1875503571528704:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 230us (681us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619921.553541:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 727 00002000:00100000:1.0:1788619921.553549:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5027b80 x1875503571528704/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:17/0 lens 488/4536 e 0 to 0 dl 1788619932 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619921.553557:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5027b80 with x1875503571528704 00000100:00100000:2.0:1788619921.557873:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619921.557951:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571528832 00000100:00100000:3.0:1788619921.557960:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571528832 00002000:00100000:3.0:1788619921.558078:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fd93bb80 x1875503571528832/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:17/0 lens 488/0 e 0 to 0 dl 1788619932 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619921.558089:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fd93bb80 with x1875503571528832 00000100:00100000:3.0:1788619921.558158:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 728 00000100:00100000:3.0:1788619921.558168:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd93bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+14:2354:x1875503571528832:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619921.558406:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd93bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+14:2354:x1875503571528832:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 239us (542us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619921.558413:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 728 00002000:00100000:3.0:1788619921.558420:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fd93bb80 x1875503571528832/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:17/0 lens 488/4536 e 0 to 0 dl 1788619932 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619921.558428:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fd93bb80 with x1875503571528832 00000100:00100000:0.0:1788619921.561214:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619921.561363:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571528960 00000100:00100000:3.0:1788619921.561372:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571528960 00002000:00100000:3.0:1788619921.561416:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5027800 x1875503571528960/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:17/0 lens 488/0 e 0 to 0 dl 1788619932 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619921.561427:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5027800 with x1875503571528960 00000100:00100000:3.0:1788619921.561464:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 729 00000100:00100000:3.0:1788619921.561473:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5027800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+14:2355:x1875503571528960:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619921.561676:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5027800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+14:2355:x1875503571528960:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 204us (471us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619921.561684:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 729 00002000:00100000:3.0:1788619921.561690:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5027800 x1875503571528960/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:17/0 lens 488/4536 e 0 to 0 dl 1788619932 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619921.561697:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5027800 with x1875503571528960 00000100:00100000:0.0:1788619921.564957:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619921.565086:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571529088 00000100:00100000:3.0:1788619921.565096:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571529088 00002000:00100000:3.0:1788619921.565145:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5025180 x1875503571529088/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:17/0 lens 488/0 e 0 to 0 dl 1788619932 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619921.565155:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5025180 with x1875503571529088 00000100:00100000:3.0:1788619921.565197:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 730 00000100:00100000:3.0:1788619921.565206:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5025180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+14:2354:x1875503571529088:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619921.565430:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5025180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+14:2354:x1875503571529088:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 224us (482us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619921.565437:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 730 00002000:00100000:3.0:1788619921.565444:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5025180 x1875503571529088/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:17/0 lens 488/4536 e 0 to 0 dl 1788619932 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619921.565469:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5025180 with x1875503571529088 00000100:00100000:0.0:1788619921.568427:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.568523:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571529216 00000100:00100000:2.0:1788619921.568542:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571529216 00002000:00100000:2.0:1788619921.568589:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5025f80 x1875503571529216/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:17/0 lens 488/0 e 0 to 0 dl 1788619932 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619921.568599:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5025f80 with x1875503571529216 00000100:00100000:2.0:1788619921.568634:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 731 00000100:00100000:2.0:1788619921.568643:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5025f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+14:2355:x1875503571529216:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619921.568858:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5025f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+14:2355:x1875503571529216:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 216us (442us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619921.568865:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 731 00002000:00100000:2.0:1788619921.568872:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5025f80 x1875503571529216/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:17/0 lens 488/4536 e 0 to 0 dl 1788619932 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619921.568879:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5025f80 with x1875503571529216 00000100:00100000:0.0:1788619921.572035:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619921.572142:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571529344 00000100:00100000:1.0:1788619921.572152:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571529344 00002000:00100000:1.0:1788619921.572204:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5024e00 x1875503571529344/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:17/0 lens 488/0 e 0 to 0 dl 1788619932 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619921.572215:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5024e00 with x1875503571529344 00000100:00100000:1.0:1788619921.572255:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 732 00000100:00100000:1.0:1788619921.572264:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5024e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+14:2354:x1875503571529344:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619921.572526:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5024e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+14:2354:x1875503571529344:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 262us (501us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619921.572545:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 732 00002000:00100000:1.0:1788619921.572553:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5024e00 x1875503571529344/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:17/0 lens 488/4536 e 0 to 0 dl 1788619932 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619921.572560:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5024e00 with x1875503571529344 00000100:00100000:0.0:1788619921.575864:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.576070:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571529472 00000100:00100000:2.0:1788619921.576081:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571529472 00000100:00100000:2.0:1788619921.576141:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1134 00000100:00100000:2.0:1788619921.576151:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79008aea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2355:x1875503571529472:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619921.576174:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619921.576195:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7600/0x1cd7fe53687353fe lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c70c expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.576231:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7600/0x1cd7fe53687353fe lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c70c expref: 25 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788619921.576244:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:2.0:1788619921.576250:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7600/0x1cd7fe53687353fe lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c70c expref: 25 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.576262:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619921.576264:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619921.576267:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7600, rc 0) 00000100:00100000:2.0:1788619921.576364:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79008aea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+25:2355:x1875503571529472:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 215us (511us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619921.576372:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1134 00000100:00100000:0.0:1788619921.579830:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.580017:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571529728 00000100:00100000:2.0:1788619921.580028:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571529728 00000100:00100000:2.0:1788619921.580088:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1135 00000100:00100000:2.0:1788619921.580099:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79008ac000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+14:14908:x1875503571529728:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:2.0:1788619921.580160:0:5844:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619921.580189:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678400/0x1cd7fe536873532c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c641 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.580208:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619921.580214:0:5844:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788619921.580465:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6e00/0x1cd7fe5368735405 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.580480:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe5368735356 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c664 expref: 24 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.580489:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6e00/0x1cd7fe5368735405 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.580533:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe5368735356 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c664 expref: 24 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.580548:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe5368735356 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c664 expref: 24 pid: 11397 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619921.580563:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fd938380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589154304:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619921.580655:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78ce7f3000 going to sleep for 11 seconds 00000100:00100000:1.0:1788619921.583873:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619921.584028:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571529856 00000100:00100000:0.0:1788619921.584040:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571529856 00010000:00010000:0.0:1788619921.584083:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe5368735356 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c664 expref: 25 pid: 11397 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619921.584131:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 377 00000100:00100000:0.0:1788619921.584141:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f6680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+25:2355:x1875503571529856:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619921.584155:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619921.584162:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe5368735356 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c664 expref: 25 pid: 11397 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.584174:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe5368735356 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c664 expref: 25 pid: 11397 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.584210:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619921.584215:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6e00/0x1cd7fe5368735405 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.584225:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6e00/0x1cd7fe5368735405 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x50290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.584234:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6e00/0x1cd7fe5368735405 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x50290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.584255:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619921.584313:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f6680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+25:2355:x1875503571529856:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 173us (449us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619921.584321:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 377 00010000:00010000:2.0:1788619921.585197:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe5368735356 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c664 expref: 24 pid: 11397 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619921.585217:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fd938380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589154304:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619921.585238:0:5844:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6e00/0x1cd7fe5368735405 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.585247:0:5844:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6e00/0x1cd7fe5368735405 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.585252:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6e00/0x1cd7fe5368735405 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788619921.585407:0:5844:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967716 00000004:00010000:2.0:1788619921.585456:0:5844:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 994373 -> 994373 00000004:00010000:2.0:1788619921.585460:0:5844:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619921 -> 1788619921 00000004:00010000:2.0:1788619921.585464:0:5844:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619921 -> 1788619919 00000004:00010000:2.0:1788619921.585466:0:5844:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619921 -> 1788619921 00000004:00010000:2.0:1788619921.585468:0:5844:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1030 -> 1030 00000004:00010000:2.0:1788619921.585477:0:5844:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000009e89a118 reply state 00000000899da346 transno 4294967717 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6e00/0x1cd7fe5368735405 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619921.585486:0:5844:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788619921.585577:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79008ac000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+16:14908:x1875503571529728:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 5479us (5756us total) trans 4294967717 rc 0/0 00000100:00100000:2.0:1788619921.585584:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1135 00000020:00080000:2.0:1788619921.585588:0:5844:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:2.0:1788619921.593461:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6e00/0x1cd7fe5368735405 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.593480:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6e00/0x1cd7fe5368735405 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.593488:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6e00/0x1cd7fe5368735405 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.593495:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78cdcc6e00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:2.0:1788619921.593500:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6e00/0x1cd7fe5368735405 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.593509:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6e00/0x1cd7fe5368735405 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.593536:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619921.593540:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788619921.593545:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6e00/0x1cd7fe5368735405 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.593556:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6e00/0x1cd7fe5368735405 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619921.595857:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.596025:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571529984 00000100:00100000:2.0:1788619921.596036:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571529984 00000100:00100000:2.0:1788619921.596100:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1136 00000100:00100000:2.0:1788619921.596111:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+15:14908:x1875503571529984:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619921.596133:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619921.596136:0:11397:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1788619921.596181:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07400/0x1cd7fe53687353f7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c705 expref: 15 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.596200:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619921.596205:0:11397:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788619921.596218:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6400/0x1cd7fe536873540c lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c713 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.596242:0:11397:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7600/0x1cd7fe53687353fe lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c70c expref: 23 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.596251:0:11397:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6400/0x1cd7fe536873540c lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c713 expref: 15 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.596281:0:11397:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7600/0x1cd7fe53687353fe lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c70c expref: 23 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.596293:0:11397:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7600/0x1cd7fe53687353fe lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c70c expref: 23 pid: 5846 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619921.596307:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fd938380 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589154432:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619921.596408:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78ce7f3000 going to sleep for 11 seconds 00000100:00100000:1.0:1788619921.599490:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619921.599594:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571530112 00000100:00100000:0.0:1788619921.599604:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571530112 00010000:00010000:0.0:1788619921.599642:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7600/0x1cd7fe53687353fe lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c70c expref: 24 pid: 5846 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619921.599687:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 378 00000100:00100000:0.0:1788619921.599696:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f4000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2354:x1875503571530112:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619921.599722:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619921.599748:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7600/0x1cd7fe53687353fe lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c70c expref: 24 pid: 5846 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.599765:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7600/0x1cd7fe53687353fe lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c70c expref: 24 pid: 5846 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.599808:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619921.599815:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6400/0x1cd7fe536873540c lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c713 expref: 15 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.599827:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6400/0x1cd7fe536873540c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c713 expref: 15 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.599836:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6400/0x1cd7fe536873540c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c713 expref: 15 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.599873:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6400/0x1cd7fe536873540c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c713 expref: 15 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619921.599887:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6f86a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589154560:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619921.599983:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebc00 going to sleep for 11 seconds 00010000:00010000:2.0:1788619921.601136:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7600/0x1cd7fe53687353fe lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c70c expref: 24 pid: 5846 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619921.601155:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fd938380 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589154432:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619921.601181:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6400/0x1cd7fe536873540c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c713 expref: 15 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.601194:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619921.601197:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619921.601200:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6400, rc 0) 00000100:00100000:2.0:1788619921.601250:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+15:14908:x1875503571529984:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5141us (5401us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619921.601258:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1136 00000100:00100000:1.0:1788619921.604956:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619921.605129:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571530240 00000100:00100000:3.0:1788619921.605138:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571530240 00000100:00100000:3.0:1788619921.605195:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 903 00000100:00100000:3.0:1788619921.605205:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:14908:x1875503571530240:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619921.605223:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619921.605242:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe5368735413 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c71a expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619921.605273:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000005cb4402d/0x1cd7fe5368735299 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1277952->1327103) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5d1 expref: 5 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619921.605281:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=1s v=(4 7) 00010000:00010000:3.0:1788619921.605285:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(5 7) 00010000:00010000:3.0:1788619921.605290:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe5368735413 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c71a expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619921.605323:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000005cb4402d/0x1cd7fe5368735299 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1277952->1327103) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5d1 expref: 5 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619921.605371:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000005cb4402d/0x1cd7fe5368735299 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1277952->1327103) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5d1 expref: 5 pid: 6697 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619921.605385:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6533800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589154688:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619921.605456:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48cd80 going to sleep for 11 seconds 00000100:00100000:0.0:1788619921.605699:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6f86a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589154560:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619921.605729:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619921.605756:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f4000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+23:2354:x1875503571530112:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 6060us (6275us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619921.605766:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 378 00000100:00100000:1.0:1788619921.609510:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619921.609691:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571530368 00000100:00100000:0.0:1788619921.609711:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571530368 00010000:00010000:0.0:1788619921.609752:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000005cb4402d/0x1cd7fe5368735299 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1277952->1327103) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5d1 expref: 6 pid: 6697 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619921.609816:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 379 00000100:00100000:0.0:1788619921.609827:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571530368:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619921.609842:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619921.609866:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000005cb4402d/0x1cd7fe5368735299 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1277952->1327103) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5d1 expref: 6 pid: 6697 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.609881:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000005cb4402d/0x1cd7fe5368735299 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1277952->1327103) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5d1 expref: 6 pid: 6697 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.609908:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(5 7) 00010000:00010000:0.0:1788619921.609912:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(5 7) 00010000:00010000:0.0:1788619921.609920:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe5368735413 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c71a expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.609930:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe5368735413 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c71a expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.609967:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe5368735413 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c71a expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619921.609989:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6f86a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589154816:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619921.610069:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebc00 going to sleep for 11 seconds 00010000:00010000:3.0:1788619921.612752:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cb4402d/0x1cd7fe5368735299 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1277952->1327103) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c5d1 expref: 6 pid: 6697 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619921.612774:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6533800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589154688:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619921.612805:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe5368735413 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c71a expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619921.612821:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683dc00, rc 0) 00000100:00100000:3.0:1788619921.612875:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:14908:x1875503571530240:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7671us (7929us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619921.612884:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 903 00000100:00100000:2.0:1788619921.613256:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6f86a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589154816:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619921.613271:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619921.613306:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571530368:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3481us (3807us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619921.613315:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 379 00000100:00100000:0.0:1788619921.616701:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619921.616801:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571530496 00000100:00100000:3.0:1788619921.616810:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571530496 00002000:00010000:3.0:1788619921.616846:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1918315->18446744073709551615) 00002000:00100000:3.0:1788619921.616850:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6f84a80 with x1875503571530496 ext(1918315->18446744073709551615) 00002000:00010000:3.0:1788619921.616861:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6f84a80. 00000100:00100000:3.0:1788619921.616892:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 512 00000100:00100000:3.0:1788619921.616902:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6f84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571530496:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:3.0:1788619921.617158:0:6703:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1536430 -> 1918315 00002000:00010000:3.0:1788619921.617161:0:6703:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619921 -> 1788619921 00002000:00010000:3.0:1788619921.617164:0:6703:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619921 -> 1788619921 00002000:00010000:3.0:1788619921.617166:0:6703:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619921 -> 1788619921 00002000:00010000:3.0:1788619921.617168:0:6703:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 775 -> 391 00000100:00100000:3.0:1788619921.617228:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6f84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571530496:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 326us (536us total) trans 4294967449 rc 0/0 00000100:00100000:3.0:1788619921.617244:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 512 00002000:00010000:3.0:1788619921.617249:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1918315->18446744073709551615) 00002000:00100000:3.0:1788619921.617251:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6f84a80 with x1875503571530496 ext(1918315->18446744073709551615) 00002000:00010000:3.0:1788619921.617256:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6f84a80. 00000100:00100000:0.0:1788619921.620199:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619921.620426:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571530624 00000100:00100000:1.0:1788619921.620437:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571530624 00000100:00100000:1.0:1788619921.620493:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1137 00000100:00100000:1.0:1788619921.620502:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6f86a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+15:14908:x1875503571530624:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619921.620522:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619921.620553:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06c00/0x1cd7fe536873541a lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c721 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.620634:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06e00/0x1cd7fe5368735421 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.620648:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06e00/0x1cd7fe5368735421 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.620660:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06e00/0x1cd7fe5368735421 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788619921.620687:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1918315 00000004:00200000:1.0:1788619921.620691:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788619921.620733:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06e00/0x1cd7fe5368735421 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.620757:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06c00/0x1cd7fe536873541a lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c721 expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.620774:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06e00/0x1cd7fe5368735421 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c721 expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.620782:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619921.620784:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788619921.620788:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa06e00, rc 0) 00000100:00100000:1.0:1788619921.620843:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6f86a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+16:14908:x1875503571530624:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 342us (654us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619921.620850:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1137 00000100:00100000:1.0:1788619921.629589:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.629716:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571530752 00000100:00100000:2.0:1788619921.629727:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571530752 00002000:00100000:2.0:1788619921.629781:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1921023) req@ffff8e78df8b1500 x1875503571530752/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:17/0 lens 488/0 e 0 to 0 dl 1788619932 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619921.629798:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df8b1500 with x1875503571530752 ext(1048576->1921023) 00002000:00010000:2.0:1788619921.629804:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df8b1500 00000100:00100000:2.0:1788619921.629842:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 513 00000100:00100000:2.0:1788619921.629855:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571530752:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619921.635850:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619921.637004:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571530880 00000100:00100000:3.0:1788619921.637017:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571530880 00002000:00100000:3.0:1788619921.637066:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5ff7100 x1875503571530880/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:17/0 lens 488/0 e 0 to 0 dl 1788619932 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619921.637077:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5ff7100 with x1875503571530880 00000100:00100000:3.0:1788619921.637117:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 733 00000100:00100000:3.0:1788619921.637126:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+16:2354:x1875503571530880:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619921.645135:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571530752:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 15273us (15545us total) trans 0 rc 872448/872448 00000100:00100000:2.0:1788619921.645150:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 513 00002000:00100000:2.0:1788619921.645163:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1921023) req@ffff8e78df8b1500 x1875503571530752/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:17/0 lens 488/440 e 0 to 0 dl 1788619932 ref 1 fl Complete:/600/0 rc 872448/872448 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619921.645173:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df8b1500 with x1875503571530752 ext(1048576->1921023) 00002000:00010000:2.0:1788619921.645180:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df8b1500 00000100:00100000:2.0:1788619921.649994:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+16:2354:x1875503571530880:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 12856us (14140us total) trans 0 rc 995328/995328 00000100:00100000:2.0:1788619921.650009:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 733 00002000:00100000:2.0:1788619921.650023:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5ff7100 x1875503571530880/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:17/0 lens 488/440 e 0 to 0 dl 1788619932 ref 1 fl Complete:/600/0 rc 995328/995328 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619921.650033:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5ff7100 with x1875503571530880 00000100:00100000:3.0:1788619921.654000:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.654148:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571531008 00000100:00100000:2.0:1788619921.654157:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571531008 00000100:00100000:2.0:1788619921.654220:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1138 00000100:00100000:2.0:1788619921.654230:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6533800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+23:14908:x1875503571531008:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619921.654250:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619921.654271:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7600/0x1cd7fe5368735428 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c728 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.654299:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6400/0x1cd7fe536873540c lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c713 expref: 15 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.654309:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7600/0x1cd7fe5368735428 lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c728 expref: 24 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.654383:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6400/0x1cd7fe536873540c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c713 expref: 15 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.654396:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6400/0x1cd7fe536873540c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c713 expref: 15 pid: 11397 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619921.654409:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fd938380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589154944:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619921.654487:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78ce7f3000 going to sleep for 11 seconds 00000100:00100000:1.0:1788619921.659985:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619921.660105:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571531136 00000100:00100000:3.0:1788619921.660114:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571531136 00010000:00010000:3.0:1788619921.660157:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6400/0x1cd7fe536873540c lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c713 expref: 16 pid: 11397 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619921.660198:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 380 00000100:00100000:3.0:1788619921.660207:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b16300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571531136:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619921.660221:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619921.660247:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6400/0x1cd7fe536873540c lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c713 expref: 16 pid: 11397 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619921.660261:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6400/0x1cd7fe536873540c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c713 expref: 16 pid: 11397 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619921.660305:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619921.660312:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7600/0x1cd7fe5368735428 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c728 expref: 24 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619921.660358:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7600/0x1cd7fe5368735428 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c728 expref: 24 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619921.660369:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7600/0x1cd7fe5368735428 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c728 expref: 24 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619921.660406:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7600/0x1cd7fe5368735428 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c728 expref: 24 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619921.660420:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6530700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589155072:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619921.660497:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48cd80 going to sleep for 11 seconds 00000100:00100000:3.0:1788619921.662642:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6530700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589155072:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619921.662665:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619921.662728:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b16300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571531136:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 2521us (2762us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619921.662740:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 380 00010000:00010000:2.0:1788619921.664110:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6400/0x1cd7fe536873540c lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c713 expref: 15 pid: 11397 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619921.664132:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fd938380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589154944:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619921.664162:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7600/0x1cd7fe5368735428 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c728 expref: 24 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.664178:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619921.664181:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619921.664185:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7600, rc 0) 00000100:00100000:2.0:1788619921.664297:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6533800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+24:14908:x1875503571531008:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 10067us (10306us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619921.664306:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1138 00000100:00100000:1.0:1788619921.670880:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.670987:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571531264 00000100:00100000:2.0:1788619921.670997:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571531264 00002000:00100000:2.0:1788619921.671050:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b14a80 x1875503571531264/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:17/0 lens 488/0 e 0 to 0 dl 1788619932 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619921.671061:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b14a80 with x1875503571531264 00000100:00100000:2.0:1788619921.671099:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 734 00000100:00100000:2.0:1788619921.671108:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b14a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2352:x1875503571531264:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619921.684659:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b14a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2352:x1875503571531264:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 13541us (13780us total) trans 0 rc 995328/995328 00000100:00100000:2.0:1788619921.684673:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 734 00002000:00100000:2.0:1788619921.684685:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b14a80 x1875503571531264/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:17/0 lens 488/440 e 0 to 0 dl 1788619932 ref 1 fl Complete:/600/0 rc 995328/995328 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619921.684696:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b14a80 with x1875503571531264 00000100:00100000:3.0:1788619921.688271:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.688374:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571531392 00000100:00100000:2.0:1788619921.688385:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571531392 00000100:00100000:2.0:1788619921.688446:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1139 00000100:00100000:2.0:1788619921.688457:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6530700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+24:14908:x1875503571531392:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619921.688478:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619921.688499:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6400/0x1cd7fe536873542f lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c72f expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.688712:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6e00/0x1cd7fe5368735436 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.688736:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6e00/0x1cd7fe5368735436 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.688748:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6e00/0x1cd7fe5368735436 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788619921.688775:0:11397:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1918315 00000004:00200000:2.0:1788619921.688780:0:11397:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788619921.688798:0:11397:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6e00/0x1cd7fe5368735436 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.688827:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6400/0x1cd7fe536873542f lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c72f expref: 26 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.688846:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6e00/0x1cd7fe5368735436 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c72f expref: 25 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.688855:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619921.688858:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788619921.688862:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6e00, rc 0) 00000100:00100000:2.0:1788619921.688922:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6530700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+25:14908:x1875503571531392:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 467us (659us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619921.688930:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1139 00000100:00100000:0.0:1788619921.691739:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619921.692144:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571531520 00000100:00100000:1.0:1788619921.692154:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571531520 00000100:00100000:1.0:1788619921.692211:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 904 00000100:00100000:1.0:1788619921.692222:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571531520:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619921.692240:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619921.692258:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fa452954/0x1cd7fe536873543d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c736 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619921.692316:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b16680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589155200:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619921.692426:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7600 going to sleep for 11 seconds 00000100:00100000:0.0:1788619921.694842:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b16680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589155200:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619921.694881:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000fa452954/0x1cd7fe536873543d lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c736 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.694907:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fa452954/0x1cd7fe536873543d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c736 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.694920:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5dee800, rc 0) 00000100:00100000:0.0:1788619921.694985:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571531520:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2765us (3254us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619921.694994:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 904 00000100:00100000:0.0:1788619921.697288:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.699479:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571531648 00000100:00100000:2.0:1788619921.699494:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571531648 00000100:00100000:2.0:1788619921.699573:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 905 00000100:00100000:2.0:1788619921.699584:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571531648:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619921.699601:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619921.699621:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067546236/0x1cd7fe5368735444 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c73d expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619921.699688:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fd938380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589155328:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619921.699778:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78ce7f3000 going to sleep for 11 seconds 00000100:00100000:2.0:1788619921.702242:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fd938380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589155328:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619921.702281:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000067546236/0x1cd7fe5368735444 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c73d expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.702307:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067546236/0x1cd7fe5368735444 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c73d expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.702319:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6400, rc 0) 00000100:00100000:2.0:1788619921.702416:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571531648:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2833us (5135us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619921.702425:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 905 00000100:00100000:1.0:1788619921.705293:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619921.705407:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571531776 00000100:00100000:3.0:1788619921.705417:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571531776 00000100:00100000:3.0:1788619921.705478:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1140 00000100:00100000:3.0:1788619921.705488:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b15f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+15:2352:x1875503571531776:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619921.705524:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619921.705554:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe536873544b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c744 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619921.705588:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe536873544b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c744 expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788619921.705600:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:3.0:1788619921.705606:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe536873544b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c744 expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619921.705619:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619921.705622:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619921.705626:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683da00, rc 0) 00000100:00100000:3.0:1788619921.705684:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b15f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+16:2352:x1875503571531776:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 197us (400us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619921.705690:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1140 00000100:00100000:0.0:1788619921.709627:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.709847:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571531904 00000100:00100000:2.0:1788619921.709857:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571531904 00000100:00100000:2.0:1788619921.709916:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 906 00000100:00100000:2.0:1788619921.709926:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d5ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571531904:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619921.709946:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619921.709964:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067546236/0x1cd7fe5368735452 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c74b expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619921.710032:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fd938380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589155456:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619921.710119:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78ce7f3000 going to sleep for 11 seconds 00000100:00100000:2.0:1788619921.713305:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fd938380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589155456:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619921.713378:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000067546236/0x1cd7fe5368735452 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c74b expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.713401:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067546236/0x1cd7fe5368735452 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c74b expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.713413:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6400, rc 0) 00000100:00100000:2.0:1788619921.713463:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d5ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571531904:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3538us (3846us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619921.713472:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 906 00000100:00100000:1.0:1788619921.716769:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619921.716903:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571532032 00000100:00100000:3.0:1788619921.716914:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571532032 00000100:00100000:3.0:1788619921.716981:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 907 00000100:00100000:3.0:1788619921.716993:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b17100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571532032:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619921.717011:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619921.717030:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003b84bbc9/0x1cd7fe5368735459 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c752 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619921.717089:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6532d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589155584:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619921.717172:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48cd80 going to sleep for 11 seconds 00000100:00100000:3.0:1788619921.719398:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6532d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589155584:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619921.719436:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003b84bbc9/0x1cd7fe5368735459 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c752 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619921.719463:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b84bbc9/0x1cd7fe5368735459 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c752 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619921.719477:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683de00, rc 0) 00000100:00100000:3.0:1788619921.719543:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b17100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571532032:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2552us (2783us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619921.719551:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 907 00000100:00100000:1.0:1788619921.727626:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.727761:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571532160 00000100:00100000:2.0:1788619921.727772:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571532160 00002000:00100000:2.0:1788619921.727824:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1466368->1519615) req@ffff8e77c6b14000 x1875503571532160/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:17/0 lens 488/0 e 0 to 0 dl 1788619932 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619921.727835:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6b14000 with x1875503571532160 ext(1466368->1519615) 00002000:00010000:2.0:1788619921.727840:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6b14000 00000100:00100000:2.0:1788619921.727875:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 514 00000100:00100000:2.0:1788619921.727886:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b14000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571532160:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:2.0:1788619921.735688:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b14000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571532160:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 7794us (8064us total) trans 4294967450 rc 0/0 00000100:00100000:2.0:1788619921.735713:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 514 00002000:00100000:2.0:1788619921.735726:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1466368->1519615) req@ffff8e77c6b14000 x1875503571532160/t4294967450(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:17/0 lens 488/448 e 0 to 0 dl 1788619932 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619921.735737:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6b14000 with x1875503571532160 ext(1466368->1519615) 00002000:00010000:2.0:1788619921.735744:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6b14000 00000100:00100000:1.0:1788619921.739864:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619921.739965:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571532288 00000100:00100000:0.0:1788619921.739985:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571532288 00000100:00100000:0.0:1788619921.740050:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 908 00000100:00100000:0.0:1788619921.740061:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b16a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571532288:12345-192.168.201.5@tcp:16:fsx.0 00000001:00080000:3.0:1788619921.768271:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967450 is committed 00000100:00100000:0.0:1788619921.768444:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b16a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571532288:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 28378us (28581us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619921.768457:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 908 00000100:00100000:0.0:1788619921.771579:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.771731:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571532416 00000100:00100000:2.0:1788619921.771742:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571532416 00000100:00100000:2.0:1788619921.771803:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 909 00000100:00100000:2.0:1788619921.771813:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571532416:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619921.771835:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619921.771852:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041aa2bf0/0x1cd7fe5368735460 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c759 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619921.771908:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fd938380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589155712:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619921.772055:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164540 going to sleep for 11 seconds 00002000:00010000:1.0:1788619921.774215:0:6696:0:(ofd_lvb.c:284:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks: 391 -> 519 00000100:00100000:1.0:1788619921.774246:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fd938380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589155712:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619921.774285:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000041aa2bf0/0x1cd7fe5368735460 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c759 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.774308:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041aa2bf0/0x1cd7fe5368735460 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c759 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.774322:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7a00, rc 0) 00000100:00100000:1.0:1788619921.774421:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571532416:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2609us (2850us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619921.774430:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 909 00000100:00100000:1.0:1788619921.776750:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.777088:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571532544 00000100:00100000:2.0:1788619921.777097:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571532544 00000100:00100000:2.0:1788619921.777178:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 910 00000100:00100000:2.0:1788619921.777187:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b16680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571532544:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619921.777205:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619921.777220:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007527590a/0x1cd7fe5368735467 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c760 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619921.777270:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fd938a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589155840:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619921.777367:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164a80 going to sleep for 11 seconds 00000100:00100000:2.0:1788619921.779409:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fd938a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589155840:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619921.779446:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007527590a/0x1cd7fe5368735467 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c760 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.779472:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007527590a/0x1cd7fe5368735467 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c760 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.779484:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6800, rc 0) 00000100:00100000:2.0:1788619921.779534:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b16680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571532544:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2347us (2792us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619921.779542:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 910 00000100:00100000:3.0:1788619921.785611:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619921.785772:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571532672 00000100:00100000:1.0:1788619921.785783:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571532672 00000100:00100000:1.0:1788619921.785836:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 911 00000100:00100000:1.0:1788619921.785846:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6532d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571532672:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619921.785864:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619921.785882:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fa452954/0x1cd7fe536873546e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c767 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619921.785961:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df96a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589155968:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619921.786053:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7780 going to sleep for 11 seconds 00000100:00100000:3.0:1788619921.789282:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df96a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589155968:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619921.789319:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000fa452954/0x1cd7fe536873546e lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c767 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619921.789376:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fa452954/0x1cd7fe536873546e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c767 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619921.789389:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5dee800, rc 0) 00000100:00100000:3.0:1788619921.789447:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6532d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571532672:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3603us (3846us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619921.789456:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 911 00000100:00100000:3.0:1788619921.791871:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619921.791998:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571532800 00000100:00100000:0.0:1788619921.792009:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571532800 00000100:00100000:0.0:1788619921.792075:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 912 00000100:00100000:0.0:1788619921.792085:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6533480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571532800:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619921.792103:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619921.792120:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c5c78e09/0x1cd7fe5368735475 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c76e expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619921.792179:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5dc9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589156096:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619921.792264:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb600 going to sleep for 11 seconds 00000100:00100000:0.0:1788619921.794672:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5dc9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589156096:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619921.794721:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c5c78e09/0x1cd7fe5368735475 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c76e expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.794743:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c5c78e09/0x1cd7fe5368735475 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c76e expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.794756:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed1400, rc 0) 00000100:00100000:0.0:1788619921.794809:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6533480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571532800:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2726us (2946us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619921.794817:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 912 00000100:00100000:1.0:1788619921.800711:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.800810:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571532928 00000100:00100000:2.0:1788619921.800819:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571532928 00000100:00100000:2.0:1788619921.800878:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1141 00000100:00100000:2.0:1788619921.800887:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df968000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+25:14908:x1875503571532928:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619921.800918:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619921.800922:0:5844:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1788619921.800965:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7600/0x1cd7fe5368735428 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c728 expref: 25 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.800997:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619921.801002:0:5844:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788619921.801015:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe536873547c lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c775 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.801040:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe536873544b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c744 expref: 15 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.801050:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe536873547c lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c775 expref: 25 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.801084:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe536873544b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c744 expref: 15 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.801098:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe536873544b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c744 expref: 15 pid: 9417 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619921.801111:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fd938a80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589156224:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619921.801192:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164a80 going to sleep for 11 seconds 00000100:00100000:3.0:1788619921.808444:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.808567:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fd938a80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589156224:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619921.808616:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe536873547c lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c775 expref: 25 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.808635:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619921.808638:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619921.808643:0:5844:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe536873547c lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c775 expref: 25 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.808653:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7000, rc 0) 00000100:00100000:2.0:1788619921.808729:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df968000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+25:14908:x1875503571532928:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7843us (8039us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619921.808738:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1141 00000100:00100000:0.0:1788619921.809364:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571533056 00000100:00100000:0.0:1788619921.809372:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571533056 00010000:00010000:0.0:1788619921.809405:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe536873544b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c744 expref: 16 pid: 9417 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619921.809441:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 381 00000100:00100000:0.0:1788619921.809450:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6531880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571533056:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619921.809480:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619921.809505:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe536873544b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c744 expref: 16 pid: 9417 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.809518:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe536873544b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c744 expref: 16 pid: 9417 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.809551:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe536873544b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c744 expref: 16 pid: 9417 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.809567:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619921.809571:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe536873547c lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c775 expref: 24 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.809581:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe536873547c lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c775 expref: 24 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.809709:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe536873547c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c775 expref: 24 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.809749:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe536873547c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c775 expref: 24 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619921.809762:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5dc9c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589156352:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619921.809849:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb600 going to sleep for 11 seconds 00000100:00100000:0.0:1788619921.813250:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5dc9c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589156352:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619921.813297:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619921.813385:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6531880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+15:2355:x1875503571533056:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3934us (4948us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619921.813396:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 381 00000100:00100000:2.0:1788619921.816216:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619921.816505:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571533184 00000100:00100000:3.0:1788619921.816514:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571533184 00000100:00100000:3.0:1788619921.816604:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 735 00000100:00100000:3.0:1788619921.816614:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd938a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+25:2354:x1875503571533184:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:3.0:1788619921.816870:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 994373 -> 994373 00000004:00010000:3.0:1788619921.816875:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619921 -> 1788619921 00000004:00010000:3.0:1788619921.816877:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619919 -> 1788619919 00000004:00010000:3.0:1788619921.816879:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619921 -> 1788619921 00000004:00010000:3.0:1788619921.816881:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1030 -> 1030 00000100:00100000:3.0:1788619921.816975:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd938a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+26:2354:x1875503571533184:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 355us (761us total) trans 4294967718 rc 0/0 00000100:00100000:3.0:1788619921.816982:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 735 00000100:00100000:1.0:1788619921.826836:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.826933:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571533312 00000100:00100000:2.0:1788619921.826943:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571533312 00000100:00100000:2.0:1788619921.827010:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1142 00000100:00100000:2.0:1788619921.827022:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df969880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+15:2355:x1875503571533312:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619921.827042:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619921.827061:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7600/0x1cd7fe5368735483 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c77c expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.827095:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe536873547c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c775 expref: 25 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619921.827133:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fd938700 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589156480:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619921.827274:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164a80 going to sleep for 11 seconds 00000004:00010000:2.0:1788619921.830653:0:11397:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619919 -> 1788619921 00000100:00100000:2.0:1788619921.830687:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fd938700 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589156480:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619921.830751:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:2.0:1788619921.830770:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7600/0x1cd7fe5368735483 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c77c expref: 16 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.830791:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619921.830794:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619921.830799:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7600/0x1cd7fe5368735483 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c77c expref: 16 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.830814:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7600, rc 0) 00000100:00100000:2.0:1788619921.830880:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df969880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+15:2355:x1875503571533312:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3859us (4053us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619921.830889:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1142 00000100:00100000:0.0:1788619921.834213:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.834299:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571533440 00000100:00100000:2.0:1788619921.834311:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571533440 00000100:00100000:2.0:1788619921.834400:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1143 00000100:00100000:2.0:1788619921.834410:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dc9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+15:2355:x1875503571533440:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619921.834430:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619921.834451:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe536873548a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c783 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.834487:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe536873547c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c775 expref: 25 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619921.834538:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fd938700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589156608:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619921.834614:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164a80 going to sleep for 11 seconds 00000100:00100000:2.0:1788619921.837043:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fd938700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589156608:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619921.837070:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:2.0:1788619921.837088:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe536873548a lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c783 expref: 16 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.837108:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619921.837110:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619921.837116:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe536873548a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c783 expref: 16 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.837130:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7c00, rc 0) 00000100:00100000:2.0:1788619921.837189:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dc9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+15:2355:x1875503571533440:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2780us (2985us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619921.837197:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1143 00000100:00100000:0.0:1788619921.842399:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.842477:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571533568 00000100:00100000:2.0:1788619921.842486:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571533568 00000100:00100000:2.0:1788619921.842554:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 913 00000100:00100000:2.0:1788619921.842564:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dcad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571533568:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619921.842583:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619921.842600:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000073be6ea/0x1cd7fe5368735491 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c78a expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619921.842658:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fd938700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589156736:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619921.842743:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164a80 going to sleep for 11 seconds 00000100:00100000:2.0:1788619921.844832:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fd938700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589156736:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619921.844869:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000073be6ea/0x1cd7fe5368735491 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c78a expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.844892:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000073be6ea/0x1cd7fe5368735491 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c78a expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.844906:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7c00, rc 0) 00000100:00100000:2.0:1788619921.845006:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dcad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571533568:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2444us (2615us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619921.845015:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 913 00000100:00100000:1.0:1788619921.848760:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.848857:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571533696 00000100:00100000:2.0:1788619921.848868:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571533696 00000100:00100000:2.0:1788619921.849132:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 914 00000100:00100000:2.0:1788619921.849144:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df96a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571533696:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619921.849163:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619921.849180:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003a2d9d44/0x1cd7fe5368735498 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c791 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619921.849236:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fd938700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589156864:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619921.849403:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164a80 going to sleep for 11 seconds 00000100:00100000:2.0:1788619921.852045:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fd938700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589156864:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619921.852089:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003a2d9d44/0x1cd7fe5368735498 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c791 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.852114:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a2d9d44/0x1cd7fe5368735498 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c791 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.852128:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7600, rc 0) 00000100:00100000:2.0:1788619921.852229:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df96a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571533696:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3088us (3478us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619921.852237:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 914 00000100:00100000:1.0:1788619921.857154:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619921.857255:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571533824 00000100:00100000:0.0:1788619921.857265:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571533824 00000100:00100000:0.0:1788619921.857352:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1144 00000100:00100000:0.0:1788619921.857363:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+15:2355:x1875503571533824:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619921.857383:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619921.857404:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1200/0x1cd7fe536873549f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c798 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.857440:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe536873547c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c775 expref: 25 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619921.857495:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6343800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589156992:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619921.857787:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb600 going to sleep for 11 seconds 00000100:00100000:0.0:1788619921.860196:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6343800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589156992:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619921.860232:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:0.0:1788619921.860251:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1200/0x1cd7fe536873549f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c798 expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.860273:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619921.860276:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619921.860298:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1200/0x1cd7fe536873549f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c798 expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.860314:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed1200, rc 0) 00000100:00100000:0.0:1788619921.860417:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+15:2355:x1875503571533824:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3056us (3273us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619921.860426:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1144 00000100:00100000:3.0:1788619921.863312:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619921.863539:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571533952 00000100:00100000:1.0:1788619921.863548:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571533952 00000100:00100000:1.0:1788619921.863606:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1145 00000100:00100000:1.0:1788619921.863616:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6531500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+15:2353:x1875503571533952:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619921.863635:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619921.863655:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5def000/0x1cd7fe53687354a6 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c79f expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.863691:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe536873547c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c775 expref: 25 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619921.863739:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4ba1500 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589157120:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619921.863882:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7f00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619921.866578:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4ba1500 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589157120:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619921.866605:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:2.0:1788619921.866624:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5def000/0x1cd7fe53687354a6 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c79f expref: 16 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.866643:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619921.866645:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619921.866651:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5def000/0x1cd7fe53687354a6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c79f expref: 16 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.866665:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5def000, rc 0) 00000100:00100000:2.0:1788619921.866728:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6531500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+15:2353:x1875503571533952:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3113us (3424us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619921.866736:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1145 00000100:00100000:3.0:1788619921.869502:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619921.869629:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571534080 00000100:00100000:0.0:1788619921.869639:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571534080 00000100:00100000:0.0:1788619921.869707:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 915 00000100:00100000:0.0:1788619921.869719:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:14908:x1875503571534080:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619921.869738:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619921.869755:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd8a44c0/0x1cd7fe53687354ad lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c7a6 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.869786:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe5368735413 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c71a expref: 5 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.869794:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(5 7) 00010000:00010000:0.0:1788619921.869798:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(6 7) 00010000:00010000:0.0:1788619921.869803:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd8a44c0/0x1cd7fe53687354ad lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1519616->1544191] (req 1519616->1544191) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7a6 expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.869835:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe5368735413 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c71a expref: 5 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.869849:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe5368735413 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c71a expref: 5 pid: 6697 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619921.869878:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6343800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589157248:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619921.869947:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb600 going to sleep for 11 seconds 00000100:00100000:0.0:1788619921.872646:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6343800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589157248:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619921.872683:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000dd8a44c0/0x1cd7fe53687354ad lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1519616->1544191] (req 1519616->1544191) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7a6 expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.872713:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed1600, rc 0) 00000100:00100000:0.0:1788619921.872770:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:14908:x1875503571534080:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3053us (3278us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619921.872779:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 915 00000100:00100000:3.0:1788619921.876093:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619921.876219:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571534208 00000100:00100000:0.0:1788619921.876228:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571534208 00010000:00010000:0.0:1788619921.876263:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe5368735413 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c71a expref: 6 pid: 6697 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619921.876302:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 382 00000100:00100000:0.0:1788619921.876311:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571534208:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619921.876359:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619921.876380:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe5368735413 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c71a expref: 6 pid: 6697 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.876393:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe5368735413 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c71a expref: 6 pid: 6697 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.876435:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe5368735413 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c71a expref: 6 pid: 6697 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.876451:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(6 7) 00010000:00010000:0.0:1788619921.876456:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(6 7) 00010000:00010000:0.0:1788619921.876463:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000dd8a44c0/0x1cd7fe53687354ad lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1519616->1544191) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7a6 expref: 5 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.876474:0:5829:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd8a44c0/0x1cd7fe53687354ad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1519616->1544191) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7a6 expref: 5 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.876508:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000dd8a44c0/0x1cd7fe53687354ad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1519616->1544191) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7a6 expref: 5 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619921.876533:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6343800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1875503589157376:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619921.876606:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb600 going to sleep for 11 seconds 00000100:00100000:3.0:1788619921.879989:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.880080:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571534336 00000100:00100000:2.0:1788619921.880090:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571534336 00000100:00100000:2.0:1788619921.880159:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 515 00000100:00100000:2.0:1788619921.880170:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b34e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571534336:12345-192.168.201.5@tcp:22:fsx.0 00002000:00010000:2.0:1788619921.880538:0:6702:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1918315 -> 1918315 00002000:00010000:2.0:1788619921.880542:0:6702:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619921 -> 1788619921 00002000:00010000:2.0:1788619921.880545:0:6702:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619921 -> 1788619921 00002000:00010000:2.0:1788619921.880547:0:6702:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619921 -> 1788619921 00002000:00010000:2.0:1788619921.880549:0:6702:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 519 -> 519 00000100:00100000:2.0:1788619921.880615:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b34e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571534336:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 447us (644us total) trans 4294967451 rc 0/0 00000100:00100000:2.0:1788619921.880622:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 515 00000100:00100000:0.0:1788619921.881020:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6343800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1875503589157376:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619921.881042:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619921.881069:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571534208:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 4760us (4986us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619921.881078:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 382 00000100:00100000:3.0:1788619921.884928:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619921.884983:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619921.885124:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571534592 00000100:00100000:0.0:1788619921.885134:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571534592 00000100:00100000:0.0:1788619921.885201:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 916 00000100:00100000:0.0:1788619921.885224:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b34000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571534592:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619921.885241:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619921.885258:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a7b141ab/0x1cd7fe53687354b4 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c7b4 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619921.885314:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6343800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589157504:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619921.885426:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb600 going to sleep for 11 seconds 00000100:00100000:2.0:1788619921.885854:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571534464 00000100:00100000:2.0:1788619921.885859:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571534464 00000100:00100000:2.0:1788619921.885901:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1146 00000100:00100000:2.0:1788619921.885919:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b36300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+15:2355:x1875503571534464:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619921.885929:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619921.885942:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7600/0x1cd7fe53687354bb lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c7ad expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.885978:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe536873547c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c775 expref: 25 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619921.886003:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fd938700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589157632:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619921.886045:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164a80 going to sleep for 11 seconds 00000100:00100000:2.0:1788619921.889277:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fd938700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589157632:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619921.889305:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:2.0:1788619921.889365:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7600/0x1cd7fe53687354bb lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7ad expref: 16 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.889386:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619921.889390:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619921.889395:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7600/0x1cd7fe53687354bb lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7ad expref: 16 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.889407:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7600, rc 0) 00000100:00100000:2.0:1788619921.889456:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b36300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+15:2355:x1875503571534464:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3539us (4536us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619921.889466:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1146 00000100:00100000:0.0:1788619921.890603:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6343800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589157504:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619921.890637:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a7b141ab/0x1cd7fe53687354b4 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7b4 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.890672:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a7b141ab/0x1cd7fe53687354b4 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7b4 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.890685:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed1200, rc 0) 00000100:00100000:0.0:1788619921.890746:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b34000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571534592:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5523us (5762us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619921.890754:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 916 00000100:00100000:3.0:1788619921.893317:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619921.893714:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571534720 00000100:00100000:0.0:1788619921.893723:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571534720 00000100:00100000:0.0:1788619921.893777:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1147 00000100:00100000:0.0:1788619921.893786:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b36680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+15:2352:x1875503571534720:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619921.893807:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619921.893826:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1e00/0x1cd7fe53687354c2 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c7bb expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.893884:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe536873547c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c775 expref: 25 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619921.893924:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6343800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589157760:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619921.894016:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb600 going to sleep for 11 seconds 00000100:00100000:1.0:1788619921.895183:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.895245:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571534848 00000100:00100000:2.0:1788619921.895252:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571534848 00000100:00100000:2.0:1788619921.895297:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 917 00000100:00100000:2.0:1788619921.895306:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571534848:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619921.895319:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619921.895359:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000073be6ea/0x1cd7fe53687354c9 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c7c2 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619921.895408:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fd938700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589157888:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619921.895470:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164a80 going to sleep for 11 seconds 00000100:00100000:0.0:1788619921.897570:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6343800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589157760:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619921.897596:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 994373 00010000:00010000:0.0:1788619921.897615:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1e00/0x1cd7fe53687354c2 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7bb expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.897635:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619921.897637:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619921.897643:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1e00/0x1cd7fe53687354c2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7bb expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.897658:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed1e00, rc 0) 00000100:00100000:0.0:1788619921.897726:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b36680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+15:2352:x1875503571534720:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3940us (4417us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619921.897734:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1147 00000100:00100000:2.0:1788619921.899648:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fd938700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589157888:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619921.899687:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000073be6ea/0x1cd7fe53687354c9 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7c2 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.899720:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000073be6ea/0x1cd7fe53687354c9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7c2 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.899732:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7c00, rc 0) 00000100:00100000:2.0:1788619921.899846:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571534848:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4540us (4667us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619921.899855:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 917 00000100:00100000:3.0:1788619921.903505:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.903597:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571535104 00000100:00100000:2.0:1788619921.903607:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571535104 00000100:00100000:2.0:1788619921.903663:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1148 00000100:00100000:2.0:1788619921.903673:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b34a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+15:14908:x1875503571535104:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:2.0:1788619921.903735:0:11397:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619921.903760:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06e00/0x1cd7fe5368735421 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c721 expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.903777:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619921.903781:0:11397:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788619921.904019:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe53687354d0 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.904033:0:11397:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6e00/0x1cd7fe5368735436 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c72f expref: 25 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.904041:0:11397:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe53687354d0 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.904075:0:11397:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6e00/0x1cd7fe5368735436 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c72f expref: 25 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.904087:0:11397:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6e00/0x1cd7fe5368735436 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c72f expref: 25 pid: 11397 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619921.904100:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fd938700 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589158016:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619921.904233:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164a80 going to sleep for 11 seconds 00000100:00100000:3.0:1788619921.907939:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619921.908011:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571535232 00000100:00100000:0.0:1788619921.908019:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571535232 00010000:00010000:0.0:1788619921.908059:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6e00/0x1cd7fe5368735436 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c72f expref: 26 pid: 11397 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619921.908101:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 383 00000100:00100000:0.0:1788619921.908110:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8559850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+26:2355:x1875503571535232:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619921.908140:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619921.908147:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6e00/0x1cd7fe5368735436 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c72f expref: 26 pid: 11397 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.908160:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6e00/0x1cd7fe5368735436 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c72f expref: 26 pid: 11397 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.908181:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619921.908187:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe53687354d0 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.908197:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe53687354d0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.908205:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe53687354d0 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.908224:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619921.908280:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8559850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+26:2355:x1875503571535232:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 170us (350us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619921.908287:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 383 00010000:00010000:2.0:1788619921.910599:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6e00/0x1cd7fe5368735436 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c72f expref: 25 pid: 11397 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619921.910625:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fd938700 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589158016:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619921.910649:0:11397:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe53687354d0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.910660:0:11397:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe53687354d0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.910665:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe53687354d0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788619921.910799:0:11397:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967719 00000004:00010000:2.0:1788619921.910850:0:11397:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 994373 -> 994373 00000004:00010000:2.0:1788619921.910853:0:11397:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619921 -> 1788619921 00000004:00010000:2.0:1788619921.910856:0:11397:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619921 -> 1788619919 00000004:00010000:2.0:1788619921.910858:0:11397:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619921 -> 1788619921 00000004:00010000:2.0:1788619921.910860:0:11397:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1030 -> 1030 00000004:00010000:2.0:1788619921.911479:0:11397:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000037d4a8b2 reply state 00000000802ffbeb transno 4294967720 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe53687354d0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619921.911491:0:11397:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788619921.911930:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b34a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+17:14908:x1875503571535104:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 8255us (8430us total) trans 4294967720 rc 0/0 00000100:00100000:2.0:1788619921.911940:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1148 00000020:00080000:2.0:1788619921.911945:0:11397:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1788619921.915629:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619921.915767:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571535360 00000100:00100000:0.0:1788619921.915778:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571535360 00000100:00100000:0.0:1788619921.915844:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1149 00000100:00100000:0.0:1788619921.915855:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+17:14908:x1875503571535360:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619921.915878:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619921.915897:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe53687354d7 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c7c9 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.915960:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe536873547c lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c775 expref: 24 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.915976:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe53687354d7 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7c9 expref: 18 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.916008:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe536873547c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c775 expref: 24 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.916020:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe536873547c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c775 expref: 24 pid: 5844 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619921.916033:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6343800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589158144:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619921.916108:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619921.919315:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6343800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589158144:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619921.919387:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe53687354d7 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7c9 expref: 18 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.919406:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619921.919409:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619921.919413:0:5846:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe53687354d7 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7c9 expref: 18 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.919422:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed1c00, rc 0) 00000100:00100000:0.0:1788619921.919478:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+18:14908:x1875503571535360:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3625us (3858us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619921.919500:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1149 00000100:00100000:3.0:1788619921.920867:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619921.920955:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571535488 00000100:00100000:1.0:1788619921.920962:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571535488 00010000:00010000:1.0:1788619921.921006:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe536873547c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c775 expref: 25 pid: 5844 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619921.921043:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 384 00000100:00100000:1.0:1788619921.921054:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc0380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+25:2355:x1875503571535488:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619921.921069:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619921.921096:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe536873547c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c775 expref: 25 pid: 5844 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.921110:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe536873547c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c775 expref: 25 pid: 5844 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.921138:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe536873547c lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c775 expref: 25 pid: 5844 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.921154:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619921.921160:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe53687354d7 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7c9 expref: 17 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.921169:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe53687354d7 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7c9 expref: 17 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.921177:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe53687354d7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7c9 expref: 17 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619921.921213:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe53687354d7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7c9 expref: 17 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619921.921226:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4ba2d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589158272:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619921.921298:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7b40 going to sleep for 11 seconds 00010000:00010000:2.0:1788619921.923742:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe53687354d0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.923760:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe53687354d0 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.923766:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe53687354d0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.923772:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78cdcc6000 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:2.0:1788619921.923777:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe53687354d0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.923785:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe53687354d0 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.923801:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619921.923804:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788619921.923809:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe53687354d0 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.923815:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe53687354d0 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619921.924315:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4ba2d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589158272:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619921.924366:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619921.924430:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc0380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2355:x1875503571535488:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3378us (3569us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619921.924440:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 384 00000100:00100000:3.0:1788619921.926783:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.926871:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571535616 00000100:00100000:2.0:1788619921.926881:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571535616 00000100:00100000:2.0:1788619921.926948:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 918 00000100:00100000:2.0:1788619921.926959:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:14908:x1875503571535616:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619921.926987:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619921.927008:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fb640cb5/0x1cd7fe53687354de lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c7d0 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.927042:0:6698:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000dd8a44c0/0x1cd7fe53687354ad lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1519616->1544191) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7a6 expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.927052:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(6 7) 00010000:00010000:2.0:1788619921.927057:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(7 7) 00010000:00010000:2.0:1788619921.927062:0:6698:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fb640cb5/0x1cd7fe53687354de lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7d0 expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.927098:0:6698:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000dd8a44c0/0x1cd7fe53687354ad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1519616->1544191) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7a6 expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.927112:0:6698:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000dd8a44c0/0x1cd7fe53687354ad lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1519616->1544191) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7a6 expref: 6 pid: 6697 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619921.927124:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fd938700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589158400:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619921.927205:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164a80 going to sleep for 11 seconds 00000100:00100000:3.0:1788619921.930602:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.930777:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fd938700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589158400:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619921.930812:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000fb640cb5/0x1cd7fe53687354de lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7d0 expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.930834:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6e00, rc 0) 00000100:00100000:2.0:1788619921.930892:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:14908:x1875503571535616:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3935us (4117us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619921.930900:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 918 00000100:00100000:0.0:1788619921.931003:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571535744 00000100:00100000:0.0:1788619921.931010:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571535744 00010000:00010000:0.0:1788619921.931043:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000dd8a44c0/0x1cd7fe53687354ad lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1519616->1544191) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7a6 expref: 7 pid: 6697 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619921.931081:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 385 00000100:00100000:0.0:1788619921.931089:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6858380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571535744:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619921.931103:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619921.931126:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000dd8a44c0/0x1cd7fe53687354ad lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1519616->1544191) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7a6 expref: 7 pid: 6697 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.931138:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000dd8a44c0/0x1cd7fe53687354ad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1519616->1544191) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7a6 expref: 7 pid: 6697 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.931159:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd8a44c0/0x1cd7fe53687354ad lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1519616->1544191) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7a6 expref: 7 pid: 6697 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.931172:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(7 7) 00010000:00010000:0.0:1788619921.931176:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(7 7) 00010000:00010000:0.0:1788619921.931183:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000fb640cb5/0x1cd7fe53687354de lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7d0 expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.931194:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fb640cb5/0x1cd7fe53687354de lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7d0 expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.931245:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000fb640cb5/0x1cd7fe53687354de lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7d0 expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619921.931259:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6343800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589158528:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619921.931311:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619921.934517:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6343800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589158528:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619921.934553:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619921.934612:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6858380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571535744:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3521us (4018us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619921.934623:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 385 00000100:00100000:3.0:1788619921.936802:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.936919:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571535872 00000100:00100000:2.0:1788619921.936930:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571535872 00002000:00010000:2.0:1788619921.936979:0:5857:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:2.0:1788619921.936984:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c685b800 with x1875503571535872 00000100:00100000:2.0:1788619921.937020:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 736 00000100:00100000:2.0:1788619921.937031:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c685b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+17:2354:x1875503571535872:12345-192.168.201.5@tcp:10:fsx.0 00000004:00010000:2.0:1788619921.937402:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 994373 -> 566057 00000004:00010000:2.0:1788619921.937406:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619921 -> 1788619921 00000004:00010000:2.0:1788619921.937409:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619919 -> 1788619919 00000004:00010000:2.0:1788619921.937411:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619921 -> 1788619921 00000004:00010000:2.0:1788619921.937413:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1030 -> 1030 00000100:00100000:2.0:1788619921.937478:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c685b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+18:2354:x1875503571535872:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 449us (686us total) trans 4294967721 rc 0/0 00000100:00100000:2.0:1788619921.937485:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 736 00002000:00010000:2.0:1788619921.937489:0:5857:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:2.0:1788619921.937492:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c685b800 with x1875503571535872 00000100:00100000:3.0:1788619921.943388:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.943536:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571536000 00000100:00100000:2.0:1788619921.943546:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571536000 00002000:00010000:2.0:1788619921.943585:0:6701:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:2.0:1788619921.943590:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6858e00 with x1875503571536000 ext(0->18446744073709551615) 00002000:00010000:2.0:1788619921.943602:0:6701:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6858e00. 00000100:00100000:2.0:1788619921.943690:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 516 00000100:00100000:2.0:1788619921.943711:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6858e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571536000:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:2.0:1788619921.944223:0:6701:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1918315 -> 0 00002000:00010000:2.0:1788619921.944228:0:6701:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619921 -> 1788619921 00002000:00010000:2.0:1788619921.944231:0:6701:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619921 -> 1788619921 00002000:00010000:2.0:1788619921.944233:0:6701:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619921 -> 1788619921 00002000:00010000:2.0:1788619921.944236:0:6701:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 519 -> 519 00000100:00100000:2.0:1788619921.944306:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6858e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571536000:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 607us (927us total) trans 4294967452 rc 0/0 00000100:00100000:2.0:1788619921.944313:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 516 00002000:00010000:2.0:1788619921.944317:0:6701:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:2.0:1788619921.944320:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6858e00 with x1875503571536000 ext(0->18446744073709551615) 00002000:00010000:2.0:1788619921.944351:0:6701:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6858e00. 00000100:00100000:1.0:1788619921.947616:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619921.947762:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571536128 00000100:00100000:0.0:1788619921.947774:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571536128 00000100:00100000:0.0:1788619921.947840:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1150 00000100:00100000:0.0:1788619921.947850:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+18:14908:x1875503571536128:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619921.947876:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619921.947897:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1600/0x1cd7fe53687354e5 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c7d7 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.948005:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1e00/0x1cd7fe53687354ec lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.948022:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1e00/0x1cd7fe53687354ec lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.948034:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1e00/0x1cd7fe53687354ec lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619921.948086:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 566057 00000004:00200000:0.0:1788619921.948090:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619921.948110:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1e00/0x1cd7fe53687354ec lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.948138:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1600/0x1cd7fe53687354e5 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7d7 expref: 20 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.948156:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1e00/0x1cd7fe53687354ec lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7d7 expref: 19 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.948166:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619921.948168:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619921.948172:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed1e00, rc 0) 00000100:00100000:0.0:1788619921.948247:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+19:14908:x1875503571536128:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 398us (640us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619921.948254:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1150 00000100:00100000:3.0:1788619921.952110:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.952263:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571536384 00000100:00100000:2.0:1788619921.952274:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571536384 00000100:00100000:2.0:1788619921.952384:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1151 00000100:00100000:2.0:1788619921.952396:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c685bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+24:14908:x1875503571536384:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:2.0:1788619921.952679:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe53687354f3 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.952706:0:11397:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1e00/0x1cd7fe53687354ec lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7d7 expref: 18 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.952715:0:11397:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe53687354f3 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.952752:0:11397:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1e00/0x1cd7fe53687354ec lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7d7 expref: 18 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.952764:0:11397:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1e00/0x1cd7fe53687354ec lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7d7 expref: 18 pid: 5844 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619921.952778:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fd938700 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589158656:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619921.952988:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164a80 going to sleep for 11 seconds 00000100:00100000:1.0:1788619921.956117:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619921.956188:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571536512 00000100:00100000:0.0:1788619921.956197:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571536512 00010000:00010000:0.0:1788619921.956236:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1e00/0x1cd7fe53687354ec lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7d7 expref: 19 pid: 5844 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619921.956273:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 386 00000100:00100000:0.0:1788619921.956283:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba3100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+19:2355:x1875503571536512:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619921.956299:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619921.956306:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1e00/0x1cd7fe53687354ec lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7d7 expref: 19 pid: 5844 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.956374:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1e00/0x1cd7fe53687354ec lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7d7 expref: 19 pid: 5844 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.956396:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619921.956401:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe53687354f3 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.956412:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe53687354f3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.956419:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe53687354f3 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.956440:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619921.956487:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba3100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+19:2355:x1875503571536512:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 205us (379us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619921.956495:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 386 00010000:00010000:2.0:1788619921.957230:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1e00/0x1cd7fe53687354ec lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7d7 expref: 18 pid: 5844 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619921.957248:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fd938700 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589158656:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619921.957267:0:11397:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe53687354f3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.957277:0:11397:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe53687354f3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.957283:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe53687354f3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788619921.957440:0:11397:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967722 00000004:00010000:2.0:1788619921.957486:0:11397:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 566057 -> 566057 00000004:00010000:2.0:1788619921.957490:0:11397:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619921 -> 1788619921 00000004:00010000:2.0:1788619921.957493:0:11397:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619919 -> 1788619919 00000004:00010000:2.0:1788619921.957495:0:11397:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619921 -> 1788619921 00000004:00010000:2.0:1788619921.957497:0:11397:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1030 -> 1030 00000004:00010000:2.0:1788619921.957506:0:11397:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000087cd91cc reply state 00000000c96ba593 transno 4294967723 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe53687354f3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619921.957515:0:11397:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788619921.957617:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c685bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+27:14908:x1875503571536384:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 5224us (5516us total) trans 4294967723 rc 0/0 00000100:00100000:2.0:1788619921.957624:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1151 00000020:00080000:2.0:1788619921.957628:0:11397:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:2.0:1788619921.959687:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe53687354f3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.959714:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe53687354f3 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.959720:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe53687354f3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.959726:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78cdcc6000 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:2.0:1788619921.959730:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe53687354f3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.959738:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe53687354f3 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.959754:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619921.959757:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788619921.959762:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe53687354f3 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619921.959768:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe53687354f3 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619921.961005:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619921.961092:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571536640 00000100:00100000:0.0:1788619921.961101:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571536640 00000100:00100000:0.0:1788619921.961161:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1152 00000100:00100000:0.0:1788619921.961171:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c685b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+26:14908:x1875503571536640:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619921.961205:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619921.961225:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1600/0x1cd7fe53687354fa lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c7de expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.961252:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe53687354d7 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7c9 expref: 17 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.961280:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1600/0x1cd7fe53687354fa lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7de expref: 27 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.961313:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe53687354d7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7c9 expref: 17 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.961353:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe53687354d7 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7c9 expref: 17 pid: 5846 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619921.961368:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6343800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589158784:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619921.961451:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619921.963811:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6343800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589158784:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619921.963844:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1600/0x1cd7fe53687354fa lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7de expref: 27 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.963862:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619921.963864:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619921.963867:0:5846:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1600/0x1cd7fe53687354fa lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7de expref: 27 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.963876:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed1600, rc 0) 00000100:00100000:0.0:1788619921.963936:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c685b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+27:14908:x1875503571536640:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2765us (2938us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619921.963945:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1152 00000100:00100000:1.0:1788619921.967895:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619921.968121:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571536768 00000100:00100000:0.0:1788619921.968144:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571536768 00010000:00010000:0.0:1788619921.968184:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe53687354d7 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7c9 expref: 18 pid: 5846 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619921.968375:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 387 00000100:00100000:0.0:1788619921.968385:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba1500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+18:2355:x1875503571536768:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619921.968401:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619921.968427:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe53687354d7 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7c9 expref: 18 pid: 5846 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.968440:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe53687354d7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7c9 expref: 18 pid: 5846 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.968460:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe53687354d7 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7c9 expref: 18 pid: 5846 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.968477:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619921.968482:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1600/0x1cd7fe53687354fa lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7de expref: 26 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.968492:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1600/0x1cd7fe53687354fa lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7de expref: 26 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.968500:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1600/0x1cd7fe53687354fa lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7de expref: 26 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.968546:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1600/0x1cd7fe53687354fa lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7de expref: 26 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619921.968573:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6343800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589158912:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619921.968643:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619921.971296:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6343800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589158912:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619921.971342:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619921.971392:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba1500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+17:2355:x1875503571536768:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3007us (3506us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619921.971402:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 387 00000100:00100000:1.0:1788619921.973266:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619921.973464:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571536896 00000100:00100000:0.0:1788619921.973473:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571536896 00000100:00100000:0.0:1788619921.973535:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 919 00000100:00100000:0.0:1788619921.973545:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe000380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:14908:x1875503571536896:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619921.973561:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619921.973579:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c5c78e09/0x1cd7fe5368735501 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c7e5 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.973611:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000fb640cb5/0x1cd7fe53687354de lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7d0 expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.973620:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(7 7) 00010000:00010000:0.0:1788619921.973624:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(8 7) 00010000:00010000:0.0:1788619921.973630:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c5c78e09/0x1cd7fe5368735501 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7e5 expref: 7 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.973664:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000fb640cb5/0x1cd7fe53687354de lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7d0 expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.973693:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000fb640cb5/0x1cd7fe53687354de lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7d0 expref: 6 pid: 6698 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619921.973718:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6343800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589159040:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619921.973799:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619921.976856:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6343800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589159040:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619921.976890:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c5c78e09/0x1cd7fe5368735501 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7e5 expref: 7 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.976914:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed1400, rc 0) 00000100:00100000:0.0:1788619921.976978:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe000380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:14908:x1875503571536896:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3424us (3709us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619921.976987:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 919 00000100:00100000:3.0:1788619921.979456:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619921.979677:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571537024 00000100:00100000:0.0:1788619921.979687:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571537024 00010000:00010000:0.0:1788619921.979736:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000fb640cb5/0x1cd7fe53687354de lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7d0 expref: 7 pid: 6698 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619921.979783:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 388 00000100:00100000:0.0:1788619921.979792:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571537024:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619921.979811:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619921.979835:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000fb640cb5/0x1cd7fe53687354de lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7d0 expref: 7 pid: 6698 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.979866:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000fb640cb5/0x1cd7fe53687354de lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7d0 expref: 7 pid: 6698 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.979891:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fb640cb5/0x1cd7fe53687354de lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7d0 expref: 7 pid: 6698 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.979909:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(8 7) 00010000:00010000:0.0:1788619921.979913:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(8 7) 00010000:00010000:0.0:1788619921.979922:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000c5c78e09/0x1cd7fe5368735501 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7e5 expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.979932:0:5829:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c5c78e09/0x1cd7fe5368735501 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7e5 expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619921.979984:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000c5c78e09/0x1cd7fe5368735501 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7e5 expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619921.979997:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6343800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1875503589159168:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619921.980090:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619921.982566:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6343800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1875503589159168:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619921.982594:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619921.982646:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571537024:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 2854us (3199us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619921.982674:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 388 00000100:00100000:3.0:1788619921.985808:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619921.985880:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571537152 00000100:00100000:0.0:1788619921.985891:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571537152 00002000:00010000:0.0:1788619921.985942:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1566728->18446744073709551615) 00002000:00100000:0.0:1788619921.985947:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e7900d5d180 with x1875503571537152 ext(1566728->18446744073709551615) 00002000:00010000:0.0:1788619921.985960:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e7900d5d180. 00000100:00100000:0.0:1788619921.985997:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 517 00000100:00100000:0.0:1788619921.986007:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2352:x1875503571537152:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:0.0:1788619921.986217:0:6703:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 0 -> 1566728 00002000:00010000:0.0:1788619921.986221:0:6703:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619921 -> 1788619921 00002000:00010000:0.0:1788619921.986223:0:6703:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619921 -> 1788619921 00002000:00010000:0.0:1788619921.986224:0:6703:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619921 -> 1788619921 00002000:00010000:0.0:1788619921.986226:0:6703:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 519 -> 519 00000100:00100000:0.0:1788619921.986289:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2352:x1875503571537152:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 283us (490us total) trans 4294967453 rc 0/0 00000100:00100000:0.0:1788619921.986297:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 517 00002000:00010000:0.0:1788619921.986300:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1566728->18446744073709551615) 00002000:00100000:0.0:1788619921.986303:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e7900d5d180 with x1875503571537152 ext(1566728->18446744073709551615) 00002000:00010000:0.0:1788619921.986307:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e7900d5d180. 00000100:00100000:1.0:1788619921.992217:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619921.992265:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619921.992358:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571537280 00000100:00100000:2.0:1788619921.992369:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571537280 00002000:00100000:2.0:1788619921.992421:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1568767) req@ffff8e78fe001500 x1875503571537280/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:17/0 lens 488/0 e 0 to 0 dl 1788619932 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619921.992432:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fe001500 with x1875503571537280 ext(1048576->1568767) 00002000:00010000:2.0:1788619921.992438:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fe001500 00000100:00100000:2.0:1788619921.992475:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 518 00000100:00100000:2.0:1788619921.992485:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe001500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2352:x1875503571537280:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619921.995225:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571537408 00000100:00100000:3.0:1788619921.995239:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571537408 00002000:00100000:3.0:1788619921.995290:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c74aaa00 x1875503571537408/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:17/0 lens 488/0 e 0 to 0 dl 1788619932 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619921.995301:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c74aaa00 with x1875503571537408 00000100:00100000:3.0:1788619921.995370:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 737 00000100:00100000:3.0:1788619921.995379:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74aaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+27:2352:x1875503571537408:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619921.999049:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74aaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+27:2352:x1875503571537408:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 3659us (6772us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619921.999063:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 737 00002000:00100000:0.0:1788619921.999076:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c74aaa00 x1875503571537408/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:17/0 lens 488/440 e 0 to 0 dl 1788619932 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619921.999106:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c74aaa00 with x1875503571537408 00000100:00100000:2.0:1788619922.000683:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe001500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2352:x1875503571537280:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 8192us (8469us total) trans 0 rc 520192/520192 00000100:00100000:2.0:1788619922.000705:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 518 00002000:00100000:2.0:1788619922.000716:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1568767) req@ffff8e78fe001500 x1875503571537280/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:17/0 lens 488/440 e 0 to 0 dl 1788619932 ref 1 fl Complete:/600/0 rc 520192/520192 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619922.000727:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fe001500 with x1875503571537280 ext(1048576->1568767) 00002000:00010000:2.0:1788619922.000734:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fe001500 00000100:00100000:3.0:1788619922.004782:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619922.005364:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571537536 00000100:00100000:1.0:1788619922.005375:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571537536 00002000:00100000:1.0:1788619922.005490:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1552384->1566727) req@ffff8e77c71f3b80 x1875503571537536/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:18/0 lens 14832/0 e 0 to 0 dl 1788619933 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619922.005501:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c71f3b80 with x1875503571537536 ext(1552384->1566727) 00002000:00010000:1.0:1788619922.005506:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c71f3b80 00000100:00100000:1.0:1788619922.005550:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 519 00000100:00100000:1.0:1788619922.005561:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571537536:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:1.0:1788619922.006162:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+10:2353:x1875503571537536:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 602us (1388us total) trans 4294967454 rc 0/0 00000100:00100000:1.0:1788619922.006170:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 519 00002000:00100000:1.0:1788619922.006177:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1552384->1566727) req@ffff8e77c71f3b80 x1875503571537536/t4294967454(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:18/0 lens 14832/448 e 0 to 0 dl 1788619933 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619922.006185:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c71f3b80 with x1875503571537536 ext(1552384->1566727) 00002000:00010000:1.0:1788619922.006188:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c71f3b80 00000100:00080000:2.0:1788619922.007316:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1788619922.008849:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619922.008936:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571537664 00000100:00100000:0.0:1788619922.008945:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571537664 00000100:00100000:0.0:1788619922.009009:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 920 00000100:00100000:0.0:1788619922.009020:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74a8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+10:2354:x1875503571537664:12345-192.168.201.5@tcp:16:fsx.0 00000001:00080000:3.0:1788619922.042404:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967453 is committed 00000001:00080000:3.0:1788619922.042422:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967454 is committed 00000001:00080000:3.0:1788619922.042433:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967452 is committed 00000100:00100000:1.0:1788619922.042578:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74a8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571537664:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 33555us (33729us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619922.042591:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 920 00000100:00100000:3.0:1788619922.045004:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619922.045104:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571537792 00000100:00100000:1.0:1788619922.045113:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571537792 00000100:00100000:1.0:1788619922.045194:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1153 00000100:00100000:1.0:1788619922.045205:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7430a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+17:14908:x1875503571537792:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619922.045227:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619922.045247:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5deec00/0x1cd7fe5368735508 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c7ec expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.045365:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5deee00/0x1cd7fe536873550f lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.045382:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5deee00/0x1cd7fe536873550f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.045394:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5deee00/0x1cd7fe536873550f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788619922.045423:0:9417:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1566728 00000004:00200000:1.0:1788619922.045428:0:9417:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788619922.045445:0:9417:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5deee00/0x1cd7fe536873550f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.045473:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5deec00/0x1cd7fe5368735508 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7ec expref: 19 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.045490:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5deee00/0x1cd7fe536873550f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7ec expref: 18 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.045499:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619922.045502:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788619922.045506:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5deee00, rc 0) 00000100:00100000:1.0:1788619922.045572:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7430a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+18:14908:x1875503571537792:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 369us (575us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619922.045580:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1153 00000100:00080000:3.0:1788619922.045740:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788619922.047771:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619922.047916:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571537920 00000100:00100000:1.0:1788619922.047925:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571537920 00000100:00100000:1.0:1788619922.047977:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1154 00000100:00100000:1.0:1788619922.047985:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7430000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+18:2354:x1875503571537920:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619922.048001:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619922.048016:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5defa00/0x1cd7fe5368735516 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c7f3 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.048045:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1600/0x1cd7fe53687354fa lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7de expref: 26 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619922.048080:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c74a9f80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589159296:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619922.048139:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7b40 going to sleep for 11 seconds 00000100:00100000:1.0:1788619922.049718:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619922.049793:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571538048 00000100:00100000:2.0:1788619922.049801:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571538048 00000100:00100000:2.0:1788619922.049864:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 921 00000100:00100000:2.0:1788619922.049873:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74ab800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571538048:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619922.049886:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619922.049900:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d2409dc/0x1cd7fe536873551d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c7fa expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619922.049958:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fd938700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589159424:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619922.050043:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164c00 going to sleep for 11 seconds 00000100:00080000:3.0:1788619922.050230:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000004:00010000:1.0:1788619922.052591:0:11397:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619919 -> 1788619921 00000100:00100000:1.0:1788619922.052623:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c74a9f80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589159296:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619922.052645:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 566057 00010000:00010000:1.0:1788619922.052663:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5defa00/0x1cd7fe5368735516 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7f3 expref: 19 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.052683:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619922.052687:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619922.052692:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5defa00/0x1cd7fe5368735516 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7f3 expref: 19 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.052723:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5defa00, rc 0) 00000100:00100000:1.0:1788619922.052785:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7430000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+18:2354:x1875503571537920:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4799us (5019us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619922.052793:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1154 00000100:00100000:2.0:1788619922.053406:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fd938700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589159424:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619922.053429:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000d2409dc/0x1cd7fe536873551d lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7fa expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.053446:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d2409dc/0x1cd7fe536873551d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7fa expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.053456:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6000, rc 0) 00000100:00100000:2.0:1788619922.053488:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74ab800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571538048:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3617us (3775us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619922.053495:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 921 00000100:00100000:3.0:1788619922.057169:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619922.057221:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619922.057309:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571538304 00000100:00100000:2.0:1788619922.057319:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571538304 00000100:00100000:2.0:1788619922.057410:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 922 00000100:00100000:2.0:1788619922.057419:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7431c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571538304:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619922.057438:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619922.057455:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000073be6ea/0x1cd7fe5368735524 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c808 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619922.057513:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fd938700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589159552:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619922.057583:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164c00 going to sleep for 11 seconds 00000100:00100000:0.0:1788619922.057858:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571538176 00000100:00100000:0.0:1788619922.057863:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571538176 00000100:00100000:0.0:1788619922.057901:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1155 00000100:00100000:0.0:1788619922.057908:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7430380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+18:2353:x1875503571538176:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619922.057920:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619922.057933:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe536873552b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c801 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.057961:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1600/0x1cd7fe53687354fa lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7de expref: 26 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619922.057986:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6343800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589159680:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619922.058029:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebf00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619922.060771:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fd938700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589159552:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619922.060810:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000073be6ea/0x1cd7fe5368735524 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c808 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.060834:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000073be6ea/0x1cd7fe5368735524 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c808 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.060847:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7c00, rc 0) 00000100:00100000:2.0:1788619922.060904:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7431c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571538304:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3485us (3681us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619922.060911:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 922 00000100:00100000:0.0:1788619922.061244:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6343800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589159680:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619922.061260:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 566057 00010000:00010000:0.0:1788619922.061273:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe536873552b lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c801 expref: 19 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.061286:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619922.061289:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619922.061309:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe536873552b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c801 expref: 19 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.061320:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed1c00, rc 0) 00000100:00100000:0.0:1788619922.061392:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7430380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+18:2353:x1875503571538176:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3484us (4234us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619922.061399:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1155 00000100:00100000:0.0:1788619922.065949:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619922.066169:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571538432 00000100:00100000:1.0:1788619922.066180:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571538432 00000100:00100000:1.0:1788619922.066236:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1156 00000100:00100000:1.0:1788619922.066246:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6343800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+18:14908:x1875503571538432:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619922.066268:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619922.066288:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5defa00/0x1cd7fe5368735532 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c80f expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.066318:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1600/0x1cd7fe53687354fa lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7de expref: 26 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.066358:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5defa00/0x1cd7fe5368735532 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c80f expref: 19 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.066390:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1600/0x1cd7fe53687354fa lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7de expref: 26 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.066403:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1600/0x1cd7fe53687354fa lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7de expref: 26 pid: 5846 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619922.066414:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c74a9f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589159808:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619922.066498:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7b40 going to sleep for 11 seconds 00000100:00100000:2.0:1788619922.069356:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619922.069443:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c74a9f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589159808:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619922.069474:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5defa00/0x1cd7fe5368735532 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c80f expref: 19 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.069493:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619922.069496:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619922.069500:0:5845:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5defa00/0x1cd7fe5368735532 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c80f expref: 19 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.069511:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5defa00, rc 0) 00000100:00100000:0.0:1788619922.069626:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571538560 00000100:00100000:0.0:1788619922.069633:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571538560 00010000:00010000:0.0:1788619922.069672:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1600/0x1cd7fe53687354fa lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7de expref: 27 pid: 5846 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619922.069718:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 389 00000100:00100000:0.0:1788619922.069726:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd938700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+27:2354:x1875503571538560:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619922.069740:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619922.069762:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1600/0x1cd7fe53687354fa lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7de expref: 27 pid: 5846 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.069774:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1600/0x1cd7fe53687354fa lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7de expref: 27 pid: 5846 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.069794:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1600/0x1cd7fe53687354fa lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7de expref: 27 pid: 5846 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.069808:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619922.069812:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5defa00/0x1cd7fe5368735532 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c80f expref: 19 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.069840:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5defa00/0x1cd7fe5368735532 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c80f expref: 19 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.069849:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5defa00/0x1cd7fe5368735532 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c80f expref: 19 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.069881:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5defa00/0x1cd7fe5368735532 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c80f expref: 19 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619922.069893:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6342a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589159936:192.168.201.5@tcp:105: 00000100:00080000:3.0:1788619922.070243:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1788619922.070353:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebf00 going to sleep for 11 seconds 00000100:00100000:1.0:1788619922.070640:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6343800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+19:14908:x1875503571538432:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4392us (4696us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619922.070648:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1156 00000100:00100000:0.0:1788619922.074535:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619922.074647:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571538688 00000100:00100000:1.0:1788619922.074657:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571538688 00000100:00100000:1.0:1788619922.074726:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1157 00000100:00100000:1.0:1788619922.074737:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6341f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+27:14908:x1875503571538688:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619922.074758:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619922.074777:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5dee600/0x1cd7fe5368735539 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c816 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.074862:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5def800/0x1cd7fe5368735540 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.074878:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5def800/0x1cd7fe5368735540 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.074894:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5def800/0x1cd7fe5368735540 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788619922.074935:0:11397:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1566728 00000004:00200000:1.0:1788619922.074939:0:11397:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788619922.074955:0:11397:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5def800/0x1cd7fe5368735540 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.075098:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5dee600/0x1cd7fe5368735539 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c816 expref: 29 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.075116:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5def800/0x1cd7fe5368735540 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c816 expref: 28 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.075124:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619922.075126:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788619922.075129:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5def800, rc 0) 00000100:00100000:1.0:1788619922.075180:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6341f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+28:14908:x1875503571538688:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 447us (663us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619922.075186:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1157 00000100:00100000:3.0:1788619922.077758:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619922.077831:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571538816 00000100:00100000:0.0:1788619922.077841:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571538816 00000100:00100000:0.0:1788619922.077893:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1158 00000100:00100000:0.0:1788619922.077916:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7430700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+28:2354:x1875503571538816:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619922.077934:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619922.077952:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe5368735547 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c81d expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.077999:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5defa00/0x1cd7fe5368735532 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c80f expref: 18 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619922.078040:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c75f6680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589160064:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619922.078436:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebb40 going to sleep for 11 seconds 00000004:00010000:0.0:1788619922.080869:0:5846:0:(mdt_lvb.c:177:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size: 566057 -> 703072 00000100:00100000:0.0:1788619922.080902:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c75f6680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589160064:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619922.080928:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 703072 00010000:00010000:0.0:1788619922.080946:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe5368735547 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c81d expref: 29 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.080979:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619922.080982:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619922.080988:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe5368735547 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c81d expref: 29 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.081003:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed1c00, rc 0) 00000100:00100000:0.0:1788619922.081058:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7430700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+28:2354:x1875503571538816:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3157us (3309us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619922.081067:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1158 00000100:00100000:2.0:1788619922.082410:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6342a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589159936:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619922.082433:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619922.082482:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd938700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+27:2354:x1875503571538560:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 12755us (13163us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619922.082491:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 389 00000100:00100000:3.0:1788619922.086213:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619922.086301:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571538944 00000100:00100000:1.0:1788619922.086311:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571538944 00000100:00100000:1.0:1788619922.086395:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1159 00000100:00100000:1.0:1788619922.086404:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7432300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+27:2354:x1875503571538944:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619922.086424:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619922.086445:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5def400/0x1cd7fe536873554e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c824 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.086486:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5defa00/0x1cd7fe5368735532 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c80f expref: 18 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619922.086535:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c74a9f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589160192:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619922.086614:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7b40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619922.089286:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c74a9f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589160192:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619922.089314:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 703072 00010000:00010000:0.0:1788619922.089367:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5def400/0x1cd7fe536873554e lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c824 expref: 28 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.089388:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619922.089391:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619922.089397:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5def400/0x1cd7fe536873554e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c824 expref: 28 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.089410:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5def400, rc 0) 00000100:00100000:0.0:1788619922.089472:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7432300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+27:2354:x1875503571538944:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3069us (3268us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619922.089496:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1159 00000100:00100000:3.0:1788619922.092563:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619922.092906:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571539072 00000100:00100000:0.0:1788619922.092916:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571539072 00000100:00100000:0.0:1788619922.092983:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1160 00000100:00100000:0.0:1788619922.092993:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+27:14908:x1875503571539072:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619922.093014:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619922.093034:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe5368735555 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c82b expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.093062:0:9417:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5defa00/0x1cd7fe5368735532 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c80f expref: 18 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.093071:0:9417:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe5368735555 lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c82b expref: 28 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.093103:0:9417:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5defa00/0x1cd7fe5368735532 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c80f expref: 18 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.093116:0:9417:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5defa00/0x1cd7fe5368735532 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c80f expref: 18 pid: 5845 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619922.093128:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c75f6680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589160320:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619922.093201:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebb40 going to sleep for 11 seconds 00000100:00100000:3.0:1788619922.096192:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619922.096480:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571539200 00000100:00100000:2.0:1788619922.096490:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571539200 00002000:00100000:2.0:1788619922.096545:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be6300 x1875503571539200/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:18/0 lens 3144/0 e 0 to 0 dl 1788619933 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619922.096557:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be6300 with x1875503571539200 00010000:00010000:2.0:1788619922.096565:0:5859:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5defa00/0x1cd7fe5368735532 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c80f expref: 19 pid: 5845 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.096580:0:5859:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5defa00/0x1cd7fe5368735532 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c80f expref: 19 pid: 5845 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788619922.096588:0:5859:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e77c4be6300 00000100:00100000:2.0:1788619922.096620:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 29 00000100:00100000:2.0:1788619922.096630:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+19:2355:x1875503571539200:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:2.0:1788619922.097064:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 703072 -> 703072 00000004:00010000:2.0:1788619922.097068:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619921 -> 1788619921 00000004:00010000:2.0:1788619922.097071:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619921 -> 1788619919 00000004:00010000:2.0:1788619922.097073:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619921 -> 1788619921 00000004:00010000:2.0:1788619922.097075:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1030 -> 1030 00000100:00100000:1.0:1788619922.097245:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c75f6680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589160320:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619922.097275:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe5368735555 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c82b expref: 28 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.097290:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619922.097293:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619922.097298:0:9417:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe5368735555 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c82b expref: 28 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.097309:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed1c00, rc 0) 00000100:00100000:2.0:1788619922.097398:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+21:2355:x1875503571539200:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 768us (1213us total) trans 4294967724 rc 0/0 00000100:00100000:2.0:1788619922.097407:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 29 00002000:00100000:2.0:1788619922.097415:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be6300 x1875503571539200/t4294967724(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:18/0 lens 3144/448 e 0 to 0 dl 1788619933 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619922.097424:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be6300 with x1875503571539200 00010000:00010000:2.0:1788619922.097430:0:5859:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5defa00/0x1cd7fe5368735532 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c80f expref: 21 pid: 5845 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.097441:0:5859:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5defa00/0x1cd7fe5368735532 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c80f expref: 21 pid: 5845 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788619922.097447:0:5859:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e77c4be6300 00000100:00100000:1.0:1788619922.097875:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+28:14908:x1875503571539072:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4881us (5319us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619922.097883:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1160 00000100:00100000:2.0:1788619922.101138:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619922.101237:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571539328 00000100:00100000:1.0:1788619922.101247:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571539328 00010000:00010000:1.0:1788619922.101611:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5defa00/0x1cd7fe5368735532 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c80f expref: 21 pid: 5845 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619922.101685:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 390 00000100:00100000:1.0:1788619922.101713:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c855bc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+21:2353:x1875503571539328:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619922.101734:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619922.101759:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5defa00/0x1cd7fe5368735532 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c80f expref: 21 pid: 5845 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.101772:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5defa00/0x1cd7fe5368735532 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c80f expref: 21 pid: 5845 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.101793:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5defa00/0x1cd7fe5368735532 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c80f expref: 21 pid: 5845 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.101808:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619922.101814:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe5368735555 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c82b expref: 27 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.101826:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe5368735555 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c82b expref: 27 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.101834:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe5368735555 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c82b expref: 27 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.101873:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe5368735555 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c82b expref: 27 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619922.101887:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c74aa300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589160448:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619922.101989:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7480 going to sleep for 11 seconds 00000100:00100000:1.0:1788619922.104566:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c74aa300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589160448:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619922.104594:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619922.104639:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c855bc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+20:2353:x1875503571539328:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 2946us (3510us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619922.104649:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 390 00000100:00100000:0.0:1788619922.118364:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619922.119278:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571539456 00000100:00100000:3.0:1788619922.119304:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571539456 00002000:00100000:2.0:1788619922.119435:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c75f4000 x1875503571539456/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:18/0 lens 488/0 e 0 to 0 dl 1788619933 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619922.119452:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c75f4000 with x1875503571539456 00000100:00100000:2.0:1788619922.119489:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 738 00000100:00100000:2.0:1788619922.119498:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+28:2354:x1875503571539456:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619922.130959:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+28:2354:x1875503571539456:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 11448us (12627us total) trans 0 rc 704512/704512 00000100:00100000:3.0:1788619922.130980:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 738 00002000:00100000:3.0:1788619922.130993:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c75f4000 x1875503571539456/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:18/0 lens 488/440 e 0 to 0 dl 1788619933 ref 1 fl Complete:/600/0 rc 704512/704512 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619922.131002:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c75f4000 with x1875503571539456 00000100:00100000:0.0:1788619922.138849:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619922.138896:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619922.139005:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571539712 00000100:00100000:3.0:1788619922.139016:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571539712 00000100:00100000:3.0:1788619922.139075:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 923 00000100:00100000:3.0:1788619922.139087:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571539712:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619922.139105:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619922.139123:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003b84bbc9/0x1cd7fe536873555c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c839 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619922.139178:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be4000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589160576:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619922.139261:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c300 going to sleep for 11 seconds 00000100:00100000:2.0:1788619922.139439:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571539584 00000100:00100000:2.0:1788619922.139443:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571539584 00000100:00100000:2.0:1788619922.139481:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1161 00000100:00100000:2.0:1788619922.139489:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+20:2352:x1875503571539584:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619922.139502:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619922.139528:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe5368735563 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c832 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.139559:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe5368735563 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c832 expref: 21 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788619922.139571:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 703072 00010000:00010000:2.0:1788619922.139578:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe5368735563 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c832 expref: 21 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.139590:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619922.139592:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619922.139596:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7c00, rc 0) 00000100:00100000:2.0:1788619922.139623:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+21:2352:x1875503571539584:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 136us (784us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619922.139629:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1161 00002000:00010000:3.0:1788619922.142790:0:6697:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619921 -> 1788619922 00000100:00100000:3.0:1788619922.142820:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be4000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589160576:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619922.142856:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003b84bbc9/0x1cd7fe536873555c lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c839 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.142879:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b84bbc9/0x1cd7fe536873555c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c839 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.142912:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683de00, rc 0) 00000100:00100000:3.0:1788619922.143083:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571539712:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3996us (4184us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619922.143091:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 923 00000100:00100000:0.0:1788619922.146680:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619922.146830:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571539840 00000100:00100000:1.0:1788619922.146840:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571539840 00000100:00100000:1.0:1788619922.146896:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 924 00000100:00100000:1.0:1788619922.146915:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b16300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571539840:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619922.146931:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619922.146950:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c9114dd/0x1cd7fe536873556a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c840 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619922.147017:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c74aa300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589160704:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619922.147099:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e70c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619922.149675:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c74aa300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589160704:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619922.149720:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009c9114dd/0x1cd7fe536873556a lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c840 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.149744:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c9114dd/0x1cd7fe536873556a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c840 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.149756:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c7678c00, rc 0) 00000100:00100000:1.0:1788619922.149813:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b16300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571539840:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2908us (3141us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619922.149821:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 924 00000100:00100000:3.0:1788619922.154105:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619922.154207:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571540096 00000100:00100000:0.0:1788619922.154217:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571540096 00000100:00100000:0.0:1788619922.154277:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1162 00000100:00100000:0.0:1788619922.154288:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+28:14908:x1875503571540096:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:0.0:1788619922.154381:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619922.154408:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5def800/0x1cd7fe5368735540 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c816 expref: 28 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.154427:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619922.154433:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788619922.154800:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe5368735571 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.154816:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5deee00/0x1cd7fe536873550f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7ec expref: 20 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.154824:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe5368735571 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.154857:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5deee00/0x1cd7fe536873550f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7ec expref: 20 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.154871:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5deee00/0x1cd7fe536873550f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7ec expref: 20 pid: 9417 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619922.154884:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6531880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589160832:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619922.155020:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619922.159318:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6531880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589160832:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619922.159396:0:5845:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe5368735571 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619922.160655:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619922.161054:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571540224 00000100:00100000:2.0:1788619922.161063:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571540224 00010000:00010000:2.0:1788619922.161103:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5deee00/0x1cd7fe536873550f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7ec expref: 21 pid: 9417 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619922.161141:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 391 00000100:00100000:2.0:1788619922.161150:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+21:2355:x1875503571540224:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619922.161164:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619922.161170:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5deee00/0x1cd7fe536873550f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7ec expref: 21 pid: 9417 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.161183:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5deee00/0x1cd7fe536873550f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7ec expref: 21 pid: 9417 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.161203:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5deee00/0x1cd7fe536873550f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7ec expref: 21 pid: 9417 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.161220:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619922.161225:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe5368735571 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.161235:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe5368735571 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.161242:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe5368735571 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.161278:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619922.161357:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+20:2355:x1875503571540224:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 208us (706us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619922.161364:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 391 00010000:00010000:1.0:1788619922.161890:0:5845:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe5368735571 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.161900:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe5368735571 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788619922.162049:0:5845:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967725 00000004:00010000:1.0:1788619922.162100:0:5845:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 703072 -> 703072 00000004:00010000:1.0:1788619922.162104:0:5845:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619921 -> 1788619922 00000004:00010000:1.0:1788619922.162107:0:5845:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619919 -> 1788619919 00000004:00010000:1.0:1788619922.162109:0:5845:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619921 -> 1788619922 00000004:00010000:1.0:1788619922.162111:0:5845:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1030 -> 1030 00000004:00010000:1.0:1788619922.162118:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000bb2ddffd reply state 000000004f65ad77 transno 4294967726 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe5368735571 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619922.162127:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788619922.162220:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+30:14908:x1875503571540096:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 7933us (8123us total) trans 4294967726 rc 0/0 00000100:00100000:1.0:1788619922.162227:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1162 00000020:00080000:1.0:1788619922.162232:0:5845:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788619922.166319:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe5368735571 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.166359:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe5368735571 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.166368:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe5368735571 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.166374:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c5ed0400 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788619922.166379:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe5368735571 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.166388:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe5368735571 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.166405:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619922.166408:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788619922.166413:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe5368735571 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.166419:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe5368735571 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619922.171271:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619922.172184:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571540352 00000100:00100000:1.0:1788619922.172195:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571540352 00000100:00100000:1.0:1788619922.172261:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1163 00000100:00100000:1.0:1788619922.172272:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e02a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+29:14908:x1875503571540352:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619922.172296:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619922.172300:0:9417:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:1.0:1788619922.172382:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe5368735555 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c82b expref: 29 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.172402:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619922.172407:0:9417:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788619922.172420:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678c00/0x1cd7fe5368735578 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c847 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.172443:0:9417:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe5368735563 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c832 expref: 19 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.172452:0:9417:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678c00/0x1cd7fe5368735578 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c847 expref: 29 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.172484:0:9417:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe5368735563 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c832 expref: 19 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.172497:0:9417:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe5368735563 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c832 expref: 19 pid: 5846 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619922.172510:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c74aa300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589160960:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619922.172601:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e70c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619922.175934:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619922.176141:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c74aa300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589160960:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619922.176173:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678c00/0x1cd7fe5368735578 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c847 expref: 29 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.176195:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619922.176198:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619922.176203:0:9417:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678c00/0x1cd7fe5368735578 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c847 expref: 29 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.176214:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c7678c00, rc 0) 00000100:00100000:1.0:1788619922.176275:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e02a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+29:14908:x1875503571540352:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4004us (5013us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619922.176284:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1163 00000100:00100000:0.0:1788619922.176717:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571540480 00000100:00100000:0.0:1788619922.176725:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571540480 00010000:00010000:0.0:1788619922.176758:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe5368735563 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c832 expref: 20 pid: 5846 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619922.176789:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 392 00000100:00100000:0.0:1788619922.176798:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd938e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+20:2354:x1875503571540480:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619922.176811:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619922.176849:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe5368735563 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c832 expref: 20 pid: 5846 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.176861:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe5368735563 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c832 expref: 20 pid: 5846 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.176883:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7c00/0x1cd7fe5368735563 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c832 expref: 20 pid: 5846 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.176896:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619922.176900:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678c00/0x1cd7fe5368735578 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c847 expref: 28 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.176910:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678c00/0x1cd7fe5368735578 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c847 expref: 28 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.176919:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678c00/0x1cd7fe5368735578 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c847 expref: 28 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.176952:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678c00/0x1cd7fe5368735578 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c847 expref: 28 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619922.176975:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6531880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589161088:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619922.177045:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619922.180532:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6531880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589161088:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619922.180561:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619922.180617:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd938e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+19:2354:x1875503571540480:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3819us (4693us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619922.180638:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 392 00000100:00100000:3.0:1788619922.184854:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619922.184960:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571540608 00000100:00100000:2.0:1788619922.184979:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571540608 00002000:00010000:2.0:1788619922.185020:0:6701:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (2084447->18446744073709551615) 00002000:00100000:2.0:1788619922.185025:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e7900e02d80 with x1875503571540608 ext(2084447->18446744073709551615) 00002000:00010000:2.0:1788619922.185037:0:6701:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e7900e02d80. 00000100:00100000:2.0:1788619922.185079:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 520 00000100:00100000:2.0:1788619922.185089:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571540608:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:2.0:1788619922.185865:0:6701:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1566728 -> 2084447 00002000:00010000:2.0:1788619922.185871:0:6701:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619921 -> 1788619922 00002000:00010000:2.0:1788619922.185873:0:6701:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619922 -> 1788619921 00002000:00010000:2.0:1788619922.185875:0:6701:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619921 -> 1788619922 00002000:00010000:2.0:1788619922.185877:0:6701:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 519 -> 135 00000100:00100000:2.0:1788619922.185942:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571540608:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 854us (1096us total) trans 4294967455 rc 0/0 00000100:00100000:2.0:1788619922.185950:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 520 00002000:00010000:2.0:1788619922.185953:0:6701:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (2084447->18446744073709551615) 00002000:00100000:2.0:1788619922.186083:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e7900e02d80 with x1875503571540608 ext(2084447->18446744073709551615) 00002000:00010000:2.0:1788619922.186090:0:6701:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e7900e02d80. 00000100:00100000:3.0:1788619922.193363:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619922.193469:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571540736 00000100:00100000:1.0:1788619922.193480:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571540736 00002000:00100000:1.0:1788619922.193531:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1568768->2084863) req@ffff8e7900e01500 x1875503571540736/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:18/0 lens 488/0 e 0 to 0 dl 1788619933 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619922.193543:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e7900e01500 with x1875503571540736 ext(1568768->2084863) 00002000:00010000:1.0:1788619922.193549:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e7900e01500 00000100:00100000:1.0:1788619922.193573:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 521 00000100:00100000:1.0:1788619922.193585:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e01500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571540736:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619922.200260:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e01500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571540736:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 6667us (6936us total) trans 0 rc 516096/516096 00000100:00100000:1.0:1788619922.200275:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 521 00002000:00100000:1.0:1788619922.200289:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1568768->2084863) req@ffff8e7900e01500 x1875503571540736/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:18/0 lens 488/440 e 0 to 0 dl 1788619933 ref 1 fl Complete:/600/0 rc 516096/516096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619922.200301:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e7900e01500 with x1875503571540736 ext(1568768->2084863) 00002000:00010000:1.0:1788619922.200308:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e7900e01500 00000100:00100000:3.0:1788619922.205729:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619922.206006:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571540864 00000100:00100000:2.0:1788619922.206017:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571540864 00002000:00100000:2.0:1788619922.206066:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (2043904->2084446) req@ffff8e7900e02680 x1875503571540864/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:18/0 lens 488/0 e 0 to 0 dl 1788619933 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619922.206077:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e7900e02680 with x1875503571540864 ext(2043904->2084446) 00002000:00010000:2.0:1788619922.206083:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e7900e02680 00000100:00100000:2.0:1788619922.206119:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 522 00000100:00100000:2.0:1788619922.206130:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e02680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571540864:12345-192.168.201.5@tcp:4:fsx.0 00010000:00010000:1.0:1788619922.211298:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788619922.213718:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e02680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2355:x1875503571540864:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 7579us (7989us total) trans 4294967456 rc 0/0 00000100:00100000:2.0:1788619922.213732:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 522 00002000:00100000:2.0:1788619922.213744:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (2043904->2084446) req@ffff8e7900e02680 x1875503571540864/t4294967456(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:18/0 lens 488/448 e 0 to 0 dl 1788619933 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619922.213755:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e7900e02680 with x1875503571540864 ext(2043904->2084446) 00002000:00010000:2.0:1788619922.213760:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e7900e02680 00000100:00100000:3.0:1788619922.216834:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619922.216934:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571540992 00000100:00100000:1.0:1788619922.216943:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571540992 00000100:00100000:1.0:1788619922.217013:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 925 00000100:00100000:1.0:1788619922.217023:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e02300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571540992:12345-192.168.201.5@tcp:16:fsx.0 00000001:00080000:3.0:1788619922.249299:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967456 is committed 00000100:00100000:1.0:1788619922.249475:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e02300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571540992:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 32449us (32644us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619922.249487:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 925 00000100:00100000:3.0:1788619922.251818:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619922.252005:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571541120 00000100:00100000:2.0:1788619922.252015:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571541120 00000100:00100000:2.0:1788619922.252067:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1164 00000100:00100000:2.0:1788619922.252077:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e00a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+19:14908:x1875503571541120:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619922.252097:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619922.252115:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6e00/0x1cd7fe536873557f lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c84e expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.252193:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe5368735586 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.252208:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe5368735586 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.252220:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe5368735586 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788619922.252246:0:11397:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 2084447 00000004:00200000:2.0:1788619922.252250:0:11397:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788619922.252270:0:11397:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe5368735586 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.252296:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6e00/0x1cd7fe536873557f lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c84e expref: 21 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.252314:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe5368735586 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c84e expref: 20 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.252321:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619922.252358:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788619922.252362:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7000, rc 0) 00000100:00100000:2.0:1788619922.252409:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e00a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+20:14908:x1875503571541120:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 334us (600us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619922.252416:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1164 00000100:00100000:3.0:1788619922.255090:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619922.255136:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619922.255225:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571541248 00000100:00100000:1.0:1788619922.255233:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571541248 00000100:00100000:1.0:1788619922.255295:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1165 00000100:00100000:1.0:1788619922.255304:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa43800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+20:2354:x1875503571541248:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619922.255350:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619922.255371:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678000/0x1cd7fe536873558d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c855 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.255406:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678c00/0x1cd7fe5368735578 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c847 expref: 28 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619922.255447:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c74aa300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589161216:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619922.255527:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7d80 going to sleep for 11 seconds 00000100:00100000:2.0:1788619922.256300:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571541376 00000100:00100000:2.0:1788619922.256307:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571541376 00000100:00100000:2.0:1788619922.256363:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 926 00000100:00100000:2.0:1788619922.256371:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e6479880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571541376:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619922.256382:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619922.256393:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007527590a/0x1cd7fe5368735594 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c85c expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619922.256432:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fd939500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589161344:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619922.256481:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81646c0 going to sleep for 11 seconds 00000004:00010000:1.0:1788619922.258537:0:5845:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619919 -> 1788619922 00000100:00100000:1.0:1788619922.258566:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c74aa300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589161216:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619922.258598:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 703072 00010000:00010000:1.0:1788619922.258614:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678000/0x1cd7fe536873558d lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c855 expref: 21 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.258639:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619922.258642:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619922.258648:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678000/0x1cd7fe536873558d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c855 expref: 21 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.258663:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c7678000, rc 0) 00000100:00100000:1.0:1788619922.258738:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa43800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+20:2354:x1875503571541248:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3434us (3656us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619922.258746:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1165 00002000:00010000:2.0:1788619922.259236:0:6698:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619921 -> 1788619922 00002000:00010000:2.0:1788619922.259241:0:6698:0:(ofd_lvb.c:284:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks: 135 -> 263 00000100:00100000:2.0:1788619922.259257:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fd939500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589161344:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619922.259279:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007527590a/0x1cd7fe5368735594 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c85c expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.259296:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007527590a/0x1cd7fe5368735594 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c85c expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.259306:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6800, rc 0) 00000100:00100000:2.0:1788619922.259356:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e6479880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571541376:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2986us (4219us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619922.259364:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 926 00000100:00100000:3.0:1788619922.262501:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619922.262544:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619922.262571:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571541632 00000100:00100000:3.0:1788619922.262591:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571541632 00000100:00100000:3.0:1788619922.262643:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 927 00000100:00100000:3.0:1788619922.262654:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e6479c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571541632:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619922.262672:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619922.262689:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe536873559b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c86a expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619922.262783:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5018700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589161472:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619922.262860:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c000 going to sleep for 11 seconds 00000100:00100000:2.0:1788619922.263121:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571541504 00000100:00100000:2.0:1788619922.263126:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571541504 00000100:00100000:2.0:1788619922.263166:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1166 00000100:00100000:2.0:1788619922.263174:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e6479f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+20:2353:x1875503571541504:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619922.263185:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619922.263197:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7a00/0x1cd7fe53687355a2 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c863 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.263227:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678c00/0x1cd7fe5368735578 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c847 expref: 28 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619922.263251:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fd939500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589161600:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619922.263285:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81646c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619922.268742:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5018700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589161472:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619922.268779:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe536873559b lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c86a expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.268806:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe536873559b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c86a expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.268819:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c200, rc 0) 00000100:00100000:1.0:1788619922.268868:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e6479c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571541632:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6216us (6323us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619922.268877:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 927 00000100:00100000:2.0:1788619922.270769:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fd939500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589161600:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619922.270795:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 703072 00010000:00010000:2.0:1788619922.270813:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7a00/0x1cd7fe53687355a2 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c863 expref: 21 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.270832:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619922.270834:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619922.270841:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7a00/0x1cd7fe53687355a2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c863 expref: 21 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.270853:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7a00, rc 0) 00000100:00100000:2.0:1788619922.270898:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e6479f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+20:2353:x1875503571541504:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7725us (8407us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619922.273274:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619922.273384:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571541760 00000100:00100000:0.0:1788619922.273394:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571541760 00000100:00100000:0.0:1788619922.273445:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 928 00000100:00100000:0.0:1788619922.273452:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5019f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:14908:x1875503571541760:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619922.273467:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619922.273481:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd8a44c0/0x1cd7fe53687355a9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c871 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.273507:0:7654:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000c5c78e09/0x1cd7fe5368735501 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7e5 expref: 5 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.273514:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(8 7) 00010000:00010000:0.0:1788619922.273518:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(9 7) 00010000:00010000:0.0:1788619922.273521:0:7654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd8a44c0/0x1cd7fe53687355a9 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1069056->1085439] (req 1069056->1085439) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c871 expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619922.273538:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1166 00010000:00010000:0.0:1788619922.273548:0:7654:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000c5c78e09/0x1cd7fe5368735501 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7e5 expref: 5 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.273559:0:7654:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000c5c78e09/0x1cd7fe5368735501 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7e5 expref: 5 pid: 6697 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619922.273570:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6531880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589161728:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619922.273636:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619922.275844:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6531880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589161728:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619922.275881:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000dd8a44c0/0x1cd7fe53687355a9 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1069056->1085439] (req 1069056->1085439) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c871 expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.275901:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed1600, rc 0) 00000100:00100000:0.0:1788619922.275980:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5019f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:14908:x1875503571541760:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2527us (2713us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619922.275988:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 928 00000100:00100000:1.0:1788619922.277410:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619922.277535:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571541888 00000100:00100000:2.0:1788619922.277544:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571541888 00010000:00010000:2.0:1788619922.277581:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000c5c78e09/0x1cd7fe5368735501 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7e5 expref: 6 pid: 6697 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619922.277616:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 393 00000100:00100000:2.0:1788619922.277626:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74aa300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571541888:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619922.277642:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619922.277663:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000c5c78e09/0x1cd7fe5368735501 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7e5 expref: 6 pid: 6697 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.277677:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000c5c78e09/0x1cd7fe5368735501 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7e5 expref: 6 pid: 6697 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.277708:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c5c78e09/0x1cd7fe5368735501 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c7e5 expref: 6 pid: 6697 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.277725:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(9 7) 00010000:00010000:2.0:1788619922.277730:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(9 7) 00010000:00010000:2.0:1788619922.277737:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000dd8a44c0/0x1cd7fe53687355a9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1069056->1085439) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c871 expref: 5 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.277747:0:5831:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd8a44c0/0x1cd7fe53687355a9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1069056->1085439) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c871 expref: 5 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.277782:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000dd8a44c0/0x1cd7fe53687355a9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1069056->1085439) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c871 expref: 5 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619922.277796:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fd939500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5831:1875503589161856:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619922.277872:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81646c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619922.280823:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fd939500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5831:1875503589161856:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619922.280845:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619922.280894:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74aa300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571541888:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3270us (3491us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619922.280909:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 393 00000100:00100000:3.0:1788619922.282162:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619922.282251:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571542016 00000100:00100000:1.0:1788619922.282259:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571542016 00000100:00100000:1.0:1788619922.282371:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 523 00000100:00100000:1.0:1788619922.282383:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5018700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571542016:12345-192.168.201.5@tcp:22:fsx.0 00002000:00010000:1.0:1788619922.282672:0:6702:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 2084447 -> 2084447 00002000:00010000:1.0:1788619922.282676:0:6702:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619922 -> 1788619922 00002000:00010000:1.0:1788619922.282679:0:6702:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619922 -> 1788619921 00002000:00010000:1.0:1788619922.282681:0:6702:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619922 -> 1788619922 00002000:00010000:1.0:1788619922.282683:0:6702:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 263 -> 263 00000100:00100000:1.0:1788619922.282743:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5018700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571542016:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 363us (587us total) trans 4294967457 rc 0/0 00000100:00100000:1.0:1788619922.282750:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 523 00000100:00100000:1.0:1788619922.286045:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619922.286134:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571542144 00000100:00100000:3.0:1788619922.286144:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571542144 00000100:00100000:3.0:1788619922.286203:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1167 00000100:00100000:3.0:1788619922.286213:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74a8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+29:14908:x1875503571542144:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619922.286234:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619922.286253:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe53687355b0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c878 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.286378:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe53687355b7 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.286395:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe53687355b7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.286407:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe53687355b7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788619922.286432:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 2084447 00000004:00200000:3.0:1788619922.286436:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788619922.286452:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe53687355b7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.286479:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe53687355b0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c878 expref: 31 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.286527:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe53687355b7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c878 expref: 30 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.286536:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619922.286540:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788619922.286543:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d000, rc 0) 00000100:00100000:3.0:1788619922.286599:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74a8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+30:14908:x1875503571542144:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 386us (563us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619922.286606:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1167 00000100:00100000:3.0:1788619922.289000:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619922.289095:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571542272 00000100:00100000:1.0:1788619922.289105:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571542272 00000100:00100000:1.0:1788619922.289158:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 929 00000100:00100000:1.0:1788619922.289167:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df386300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571542272:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619922.289184:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619922.289207:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f0f99a64/0x1cd7fe53687355be lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c87f expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619922.289265:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c74a9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589161984:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619922.289400:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7d80 going to sleep for 11 seconds 00002000:00010000:1.0:1788619922.291559:0:6696:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619921 -> 1788619922 00000100:00100000:1.0:1788619922.291588:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c74a9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589161984:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619922.291622:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f0f99a64/0x1cd7fe53687355be lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c87f expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.291666:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f0f99a64/0x1cd7fe53687355be lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c87f expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.291680:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c7679000, rc 0) 00000100:00100000:1.0:1788619922.291753:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df386300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571542272:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2586us (2765us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619922.291761:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 929 00000100:00100000:2.0:1788619922.294862:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619922.294934:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571542400 00000100:00100000:3.0:1788619922.294942:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571542400 00000100:00100000:3.0:1788619922.294996:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 930 00000100:00100000:3.0:1788619922.295005:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd93b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571542400:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619922.295022:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619922.295036:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f980cda4/0x1cd7fe53687355c5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c886 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619922.295088:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df387100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589162112:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619922.295163:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48cd80 going to sleep for 11 seconds 00000100:00100000:2.0:1788619922.297036:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df387100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589162112:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619922.297067:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f980cda4/0x1cd7fe53687355c5 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c886 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.297089:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f980cda4/0x1cd7fe53687355c5 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c886 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.297101:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d200, rc 0) 00000100:00100000:2.0:1788619922.297177:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd93b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571542400:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2172us (2324us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619922.297186:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 930 00000100:00100000:1.0:1788619922.300270:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619922.300534:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571542528 00000100:00100000:3.0:1788619922.300545:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571542528 00000100:00100000:3.0:1788619922.300601:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 931 00000100:00100000:3.0:1788619922.300612:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74a9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571542528:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619922.300630:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619922.300663:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe53687355cc lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c88d expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619922.300732:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df384e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589162240:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619922.300846:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c300 going to sleep for 11 seconds 00000100:00100000:3.0:1788619922.303308:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df384e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589162240:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619922.303471:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe53687355cc lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c88d expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.303497:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe53687355cc lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c88d expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.303510:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c600, rc 0) 00000100:00100000:3.0:1788619922.303810:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74a9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571542528:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3198us (3547us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619922.303819:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 931 00000100:00100000:2.0:1788619922.306744:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619922.306914:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571542656 00000100:00100000:1.0:1788619922.306923:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571542656 00000100:00100000:1.0:1788619922.306988:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 932 00000100:00100000:1.0:1788619922.306998:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd93ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:14908:x1875503571542656:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619922.307015:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619922.307031:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000aeaf52b/0x1cd7fe53687355d3 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c894 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.307059:0:7654:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000dd8a44c0/0x1cd7fe53687355a9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1069056->1085439) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c871 expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.307069:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(9 7) 00010000:00010000:1.0:1788619922.307073:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(10 7) 00010000:00010000:1.0:1788619922.307079:0:7654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000aeaf52b/0x1cd7fe53687355d3 lrc: 4/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1921024->1925119] (req 1921024->1925119) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c894 expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.307115:0:7654:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000dd8a44c0/0x1cd7fe53687355a9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1069056->1085439) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c871 expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.307128:0:7654:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000dd8a44c0/0x1cd7fe53687355a9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1069056->1085439) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c871 expref: 6 pid: 7654 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619922.307142:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78cdcc0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589162368:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619922.307314:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7d80 going to sleep for 11 seconds 00000100:00100000:1.0:1788619922.310485:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619922.310559:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571542784 00000100:00100000:3.0:1788619922.310568:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571542784 00010000:00010000:3.0:1788619922.310606:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000dd8a44c0/0x1cd7fe53687355a9 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1069056->1085439) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c871 expref: 7 pid: 7654 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619922.310650:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 394 00000100:00100000:3.0:1788619922.310659:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6858380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571542784:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619922.310672:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619922.310702:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000dd8a44c0/0x1cd7fe53687355a9 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1069056->1085439) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c871 expref: 7 pid: 7654 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.310734:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000dd8a44c0/0x1cd7fe53687355a9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1069056->1085439) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c871 expref: 7 pid: 7654 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.310756:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(10 7) 00010000:00010000:3.0:1788619922.310762:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(10 7) 00010000:00010000:3.0:1788619922.310769:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000000aeaf52b/0x1cd7fe53687355d3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1921024->1925119) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c894 expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.310778:0:13330:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000aeaf52b/0x1cd7fe53687355d3 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1921024->1925119) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c894 expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.310813:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000000aeaf52b/0x1cd7fe53687355d3 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1921024->1925119) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c894 expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619922.310825:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df384e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589162496:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619922.310896:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c300 going to sleep for 11 seconds 00000100:00100000:3.0:1788619922.313138:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df384e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589162496:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619922.313159:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619922.313208:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6858380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571542784:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 2550us (2732us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619922.313218:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 394 00010000:00010000:0.0:1788619922.315072:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd8a44c0/0x1cd7fe53687355a9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1069056->1085439) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c871 expref: 6 pid: 7654 timeout: 514 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619922.315096:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78cdcc0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589162368:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619922.315123:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000aeaf52b/0x1cd7fe53687355d3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1921024->1925119) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c894 expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.315136:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c7678a00, rc 0) 00000100:00100000:1.0:1788619922.315315:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd93ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+6:14908:x1875503571542656:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8317us (8577us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619922.315347:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 932 00000100:00100000:2.0:1788619922.318206:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619922.318307:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571542912 00000100:00100000:0.0:1788619922.318317:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571542912 00002000:00100000:0.0:1788619922.318382:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1921024->1925119) req@ffff8e78fd939f80 x1875503571542912/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:18/0 lens 488/0 e 0 to 0 dl 1788619933 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619922.318415:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fd939f80 with x1875503571542912 ext(1921024->1925119) 00002000:00010000:0.0:1788619922.318421:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fd939f80 00000100:00100000:0.0:1788619922.318490:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 524 00000100:00100000:0.0:1788619922.318502:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd939f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571542912:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619922.318794:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd939f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571542912:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 294us (596us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619922.318802:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 524 00002000:00100000:0.0:1788619922.318809:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1921024->1925119) req@ffff8e78fd939f80 x1875503571542912/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:18/0 lens 488/4536 e 0 to 0 dl 1788619933 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619922.318817:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fd939f80 with x1875503571542912 ext(1921024->1925119) 00002000:00010000:0.0:1788619922.318821:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fd939f80 00000100:00100000:2.0:1788619922.322169:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619922.322253:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571543040 00000100:00100000:3.0:1788619922.322262:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571543040 00002000:00100000:3.0:1788619922.322310:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1925120->1929215) req@ffff8e78fd939500 x1875503571543040/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:18/0 lens 488/0 e 0 to 0 dl 1788619933 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619922.322367:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fd939500 with x1875503571543040 ext(1925120->1929215) 00002000:00010000:3.0:1788619922.322373:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fd939500 00000100:00100000:3.0:1788619922.322398:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 525 00000100:00100000:3.0:1788619922.322408:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd939500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571543040:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619922.322668:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd939500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571543040:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 260us (508us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619922.322676:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 525 00002000:00100000:3.0:1788619922.322683:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1925120->1929215) req@ffff8e78fd939500 x1875503571543040/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:18/0 lens 488/4536 e 0 to 0 dl 1788619933 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619922.322736:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fd939500 with x1875503571543040 ext(1925120->1929215) 00002000:00010000:3.0:1788619922.322740:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fd939500 00000100:00100000:2.0:1788619922.326389:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619922.326485:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571543168 00000100:00100000:3.0:1788619922.326496:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571543168 00002000:00100000:3.0:1788619922.326559:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1929216->1933311) req@ffff8e78fd938700 x1875503571543168/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:18/0 lens 488/0 e 0 to 0 dl 1788619933 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619922.326570:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fd938700 with x1875503571543168 ext(1929216->1933311) 00002000:00010000:3.0:1788619922.326575:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fd938700 00000100:00100000:3.0:1788619922.326608:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 526 00000100:00100000:3.0:1788619922.326618:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd938700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571543168:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619922.326896:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd938700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571543168:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 279us (516us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619922.326904:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 526 00002000:00100000:3.0:1788619922.326918:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1929216->1933311) req@ffff8e78fd938700 x1875503571543168/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:18/0 lens 488/4536 e 0 to 0 dl 1788619933 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619922.326926:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fd938700 with x1875503571543168 ext(1929216->1933311) 00002000:00010000:3.0:1788619922.326930:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fd938700 00000100:00100000:2.0:1788619922.329903:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619922.330009:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571543296 00000100:00100000:0.0:1788619922.330019:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571543296 00002000:00100000:0.0:1788619922.330061:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1933312->1937407) req@ffff8e78fd938380 x1875503571543296/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:18/0 lens 488/0 e 0 to 0 dl 1788619933 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619922.330071:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fd938380 with x1875503571543296 ext(1933312->1937407) 00002000:00010000:0.0:1788619922.330076:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fd938380 00000100:00100000:0.0:1788619922.330109:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 527 00000100:00100000:0.0:1788619922.330117:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd938380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571543296:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619922.330397:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd938380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571543296:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 280us (503us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619922.330405:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 527 00002000:00100000:0.0:1788619922.330413:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1933312->1937407) req@ffff8e78fd938380 x1875503571543296/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:18/0 lens 488/4536 e 0 to 0 dl 1788619933 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619922.330420:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fd938380 with x1875503571543296 ext(1933312->1937407) 00002000:00010000:0.0:1788619922.330423:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fd938380 00000100:00100000:2.0:1788619922.333381:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619922.333473:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571543424 00000100:00100000:1.0:1788619922.333484:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571543424 00002000:00100000:1.0:1788619922.333540:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1937408->1941503) req@ffff8e78fd939c00 x1875503571543424/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:18/0 lens 488/0 e 0 to 0 dl 1788619933 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619922.333550:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fd939c00 with x1875503571543424 ext(1937408->1941503) 00002000:00010000:1.0:1788619922.333556:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fd939c00 00000100:00100000:1.0:1788619922.333578:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 528 00000100:00100000:1.0:1788619922.333587:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd939c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571543424:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619922.333835:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd939c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571543424:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 248us (462us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619922.333843:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 528 00002000:00100000:1.0:1788619922.333850:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1937408->1941503) req@ffff8e78fd939c00 x1875503571543424/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:18/0 lens 488/4536 e 0 to 0 dl 1788619933 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619922.333858:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fd939c00 with x1875503571543424 ext(1937408->1941503) 00002000:00010000:1.0:1788619922.333861:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fd939c00 00000100:00100000:2.0:1788619922.337467:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619922.337557:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571543552 00000100:00100000:3.0:1788619922.337568:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571543552 00002000:00100000:3.0:1788619922.337616:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1941504->1945599) req@ffff8e78fd939880 x1875503571543552/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:18/0 lens 488/0 e 0 to 0 dl 1788619933 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619922.337628:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fd939880 with x1875503571543552 ext(1941504->1945599) 00002000:00010000:3.0:1788619922.337633:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fd939880 00000100:00100000:3.0:1788619922.337664:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 529 00000100:00100000:3.0:1788619922.337674:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd939880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571543552:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619922.337929:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd939880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571543552:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 255us (471us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619922.337938:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 529 00002000:00100000:3.0:1788619922.337945:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1941504->1945599) req@ffff8e78fd939880 x1875503571543552/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:18/0 lens 488/4536 e 0 to 0 dl 1788619933 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619922.337953:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fd939880 with x1875503571543552 ext(1941504->1945599) 00002000:00010000:3.0:1788619922.337956:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fd939880 00000100:00100000:2.0:1788619922.341459:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619922.341565:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571543680 00000100:00100000:0.0:1788619922.341575:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571543680 00002000:00100000:0.0:1788619922.341622:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1945600->1949695) req@ffff8e78fd93b100 x1875503571543680/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:18/0 lens 488/0 e 0 to 0 dl 1788619933 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619922.341634:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fd93b100 with x1875503571543680 ext(1945600->1949695) 00002000:00010000:0.0:1788619922.341639:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fd93b100 00000100:00100000:0.0:1788619922.341672:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 530 00000100:00100000:0.0:1788619922.341681:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd93b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571543680:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619922.342103:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd93b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571543680:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 421us (652us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619922.342112:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 530 00002000:00100000:0.0:1788619922.342119:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1945600->1949695) req@ffff8e78fd93b100 x1875503571543680/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:18/0 lens 488/4536 e 0 to 0 dl 1788619933 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619922.342126:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fd93b100 with x1875503571543680 ext(1945600->1949695) 00002000:00010000:0.0:1788619922.342130:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fd93b100 00000100:00100000:2.0:1788619922.345241:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619922.345629:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571543808 00000100:00100000:1.0:1788619922.345639:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571543808 00002000:00100000:1.0:1788619922.345684:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1949696->1953791) req@ffff8e78fd939180 x1875503571543808/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:18/0 lens 488/0 e 0 to 0 dl 1788619933 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619922.345704:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fd939180 with x1875503571543808 ext(1949696->1953791) 00002000:00010000:1.0:1788619922.345710:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fd939180 00000100:00100000:1.0:1788619922.345777:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 531 00000100:00100000:1.0:1788619922.345786:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd939180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571543808:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619922.346060:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd939180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571543808:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 273us (825us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619922.346067:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 531 00002000:00100000:1.0:1788619922.346073:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1949696->1953791) req@ffff8e78fd939180 x1875503571543808/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:18/0 lens 488/4536 e 0 to 0 dl 1788619933 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619922.346081:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fd939180 with x1875503571543808 ext(1949696->1953791) 00002000:00010000:1.0:1788619922.346085:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fd939180 00000100:00100000:2.0:1788619922.348955:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619922.349110:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571543936 00000100:00100000:0.0:1788619922.349119:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571543936 00002000:00100000:0.0:1788619922.349162:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1953792->1957887) req@ffff8e77c6342a00 x1875503571543936/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:18/0 lens 488/0 e 0 to 0 dl 1788619933 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619922.349174:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6342a00 with x1875503571543936 ext(1953792->1957887) 00002000:00010000:0.0:1788619922.349181:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6342a00 00000100:00100000:0.0:1788619922.349246:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 532 00000100:00100000:0.0:1788619922.349254:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6342a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571543936:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619922.349510:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6342a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571543936:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 256us (563us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619922.350277:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 532 00002000:00100000:0.0:1788619922.350286:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1953792->1957887) req@ffff8e77c6342a00 x1875503571543936/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:18/0 lens 488/4536 e 0 to 0 dl 1788619933 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619922.350296:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6342a00 with x1875503571543936 ext(1953792->1957887) 00002000:00010000:0.0:1788619922.350301:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6342a00 00000100:00100000:2.0:1788619922.352258:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619922.352374:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571544064 00000100:00100000:1.0:1788619922.352384:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571544064 00002000:00100000:1.0:1788619922.352427:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1957888->1961983) req@ffff8e78fed63b80 x1875503571544064/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:18/0 lens 488/0 e 0 to 0 dl 1788619933 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619922.352439:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fed63b80 with x1875503571544064 ext(1957888->1961983) 00002000:00010000:1.0:1788619922.352446:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fed63b80 00000100:00100000:1.0:1788619922.352471:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 533 00000100:00100000:1.0:1788619922.352480:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed63b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571544064:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619922.352731:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed63b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571544064:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 251us (482us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619922.352740:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 533 00002000:00100000:1.0:1788619922.352747:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1957888->1961983) req@ffff8e78fed63b80 x1875503571544064/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:18/0 lens 488/4536 e 0 to 0 dl 1788619933 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619922.352754:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fed63b80 with x1875503571544064 ext(1957888->1961983) 00002000:00010000:1.0:1788619922.352758:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fed63b80 00000100:00100000:2.0:1788619922.355813:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619922.356015:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571544192 00000100:00100000:1.0:1788619922.356024:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571544192 00002000:00100000:1.0:1788619922.356069:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1961984->1966079) req@ffff8e77c4be0000 x1875503571544192/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:18/0 lens 488/0 e 0 to 0 dl 1788619933 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619922.356080:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4be0000 with x1875503571544192 ext(1961984->1966079) 00002000:00010000:1.0:1788619922.356085:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4be0000 00000100:00100000:1.0:1788619922.356120:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 534 00000100:00100000:1.0:1788619922.356129:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571544192:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619922.356413:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571544192:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 283us (607us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619922.356421:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 534 00002000:00100000:1.0:1788619922.356428:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1961984->1966079) req@ffff8e77c4be0000 x1875503571544192/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:18/0 lens 488/4536 e 0 to 0 dl 1788619933 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619922.356436:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4be0000 with x1875503571544192 ext(1961984->1966079) 00002000:00010000:1.0:1788619922.356441:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4be0000 00000100:00100000:2.0:1788619922.358988:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619922.359085:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571544320 00000100:00100000:0.0:1788619922.359093:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571544320 00002000:00100000:0.0:1788619922.359137:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1966080->1970175) req@ffff8e77c4be0e00 x1875503571544320/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:18/0 lens 488/0 e 0 to 0 dl 1788619933 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619922.359148:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4be0e00 with x1875503571544320 ext(1966080->1970175) 00002000:00010000:0.0:1788619922.359153:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4be0e00 00000100:00100000:0.0:1788619922.359186:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 535 00000100:00100000:0.0:1788619922.359197:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571544320:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619922.359469:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571544320:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 274us (489us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619922.359477:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 535 00002000:00100000:0.0:1788619922.359484:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1966080->1970175) req@ffff8e77c4be0e00 x1875503571544320/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:18/0 lens 488/4536 e 0 to 0 dl 1788619933 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619922.359492:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4be0e00 with x1875503571544320 ext(1966080->1970175) 00002000:00010000:0.0:1788619922.359497:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4be0e00 00000100:00100000:2.0:1788619922.362817:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619922.362906:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571544448 00000100:00100000:3.0:1788619922.362916:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571544448 00002000:00100000:3.0:1788619922.362973:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1970176->1974271) req@ffff8e77c4be1c00 x1875503571544448/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:18/0 lens 488/0 e 0 to 0 dl 1788619933 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619922.362984:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4be1c00 with x1875503571544448 ext(1970176->1974271) 00002000:00010000:3.0:1788619922.363000:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4be1c00 00000100:00100000:3.0:1788619922.363034:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 536 00000100:00100000:3.0:1788619922.363043:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571544448:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619922.363295:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571544448:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 252us (487us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619922.363303:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 536 00002000:00100000:3.0:1788619922.363310:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1970176->1974271) req@ffff8e77c4be1c00 x1875503571544448/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:18/0 lens 488/4536 e 0 to 0 dl 1788619933 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619922.363318:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4be1c00 with x1875503571544448 ext(1970176->1974271) 00002000:00010000:3.0:1788619922.363321:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4be1c00 00000100:00100000:2.0:1788619922.366911:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619922.366958:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619922.367061:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571544576 00000100:00100000:1.0:1788619922.367071:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571544576 00000100:00100000:1.0:1788619922.367131:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1168 00000100:00100000:1.0:1788619922.367141:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+20:2354:x1875503571544576:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619922.367162:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619922.367184:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679e00/0x1cd7fe53687355da lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c89b expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.367220:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678c00/0x1cd7fe5368735578 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c847 expref: 29 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619922.367262:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4ba3100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589162624:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619922.367368:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e70c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619922.367636:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571544704 00000100:00100000:3.0:1788619922.367641:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571544704 00000100:00100000:3.0:1788619922.367678:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 933 00000100:00100000:3.0:1788619922.367686:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571544704:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619922.367704:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619922.367715:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe53687355e1 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c8a2 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.367732:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=1s v=(10 7) 00010000:00010000:3.0:1788619922.367736:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(10 7) 00010000:00010000:3.0:1788619922.367746:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe53687355e1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c8a2 expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.367765:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe53687355e1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c8a2 expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.367797:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c600, rc 0) 00000100:00100000:3.0:1788619922.367838:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571544704:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 152us (877us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619922.367844:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 933 00000100:00100000:1.0:1788619922.370983:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4ba3100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589162624:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619922.371008:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 703072 00010000:00010000:1.0:1788619922.371026:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679e00/0x1cd7fe53687355da lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c89b expref: 21 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.371044:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619922.371048:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619922.371053:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679e00/0x1cd7fe53687355da lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c89b expref: 21 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.371066:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c7679e00, rc 0) 00000100:00100000:1.0:1788619922.371126:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+20:2354:x1875503571544576:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3987us (4224us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619922.371137:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1168 00000100:00100000:2.0:1788619922.374043:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619922.374173:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571544832 00000100:00100000:1.0:1788619922.374181:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571544832 00000100:00100000:1.0:1788619922.374232:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1169 00000100:00100000:1.0:1788619922.374243:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6000a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+20:2353:x1875503571544832:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619922.374262:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619922.374281:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678800/0x1cd7fe53687355e8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c8a9 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.374316:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678c00/0x1cd7fe5368735578 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c847 expref: 29 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619922.374373:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4ba3100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589162752:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619922.374450:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e70c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619922.376421:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4ba3100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589162752:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619922.376444:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 703072 00010000:00010000:1.0:1788619922.376461:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678800/0x1cd7fe53687355e8 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c8a9 expref: 21 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.376480:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619922.376483:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619922.376489:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678800/0x1cd7fe53687355e8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c8a9 expref: 21 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.376503:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c7678800, rc 0) 00000100:00100000:1.0:1788619922.376564:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6000a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+20:2353:x1875503571544832:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2323us (2530us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619922.376573:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1169 00000100:00100000:2.0:1788619922.383455:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619922.383773:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571544960 00000100:00100000:3.0:1788619922.383785:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571544960 00000100:00100000:3.0:1788619922.383873:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1170 00000100:00100000:3.0:1788619922.383884:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6000e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+20:2354:x1875503571544960:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619922.383917:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619922.383938:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687355ef lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c8b0 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.384272:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678c00/0x1cd7fe5368735578 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c847 expref: 29 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619922.384315:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df384e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589162880:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619922.384659:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619922.387253:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df384e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589162880:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619922.387278:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 703072 00010000:00010000:3.0:1788619922.387295:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687355ef lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c8b0 expref: 21 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.387314:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619922.387317:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619922.387458:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687355ef lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c8b0 expref: 21 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.387474:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683da00, rc 0) 00000100:00100000:3.0:1788619922.387539:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6000e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+20:2354:x1875503571544960:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3657us (4095us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619922.387548:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1170 00000100:00100000:2.0:1788619922.390949:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619922.391084:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571545088 00000100:00100000:1.0:1788619922.391093:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571545088 00002000:00100000:1.0:1788619922.391141:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1511424->1515519) req@ffff8e78dfa14000 x1875503571545088/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:18/0 lens 488/0 e 0 to 0 dl 1788619933 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619922.391153:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa14000 with x1875503571545088 ext(1511424->1515519) 00002000:00010000:1.0:1788619922.391159:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa14000 00000100:00100000:1.0:1788619922.391193:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 537 00000100:00100000:1.0:1788619922.391202:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa14000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2352:x1875503571545088:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619922.391428:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa14000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2352:x1875503571545088:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 226us (487us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619922.391435:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 537 00002000:00100000:1.0:1788619922.391442:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1511424->1515519) req@ffff8e78dfa14000 x1875503571545088/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:18/0 lens 488/4536 e 0 to 0 dl 1788619933 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619922.391450:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa14000 with x1875503571545088 ext(1511424->1515519) 00002000:00010000:1.0:1788619922.391454:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa14000 00000100:00100000:2.0:1788619922.394827:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619922.395084:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571545216 00000100:00100000:3.0:1788619922.395094:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571545216 00002000:00100000:3.0:1788619922.395140:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1515520->1519615) req@ffff8e78dfa17800 x1875503571545216/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:18/0 lens 488/0 e 0 to 0 dl 1788619933 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619922.395152:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa17800 with x1875503571545216 ext(1515520->1519615) 00002000:00010000:3.0:1788619922.395159:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa17800 00000100:00100000:3.0:1788619922.395186:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 538 00000100:00100000:3.0:1788619922.395196:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa17800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571545216:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619922.395413:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa17800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571545216:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 217us (597us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619922.395420:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 538 00002000:00100000:3.0:1788619922.395428:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1515520->1519615) req@ffff8e78dfa17800 x1875503571545216/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:18/0 lens 488/4536 e 0 to 0 dl 1788619933 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619922.395435:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa17800 with x1875503571545216 ext(1515520->1519615) 00002000:00010000:3.0:1788619922.395438:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa17800 00000100:00100000:2.0:1788619922.397850:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619922.397914:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571545344 00000100:00100000:3.0:1788619922.397924:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571545344 00002000:00100000:3.0:1788619922.397973:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1519616->1523711) req@ffff8e78dfa15880 x1875503571545344/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:18/0 lens 488/0 e 0 to 0 dl 1788619933 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619922.398003:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa15880 with x1875503571545344 ext(1519616->1523711) 00002000:00010000:3.0:1788619922.398008:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa15880 00000100:00100000:3.0:1788619922.398039:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 539 00000100:00100000:3.0:1788619922.398047:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa15880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2352:x1875503571545344:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619922.398227:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa15880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2352:x1875503571545344:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 180us (384us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619922.398235:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 539 00002000:00100000:3.0:1788619922.398241:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1519616->1523711) req@ffff8e78dfa15880 x1875503571545344/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:18/0 lens 488/4536 e 0 to 0 dl 1788619933 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619922.398251:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78dfa15880 with x1875503571545344 ext(1519616->1523711) 00002000:00010000:3.0:1788619922.398254:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78dfa15880 00000100:00100000:2.0:1788619922.400954:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619922.401044:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571545472 00000100:00100000:1.0:1788619922.401055:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571545472 00002000:00100000:1.0:1788619922.401101:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1523712->1527807) req@ffff8e78c74f5880 x1875503571545472/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:18/0 lens 488/0 e 0 to 0 dl 1788619933 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619922.401112:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c74f5880 with x1875503571545472 ext(1523712->1527807) 00002000:00010000:1.0:1788619922.401118:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c74f5880 00000100:00100000:1.0:1788619922.401152:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 540 00000100:00100000:1.0:1788619922.401162:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571545472:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619922.401383:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571545472:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 222us (437us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619922.401391:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 540 00002000:00100000:1.0:1788619922.401397:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1523712->1527807) req@ffff8e78c74f5880 x1875503571545472/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:18/0 lens 488/4536 e 0 to 0 dl 1788619933 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619922.401404:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c74f5880 with x1875503571545472 ext(1523712->1527807) 00002000:00010000:1.0:1788619922.401407:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c74f5880 00000100:00100000:2.0:1788619922.404413:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619922.404519:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571545600 00000100:00100000:1.0:1788619922.404529:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571545600 00002000:00100000:1.0:1788619922.404575:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1527808->1531903) req@ffff8e78c74f5500 x1875503571545600/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:18/0 lens 488/0 e 0 to 0 dl 1788619933 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619922.404586:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c74f5500 with x1875503571545600 ext(1527808->1531903) 00002000:00010000:1.0:1788619922.404592:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c74f5500 00000100:00100000:1.0:1788619922.404626:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 541 00000100:00100000:1.0:1788619922.404635:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571545600:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619922.404849:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571545600:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 214us (444us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619922.404857:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 541 00002000:00100000:1.0:1788619922.404864:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1527808->1531903) req@ffff8e78c74f5500 x1875503571545600/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:18/0 lens 488/4536 e 0 to 0 dl 1788619933 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619922.404872:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c74f5500 with x1875503571545600 ext(1527808->1531903) 00002000:00010000:1.0:1788619922.404875:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c74f5500 00000100:00100000:2.0:1788619922.408682:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619922.408768:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571545728 00000100:00100000:3.0:1788619922.408779:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571545728 00002000:00100000:3.0:1788619922.408824:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1531904->1535999) req@ffff8e78c74f7480 x1875503571545728/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:18/0 lens 488/0 e 0 to 0 dl 1788619933 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619922.408835:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c74f7480 with x1875503571545728 ext(1531904->1535999) 00002000:00010000:3.0:1788619922.408840:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c74f7480 00000100:00100000:3.0:1788619922.408874:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 542 00000100:00100000:3.0:1788619922.408884:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2352:x1875503571545728:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619922.409088:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2352:x1875503571545728:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 204us (414us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619922.409095:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 542 00002000:00100000:3.0:1788619922.409116:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1531904->1535999) req@ffff8e78c74f7480 x1875503571545728/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:18/0 lens 488/4536 e 0 to 0 dl 1788619933 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619922.409124:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c74f7480 with x1875503571545728 ext(1531904->1535999) 00002000:00010000:3.0:1788619922.409128:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c74f7480 00000100:00100000:0.0:1788619922.412196:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619922.412297:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571545856 00000100:00100000:2.0:1788619922.412307:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571545856 00002000:00100000:2.0:1788619922.412391:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1536000->1540095) req@ffff8e77c6531880 x1875503571545856/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:18/0 lens 488/0 e 0 to 0 dl 1788619933 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619922.412402:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6531880 with x1875503571545856 ext(1536000->1540095) 00002000:00010000:2.0:1788619922.412409:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6531880 00000100:00100000:2.0:1788619922.412439:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 543 00000100:00100000:2.0:1788619922.412448:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6531880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571545856:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619922.412664:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6531880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571545856:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 217us (477us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619922.412672:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 543 00002000:00100000:2.0:1788619922.412678:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1536000->1540095) req@ffff8e77c6531880 x1875503571545856/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:18/0 lens 488/4536 e 0 to 0 dl 1788619933 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619922.412686:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6531880 with x1875503571545856 ext(1536000->1540095) 00002000:00010000:2.0:1788619922.412709:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6531880 00000100:00100000:2.0:1788619922.416413:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619922.416541:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571545984 00000100:00100000:1.0:1788619922.416551:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571545984 00002000:00100000:1.0:1788619922.416601:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1540096->1544191) req@ffff8e78c74f6a00 x1875503571545984/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:18/0 lens 488/0 e 0 to 0 dl 1788619933 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619922.416613:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c74f6a00 with x1875503571545984 ext(1540096->1544191) 00002000:00010000:1.0:1788619922.416618:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c74f6a00 00000100:00100000:1.0:1788619922.416650:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 544 00000100:00100000:1.0:1788619922.416660:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2353:x1875503571545984:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619922.416897:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2353:x1875503571545984:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 238us (492us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619922.416905:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 544 00002000:00100000:1.0:1788619922.416911:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1540096->1544191) req@ffff8e78c74f6a00 x1875503571545984/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:18/0 lens 488/4536 e 0 to 0 dl 1788619933 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619922.416919:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c74f6a00 with x1875503571545984 ext(1540096->1544191) 00002000:00010000:1.0:1788619922.416922:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c74f6a00 00000100:00100000:1.0:1788619922.420261:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619922.420361:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571546112 00000100:00100000:0.0:1788619922.420371:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571546112 00000100:00100000:0.0:1788619922.420423:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1171 00000100:00100000:0.0:1788619922.420432:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+20:2354:x1875503571546112:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619922.420452:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619922.420474:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe53687355f6 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c8b7 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.420511:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678c00/0x1cd7fe5368735578 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c847 expref: 29 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619922.420565:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78cdcc0380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589163008:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619922.420649:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb000 going to sleep for 11 seconds 00000100:00100000:1.0:1788619922.423188:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78cdcc0380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589163008:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619922.423216:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 703072 00010000:00010000:1.0:1788619922.423235:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe53687355f6 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c8b7 expref: 21 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.423255:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619922.423258:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619922.423263:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe53687355f6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c8b7 expref: 21 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.423279:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed1400, rc 0) 00000100:00100000:1.0:1788619922.423384:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+20:2354:x1875503571546112:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2952us (3132us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619922.423393:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1171 00000100:00100000:2.0:1788619922.426018:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619922.426087:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571546240 00000100:00100000:1.0:1788619922.426097:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571546240 00000100:00100000:1.0:1788619922.426149:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1172 00000100:00100000:1.0:1788619922.426159:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df964a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+20:2355:x1875503571546240:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619922.426178:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619922.426197:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679e00/0x1cd7fe53687355fd lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c8be expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.426231:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678c00/0x1cd7fe5368735578 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c847 expref: 29 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619922.426268:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4ba1500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589163136:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619922.426363:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7b40 going to sleep for 11 seconds 00000100:00100000:1.0:1788619922.428774:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4ba1500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589163136:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619922.428797:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 703072 00010000:00010000:1.0:1788619922.428814:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679e00/0x1cd7fe53687355fd lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c8be expref: 21 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.428833:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619922.428836:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619922.428841:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7679e00/0x1cd7fe53687355fd lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c8be expref: 21 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.428855:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c7679e00, rc 0) 00000100:00100000:1.0:1788619922.428927:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df964a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+20:2355:x1875503571546240:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2768us (2916us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619922.428935:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1172 00000100:00100000:0.0:1788619922.432308:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619922.432427:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571546368 00000100:00100000:3.0:1788619922.432437:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571546368 00002000:00100000:3.0:1788619922.432487:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78cdcc0380 x1875503571546368/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:18/0 lens 488/0 e 0 to 0 dl 1788619933 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619922.432499:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78cdcc0380 with x1875503571546368 00000100:00100000:3.0:1788619922.432547:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 739 00000100:00100000:3.0:1788619922.432557:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+30:2354:x1875503571546368:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619922.432843:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+30:2354:x1875503571546368:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 287us (545us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619922.432851:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 739 00002000:00100000:3.0:1788619922.432858:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78cdcc0380 x1875503571546368/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:18/0 lens 488/440 e 0 to 0 dl 1788619933 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619922.432865:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78cdcc0380 with x1875503571546368 00000100:00100000:2.0:1788619922.435738:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619922.435879:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571546496 00000100:00100000:3.0:1788619922.435888:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571546496 00000100:00100000:3.0:1788619922.435940:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 934 00000100:00100000:3.0:1788619922.435949:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df967800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:14908:x1875503571546496:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619922.435976:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619922.435995:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001391204f/0x1cd7fe5368735604 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c8c5 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.436052:0:6698:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000000aeaf52b/0x1cd7fe53687355d3 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1921024->1925119) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c894 expref: 7 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.436062:0:6698:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe53687355e1 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c8a2 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.436070:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(10 7) 00010000:00010000:3.0:1788619922.436405:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(10 7) 00010000:00010000:3.0:1788619922.436414:0:6698:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001391204f/0x1cd7fe5368735604 lrc: 5/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [1048576->1085439] (req 1048576->1085439) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c8c5 expref: 7 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.436451:0:6698:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe53687355e1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c8a2 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.436466:0:6698:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe53687355e1 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c8a2 expref: 6 pid: 6696 timeout: 515 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619922.436479:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df384e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589163264:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619922.436923:0:6698:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000000aeaf52b/0x1cd7fe53687355d3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1921024->1925119) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c894 expref: 7 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.436934:0:6698:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000000aeaf52b/0x1cd7fe53687355d3 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1921024->1925119) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c894 expref: 7 pid: 7654 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619922.436959:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df387100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589163392:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619922.436986:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c300 going to sleep for 11 seconds 00000100:00100000:3.0:1788619922.439981:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df384e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589163264:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619922.439999:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df387100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589163392:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619922.440029:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001391204f/0x1cd7fe5368735604 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [1048576->1085439] (req 1048576->1085439) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c8c5 expref: 7 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.440049:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683da00, rc 0) 00000100:00100000:3.0:1788619922.440166:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df967800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:14908:x1875503571546496:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4217us (4435us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619922.440175:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 934 00000100:00100000:2.0:1788619922.442078:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619922.442188:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619922.442938:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571546752 00000100:00100000:3.0:1788619922.442952:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571546752 00010000:00010000:3.0:1788619922.443003:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000000aeaf52b/0x1cd7fe53687355d3 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1921024->1925119) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c894 expref: 7 pid: 7654 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619922.443044:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571546624 00000100:00100000:3.0:1788619922.443048:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571546624 00010000:00010000:3.0:1788619922.443063:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe53687355e1 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c8a2 expref: 7 pid: 6696 timeout: 515 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619922.443085:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 395 00000100:00100000:3.0:1788619922.443094:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df965f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571546752:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619922.443124:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619922.443147:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000000aeaf52b/0x1cd7fe53687355d3 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1921024->1925119) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c894 expref: 7 pid: 7654 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.443159:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000000aeaf52b/0x1cd7fe53687355d3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1921024->1925119) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c894 expref: 7 pid: 7654 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.443180:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000aeaf52b/0x1cd7fe53687355d3 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1921024->1925119) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c894 expref: 7 pid: 7654 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.443198:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619922.443246:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df965f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571546752:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 152us (1176us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619922.443253:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 395 00000100:00100000:3.0:1788619922.443269:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 396 00000100:00100000:3.0:1788619922.443276:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df966a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571546624:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619922.443284:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619922.443292:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe53687355e1 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c8a2 expref: 7 pid: 6696 timeout: 515 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.443303:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe53687355e1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c8a2 expref: 7 pid: 6696 timeout: 515 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.443314:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe53687355e1 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c8a2 expref: 7 pid: 6696 timeout: 515 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.443372:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(10 7) 00010000:00010000:3.0:1788619922.443377:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(10 7) 00010000:00010000:3.0:1788619922.443386:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000001391204f/0x1cd7fe5368735604 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1085439) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c8c5 expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.443397:0:5831:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001391204f/0x1cd7fe5368735604 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1085439) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c8c5 expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.443432:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000001391204f/0x1cd7fe5368735604 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1085439) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c8c5 expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619922.443446:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df387100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5831:1875503589163520:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619922.443499:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c300 going to sleep for 11 seconds 00000100:00100000:2.0:1788619922.451179:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619922.451309:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571546880 00000100:00100000:0.0:1788619922.451319:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571546880 00002000:00100000:0.0:1788619922.451386:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1081344->1085439) req@ffff8e78df965500 x1875503571546880/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:18/0 lens 488/0 e 0 to 0 dl 1788619933 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619922.451397:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df965500 with x1875503571546880 ext(1081344->1085439) 00002000:00010000:0.0:1788619922.451402:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df965500 00000100:00100000:0.0:1788619922.451435:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 545 00000100:00100000:0.0:1788619922.451444:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df965500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571546880:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619922.451751:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df965500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571546880:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 306us (581us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619922.451763:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 545 00002000:00100000:0.0:1788619922.451770:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1081344->1085439) req@ffff8e78df965500 x1875503571546880/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:18/0 lens 488/4536 e 0 to 0 dl 1788619933 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619922.451778:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df965500 with x1875503571546880 ext(1081344->1085439) 00002000:00010000:0.0:1788619922.451781:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df965500 00000100:00100000:3.0:1788619922.452276:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df387100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5831:1875503589163520:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619922.452566:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619922.452724:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df966a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571546624:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 9447us (10534us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619922.452734:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 396 00000100:00100000:2.0:1788619922.457160:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619922.457444:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619922.457545:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571547136 00000100:00100000:1.0:1788619922.457554:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571547136 00000100:00100000:1.0:1788619922.457617:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 935 00000100:00100000:1.0:1788619922.457625:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571547136:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619922.457644:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619922.457660:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003fefefac/0x1cd7fe536873560b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c8d3 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619922.457727:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4ba1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589163648:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619922.457797:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7b40 going to sleep for 11 seconds 00000100:00100000:3.0:1788619922.458432:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571547008 00000100:00100000:3.0:1788619922.458439:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571547008 00000100:00100000:3.0:1788619922.458812:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1173 00000100:00100000:3.0:1788619922.458822:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df964e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+20:2352:x1875503571547008:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619922.458835:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619922.458860:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368735612 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c8cc expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.458908:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678c00/0x1cd7fe5368735578 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c847 expref: 29 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619922.458938:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df387100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589163776:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619922.459176:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c300 going to sleep for 11 seconds 00000004:00010000:3.0:1788619922.462925:0:5846:0:(mdt_lvb.c:177:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size: 703072 -> 1048576 00000100:00100000:3.0:1788619922.463021:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df387100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589163776:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619922.463044:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:3.0:1788619922.463065:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368735612 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c8cc expref: 21 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.463084:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619922.463087:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619922.463092:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368735612 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c8cc expref: 21 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.463107:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c600, rc 0) 00000100:00100000:3.0:1788619922.463227:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df964e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+20:2352:x1875503571547008:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4405us (6078us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619922.463236:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1173 00000100:00100000:3.0:1788619922.466829:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4ba1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589163648:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619922.466873:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003fefefac/0x1cd7fe536873560b lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c8d3 expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.466926:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003fefefac/0x1cd7fe536873560b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c8d3 expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.466939:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c7679c00, rc 0) 00000100:00100000:3.0:1788619922.467007:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571547136:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 9382us (9563us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619922.467017:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 935 00000100:00100000:1.0:1788619922.473787:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619922.473828:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619922.473965:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571547392 00000100:00100000:2.0:1788619922.473976:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571547392 00000100:00100000:2.0:1788619922.474033:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 936 00000100:00100000:2.0:1788619922.474044:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571547392:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619922.474063:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619922.474078:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1c87370/0x1cd7fe5368735619 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c8e1 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619922.474124:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f2680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589163904:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619922.474191:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164600 going to sleep for 11 seconds 00000100:00100000:0.0:1788619922.474879:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571547264 00000100:00100000:0.0:1788619922.474885:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571547264 00000100:00100000:0.0:1788619922.474971:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1174 00000100:00100000:0.0:1788619922.474978:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+20:2352:x1875503571547264:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619922.474988:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619922.475000:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe5368735620 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c8da expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.475022:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678c00/0x1cd7fe5368735578 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c847 expref: 29 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619922.475041:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fefd5880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589164032:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619922.475083:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebc00 going to sleep for 11 seconds 00000100:00100000:0.0:1788619922.478169:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fefd5880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589164032:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619922.478195:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:0.0:1788619922.478213:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe5368735620 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c8da expref: 21 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.478233:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619922.478235:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619922.478242:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe5368735620 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c8da expref: 21 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.478255:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed0400, rc 0) 00000100:00100000:2.0:1788619922.478616:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f2680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589163904:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619922.478637:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b1c87370/0x1cd7fe5368735619 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c8e1 expref: 7 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.478654:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1c87370/0x1cd7fe5368735619 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c8e1 expref: 7 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.478662:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6200, rc 0) 00000100:00100000:2.0:1788619922.478716:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571547392:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4674us (4886us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619922.478725:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 936 00000100:00100000:0.0:1788619922.479032:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+20:2352:x1875503571547264:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4052us (5252us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619922.479041:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1174 00000100:00100000:3.0:1788619922.483507:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619922.483666:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619922.483941:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571547648 00000100:00100000:2.0:1788619922.483951:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571547648 00000100:00100000:2.0:1788619922.484002:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 937 00000100:00100000:2.0:1788619922.484013:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe13a850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571547648:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619922.484030:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619922.484049:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3b293bd/0x1cd7fe5368735627 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c8ef expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619922.484113:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f2680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589164160:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619922.484199:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164600 going to sleep for 11 seconds 00000100:00100000:2.0:1788619922.487577:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f2680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589164160:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619922.487617:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f3b293bd/0x1cd7fe5368735627 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c8ef expref: 7 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619922.487631:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571547520 00000100:00100000:1.0:1788619922.487638:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571547520 00010000:00010000:2.0:1788619922.487643:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3b293bd/0x1cd7fe5368735627 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c8ef expref: 7 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.487655:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7e00, rc 0) 00000100:00100000:1.0:1788619922.487703:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1175 00000100:00100000:1.0:1788619922.487713:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df387100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+20:2354:x1875503571547520:12345-192.168.201.5@tcp:101:fsx.0 00000100:00100000:2.0:1788619922.487723:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe13a850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571547648:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3713us (4058us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619922.487730:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 937 00010000:00010000:1.0:1788619922.487730:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619922.487747:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678800/0x1cd7fe536873562e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c8e8 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.487778:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678c00/0x1cd7fe5368735578 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c847 expref: 29 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619922.487811:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900d5e680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589164288:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619922.487857:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7b40 going to sleep for 11 seconds 00000100:00100000:1.0:1788619922.490034:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900d5e680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589164288:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619922.490054:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:1.0:1788619922.490067:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678800/0x1cd7fe536873562e lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c8e8 expref: 21 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.490081:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619922.490083:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619922.490087:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678800/0x1cd7fe536873562e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c8e8 expref: 21 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.490096:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c7678800, rc 0) 00000100:00100000:1.0:1788619922.490138:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df387100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+20:2354:x1875503571547520:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2426us (6641us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619922.490147:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1175 00000100:00100000:0.0:1788619922.492526:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619922.492647:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571547776 00000100:00100000:3.0:1788619922.492657:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571547776 00000100:00100000:2.0:1788619922.492790:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1176 00000100:00100000:2.0:1788619922.492800:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+20:2354:x1875503571547776:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619922.492819:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619922.492838:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6200/0x1cd7fe5368735635 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c8f6 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.492882:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678c00/0x1cd7fe5368735578 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c847 expref: 29 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619922.492921:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f1880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589164416:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619922.492994:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164600 going to sleep for 11 seconds 00000100:00100000:2.0:1788619922.494600:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f1880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589164416:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619922.494622:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:2.0:1788619922.494638:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6200/0x1cd7fe5368735635 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c8f6 expref: 21 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.494658:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619922.494661:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619922.494666:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6200/0x1cd7fe5368735635 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c8f6 expref: 21 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.494679:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6200, rc 0) 00000100:00100000:2.0:1788619922.494734:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+20:2354:x1875503571547776:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 1935us (2226us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619922.494742:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1176 00000100:00100000:0.0:1788619922.495843:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619922.496037:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571547904 00000100:00100000:3.0:1788619922.496044:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571547904 00000100:00100000:3.0:1788619922.496088:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 938 00000100:00100000:3.0:1788619922.496097:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571547904:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619922.496110:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619922.496125:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe536873563c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c8fd expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619922.496174:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df384e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589164544:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619922.496245:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c300 going to sleep for 11 seconds 00000100:00100000:3.0:1788619922.501604:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df384e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589164544:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619922.501644:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe536873563c lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c8fd expref: 7 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.501701:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe536873563c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c8fd expref: 7 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.501715:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c600, rc 0) 00000100:00100000:3.0:1788619922.501774:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571547904:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5677us (5935us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619922.501782:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 938 00000100:00100000:0.0:1788619922.512250:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619922.512298:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619922.512399:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571548032 00000100:00100000:2.0:1788619922.512411:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571548032 00002000:00100000:2.0:1788619922.512465:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1085440->2084863) req@ffff8e78fefd7b80 x1875503571548032/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:18/0 lens 488/0 e 0 to 0 dl 1788619933 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619922.512476:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fefd7b80 with x1875503571548032 ext(1085440->2084863) 00002000:00010000:2.0:1788619922.512482:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fefd7b80 00000100:00100000:2.0:1788619922.512514:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 546 00000100:00100000:2.0:1788619922.512533:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571548032:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619922.513655:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571548160 00000100:00100000:3.0:1788619922.513665:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571548160 00002000:00100000:3.0:1788619922.513716:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fefd5f80 x1875503571548160/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:18/0 lens 488/0 e 0 to 0 dl 1788619933 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619922.513727:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fefd5f80 with x1875503571548160 00000100:00100000:3.0:1788619922.513760:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 740 00000100:00100000:3.0:1788619922.513768:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+30:2352:x1875503571548160:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619922.530855:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571548032:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 18311us (18601us total) trans 0 rc 999424/999424 00000100:00100000:2.0:1788619922.530870:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 546 00002000:00100000:2.0:1788619922.530884:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1085440->2084863) req@ffff8e78fefd7b80 x1875503571548032/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:18/0 lens 488/440 e 0 to 0 dl 1788619933 ref 1 fl Complete:/600/0 rc 999424/999424 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619922.530894:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fefd7b80 with x1875503571548032 ext(1085440->2084863) 00002000:00010000:2.0:1788619922.530901:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fefd7b80 00000100:00100000:3.0:1788619922.534257:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+30:2352:x1875503571548160:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 20478us (21946us total) trans 0 rc 704512/704512 00000100:00100000:3.0:1788619922.534272:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 740 00002000:00100000:3.0:1788619922.534284:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fefd5f80 x1875503571548160/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:18/0 lens 488/440 e 0 to 0 dl 1788619933 ref 1 fl Complete:/600/0 rc 704512/704512 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619922.534306:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fefd5f80 with x1875503571548160 00000100:00100000:1.0:1788619922.539212:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619922.539390:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571548288 00000100:00100000:3.0:1788619922.539400:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571548288 00000100:00100000:3.0:1788619922.539463:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 939 00000100:00100000:3.0:1788619922.539488:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:14908:x1875503571548288:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619922.539507:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619922.539648:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe5368735643 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c904 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.539679:0:6696:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000001391204f/0x1cd7fe5368735604 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1085439) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c8c5 expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.539699:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(10 7) 00010000:00010000:3.0:1788619922.539703:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(11 7) 00010000:00010000:3.0:1788619922.539708:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe5368735643 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1351680->1355775] (req 1351680->1355775) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c904 expref: 7 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.539743:0:6696:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000001391204f/0x1cd7fe5368735604 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1085439) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c8c5 expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.539757:0:6696:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000001391204f/0x1cd7fe5368735604 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1085439) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c8c5 expref: 5 pid: 6698 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619922.539770:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df384e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589164672:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619922.539903:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c300 going to sleep for 11 seconds 00000100:00100000:3.0:1788619922.544833:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df384e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589164672:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619922.544871:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe5368735643 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1351680->1355775] (req 1351680->1355775) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c904 expref: 7 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.544916:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683ca00, rc 0) 00000100:00100000:0.0:1788619922.545259:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:14908:x1875503571548288:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5772us (6053us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619922.545268:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 939 00000100:00100000:1.0:1788619922.545719:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619922.546155:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571548416 00000100:00100000:3.0:1788619922.546163:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571548416 00002000:00100000:3.0:1788619922.546206:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1085439) req@ffff8e78e67f6a00 x1875503571548416/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:18/0 lens 488/0 e 0 to 0 dl 1788619933 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619922.546217:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78e67f6a00 with x1875503571548416 ext(1048576->1085439) 00010000:00010000:3.0:1788619922.546226:0:6701:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000001391204f/0x1cd7fe5368735604 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1085439) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c8c5 expref: 6 pid: 6698 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.546241:0:6701:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000001391204f/0x1cd7fe5368735604 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1085439) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c8c5 expref: 6 pid: 6698 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788619922.546249:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff8e78e67f6a00 00000100:00100000:3.0:1788619922.546280:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 16 00000100:00100000:3.0:1788619922.546288:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571548416:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:3.0:1788619922.551742:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571548416:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 5443us (6013us total) trans 4294967458 rc 0/0 00000100:00100000:3.0:1788619922.551757:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 16 00002000:00100000:3.0:1788619922.551770:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1085439) req@ffff8e78e67f6a00 x1875503571548416/t4294967458(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:18/0 lens 488/448 e 0 to 0 dl 1788619933 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619922.551780:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78e67f6a00 with x1875503571548416 ext(1048576->1085439) 00010000:00010000:3.0:1788619922.551790:0:6701:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000001391204f/0x1cd7fe5368735604 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1085439) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c8c5 expref: 8 pid: 6698 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.551820:0:6701:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000001391204f/0x1cd7fe5368735604 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1085439) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c8c5 expref: 8 pid: 6698 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788619922.551827:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff8e78e67f6a00 00000100:00100000:1.0:1788619922.557218:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619922.557321:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571548544 00000100:00100000:3.0:1788619922.557405:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571548544 00010000:00010000:2.0:1788619922.557592:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000001391204f/0x1cd7fe5368735604 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1085439) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c8c5 expref: 8 pid: 6698 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619922.557672:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 397 00000100:00100000:2.0:1788619922.557683:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8558450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571548544:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619922.557709:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619922.557733:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000001391204f/0x1cd7fe5368735604 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1085439) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c8c5 expref: 8 pid: 6698 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.557747:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000001391204f/0x1cd7fe5368735604 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1085439) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c8c5 expref: 8 pid: 6698 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.557771:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001391204f/0x1cd7fe5368735604 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1085439) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c8c5 expref: 8 pid: 6698 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.557787:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(11 7) 00010000:00010000:2.0:1788619922.557792:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(11 7) 00010000:00010000:2.0:1788619922.557800:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe5368735643 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1351680->1355775) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c904 expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.557811:0:13330:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe5368735643 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1351680->1355775) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c904 expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.557849:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe5368735643 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1351680->1355775) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c904 expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619922.557863:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f1880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589164800:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619922.558104:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164600 going to sleep for 11 seconds 00000100:00100000:2.0:1788619922.561676:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619922.561927:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571548672 00000100:00100000:1.0:1788619922.561936:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571548672 00000100:00100000:1.0:1788619922.562014:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 547 00000100:00100000:1.0:1788619922.562024:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571548672:12345-192.168.201.5@tcp:22:fsx.0 00002000:00010000:1.0:1788619922.562258:0:6702:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 2084447 -> 2084447 00002000:00010000:1.0:1788619922.562262:0:6702:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619922 -> 1788619922 00002000:00010000:1.0:1788619922.562265:0:6702:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619922 -> 1788619922 00002000:00010000:1.0:1788619922.562267:0:6702:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619922 -> 1788619922 00002000:00010000:1.0:1788619922.562268:0:6702:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 263 -> 263 00000100:00100000:1.0:1788619922.562356:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+8:2355:x1875503571548672:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 332us (687us total) trans 4294967459 rc 0/0 00000100:00100000:1.0:1788619922.562364:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 547 00000100:00100000:3.0:1788619922.564637:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f1880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589164800:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619922.564664:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619922.564866:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8558450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571548544:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 7183us (7656us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619922.564876:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 397 00000100:00100000:2.0:1788619922.567281:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619922.567434:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571548800 00000100:00100000:1.0:1788619922.567443:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571548800 00000100:00100000:1.0:1788619922.567545:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 940 00000100:00100000:1.0:1788619922.567557:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2352:x1875503571548800:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619922.567575:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619922.567592:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff40cdc2/0x1cd7fe536873564a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c90b expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619922.567652:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589164928:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619922.567737:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7f00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619922.570062:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589164928:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619922.570098:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ff40cdc2/0x1cd7fe536873564a lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c90b expref: 8 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.570120:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff40cdc2/0x1cd7fe536873564a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c90b expref: 8 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.570132:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297a00, rc 0) 00000100:00100000:2.0:1788619922.570221:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2352:x1875503571548800:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2667us (2949us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619922.570229:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 940 00000100:00100000:2.0:1788619922.573504:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619922.573651:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571548928 00000100:00100000:1.0:1788619922.573662:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571548928 00000100:00100000:1.0:1788619922.573761:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 941 00000100:00100000:1.0:1788619922.573772:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2353:x1875503571548928:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619922.573791:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619922.573810:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c5e22f67/0x1cd7fe5368735651 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c912 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619922.573867:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589165056:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619922.573995:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7d80 going to sleep for 11 seconds 00000100:00100000:2.0:1788619922.576257:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589165056:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619922.576298:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c5e22f67/0x1cd7fe5368735651 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c912 expref: 8 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.576323:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c5e22f67/0x1cd7fe5368735651 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c912 expref: 8 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.576418:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297000, rc 0) 00000100:00100000:2.0:1788619922.576504:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2353:x1875503571548928:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2733us (3010us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619922.576521:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 941 00000100:00100000:2.0:1788619922.580572:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619922.580719:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571549056 00000100:00100000:3.0:1788619922.580728:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571549056 00000100:00100000:3.0:1788619922.580781:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 942 00000100:00100000:3.0:1788619922.580790:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6858380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2353:x1875503571549056:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619922.580808:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619922.580824:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062c34eb6/0x1cd7fe5368735658 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c919 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619922.580880:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df384e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589165184:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619922.580981:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c300 going to sleep for 11 seconds 00000100:00100000:3.0:1788619922.583254:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df384e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589165184:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619922.583292:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000062c34eb6/0x1cd7fe5368735658 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c919 expref: 8 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.583317:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062c34eb6/0x1cd7fe5368735658 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c919 expref: 8 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.583373:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c800, rc 0) 00000100:00100000:3.0:1788619922.583431:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6858380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2353:x1875503571549056:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2642us (2868us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619922.583439:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 942 00000100:00100000:1.0:1788619922.586441:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619922.586848:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571549184 00000100:00100000:0.0:1788619922.586858:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571549184 00000100:00100000:0.0:1788619922.586986:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 943 00000100:00100000:0.0:1788619922.586996:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2353:x1875503571549184:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619922.587012:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619922.587028:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd8a44c0/0x1cd7fe536873565f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c920 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619922.587087:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cd4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589165312:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619922.587168:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebc00 going to sleep for 11 seconds 00000100:00100000:0.0:1788619922.589822:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cd4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589165312:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619922.589865:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000dd8a44c0/0x1cd7fe536873565f lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c920 expref: 8 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.589896:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd8a44c0/0x1cd7fe536873565f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c920 expref: 8 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.589922:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed1600, rc 0) 00000100:00100000:0.0:1788619922.589986:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2353:x1875503571549184:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2991us (3553us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619922.589997:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 943 00000100:00100000:1.0:1788619922.592969:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619922.593365:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571549312 00000100:00100000:2.0:1788619922.593377:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571549312 00000100:00100000:2.0:1788619922.593433:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 944 00000100:00100000:2.0:1788619922.593443:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2352:x1875503571549312:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619922.593461:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619922.593478:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001dc9de0/0x1cd7fe5368735666 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c927 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619922.593552:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c8595c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589165440:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619922.593639:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164c00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619922.596135:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c8595c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589165440:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619922.596174:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000001dc9de0/0x1cd7fe5368735666 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c927 expref: 8 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.596206:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001dc9de0/0x1cd7fe5368735666 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c927 expref: 8 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.596220:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6a00, rc 0) 00000100:00100000:2.0:1788619922.596271:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2352:x1875503571549312:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2829us (3321us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619922.596280:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 944 00000100:00100000:1.0:1788619922.598798:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619922.598871:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571549440 00000100:00100000:2.0:1788619922.598880:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571549440 00000100:00100000:2.0:1788619922.598939:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 945 00000100:00100000:2.0:1788619922.598949:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571549440:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619922.598975:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619922.598993:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5ec8983/0x1cd7fe536873566d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c92e expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619922.599046:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c8595c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589165568:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619922.599123:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164c00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619922.601789:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c8595c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589165568:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619922.601830:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a5ec8983/0x1cd7fe536873566d lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c92e expref: 8 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.601856:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5ec8983/0x1cd7fe536873566d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c92e expref: 8 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.601875:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6600, rc 0) 00000100:00100000:2.0:1788619922.601940:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571549440:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2991us (3148us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619922.601948:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 945 00000100:00100000:1.0:1788619922.605197:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619922.605290:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571549568 00000100:00100000:0.0:1788619922.605300:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571549568 00000100:00100000:0.0:1788619922.605396:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1177 00000100:00100000:0.0:1788619922.605406:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+20:2353:x1875503571549568:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619922.605427:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619922.605446:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1600/0x1cd7fe5368735674 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c935 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.605477:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678c00/0x1cd7fe5368735578 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c847 expref: 29 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619922.605523:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cd4700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589165696:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619922.605587:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebc00 going to sleep for 11 seconds 00000100:00100000:0.0:1788619922.607880:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cd4700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589165696:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619922.607920:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:0.0:1788619922.607940:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1600/0x1cd7fe5368735674 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c935 expref: 21 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.607976:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619922.607979:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619922.607985:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1600/0x1cd7fe5368735674 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c935 expref: 21 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.608000:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed1600, rc 0) 00000100:00100000:0.0:1788619922.608100:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+20:2353:x1875503571549568:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2694us (2912us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619922.608108:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1177 00000100:00100000:0.0:1788619922.611713:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619922.611781:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571549696 00000100:00100000:2.0:1788619922.611791:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571549696 00000100:00100000:2.0:1788619922.611849:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1178 00000100:00100000:2.0:1788619922.611859:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+20:2352:x1875503571549696:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619922.611879:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619922.611899:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe536873567b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c93c expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.611947:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678c00/0x1cd7fe5368735578 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c847 expref: 29 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619922.611994:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c8595c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589165824:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619922.612078:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164c00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619922.614852:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c8595c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589165824:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619922.614880:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:2.0:1788619922.614899:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe536873567b lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c93c expref: 21 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.615295:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619922.615298:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619922.615305:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe536873567b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c93c expref: 21 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.615320:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6a00, rc 0) 00000100:00100000:2.0:1788619922.615639:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+20:2352:x1875503571549696:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3779us (3933us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619922.615648:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1178 00000100:00100000:0.0:1788619922.620156:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619922.620246:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571549824 00000100:00100000:2.0:1788619922.620257:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571549824 00000100:00100000:2.0:1788619922.620446:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1179 00000100:00100000:2.0:1788619922.620457:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d59500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+20:2355:x1875503571549824:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619922.620478:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619922.620498:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6600/0x1cd7fe5368735682 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c943 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.620605:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678c00/0x1cd7fe5368735578 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c847 expref: 29 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619922.620645:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c8595c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589165952:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619922.620763:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164c00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619922.623729:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c8595c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589165952:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619922.623754:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:2.0:1788619922.623772:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6600/0x1cd7fe5368735682 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c943 expref: 21 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.623791:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619922.623794:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619922.623799:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6600/0x1cd7fe5368735682 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c943 expref: 21 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.623812:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6600, rc 0) 00000100:00100000:2.0:1788619922.624094:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d59500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+20:2355:x1875503571549824:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3638us (3947us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619922.624102:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1179 00000100:00100000:1.0:1788619922.627210:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619922.627288:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571549952 00000100:00100000:0.0:1788619922.627297:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571549952 00000100:00100000:0.0:1788619922.627528:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1180 00000100:00100000:0.0:1788619922.627539:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe060380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+20:2352:x1875503571549952:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619922.627559:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619922.627578:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0a00/0x1cd7fe5368735689 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c94a expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.627612:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678c00/0x1cd7fe5368735578 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c847 expref: 29 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619922.627652:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b34000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589166080:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619922.627736:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebc00 going to sleep for 11 seconds 00000100:00100000:1.0:1788619922.630801:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b34000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589166080:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619922.630826:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:1.0:1788619922.630846:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0a00/0x1cd7fe5368735689 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c94a expref: 21 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.630948:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619922.631019:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619922.631026:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0a00/0x1cd7fe5368735689 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c94a expref: 21 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.631041:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed0a00, rc 0) 00000100:00100000:1.0:1788619922.631140:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe060380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+20:2352:x1875503571549952:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3603us (3938us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619922.631148:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1180 00000100:00100000:1.0:1788619922.634951:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619922.635193:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571550080 00000100:00100000:3.0:1788619922.635204:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571550080 00002000:00100000:3.0:1788619922.635256:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (2023424->2027519) req@ffff8e78fe060700 x1875503571550080/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:18/0 lens 488/0 e 0 to 0 dl 1788619933 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619922.635268:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fe060700 with x1875503571550080 ext(2023424->2027519) 00002000:00010000:3.0:1788619922.635274:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fe060700 00000100:00100000:3.0:1788619922.635306:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 548 00000100:00100000:3.0:1788619922.635317:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe060700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+8:2352:x1875503571550080:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619922.635631:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe060700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+8:2352:x1875503571550080:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 314us (688us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619922.635639:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 548 00002000:00100000:3.0:1788619922.635645:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (2023424->2027519) req@ffff8e78fe060700 x1875503571550080/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:18/0 lens 488/4536 e 0 to 0 dl 1788619933 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619922.635653:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fe060700 with x1875503571550080 ext(2023424->2027519) 00002000:00010000:3.0:1788619922.635656:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fe060700 00000100:00100000:1.0:1788619922.639320:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619922.639679:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571550208 00000100:00100000:0.0:1788619922.639699:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571550208 00002000:00100000:0.0:1788619922.639750:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (2027520->2031615) req@ffff8e78fe062a00 x1875503571550208/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:18/0 lens 488/0 e 0 to 0 dl 1788619933 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619922.639760:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fe062a00 with x1875503571550208 ext(2027520->2031615) 00002000:00010000:0.0:1788619922.639766:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fe062a00 00000100:00100000:0.0:1788619922.639795:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 549 00000100:00100000:0.0:1788619922.639804:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe062a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+8:2353:x1875503571550208:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619922.640115:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe062a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+8:2353:x1875503571550208:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 311us (804us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619922.640123:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 549 00002000:00100000:0.0:1788619922.640130:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (2027520->2031615) req@ffff8e78fe062a00 x1875503571550208/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:18/0 lens 488/4536 e 0 to 0 dl 1788619933 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619922.640138:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fe062a00 with x1875503571550208 ext(2027520->2031615) 00002000:00010000:0.0:1788619922.640141:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fe062a00 00000100:00100000:0.0:1788619922.643692:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619922.643801:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571550336 00000100:00100000:1.0:1788619922.643812:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571550336 00002000:00100000:1.0:1788619922.643866:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (2031616->2035711) req@ffff8e77c6b34000 x1875503571550336/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:18/0 lens 488/0 e 0 to 0 dl 1788619933 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619922.643877:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6b34000 with x1875503571550336 ext(2031616->2035711) 00002000:00010000:1.0:1788619922.643882:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6b34000 00000100:00100000:1.0:1788619922.643913:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 550 00000100:00100000:1.0:1788619922.643922:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b34000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+8:2352:x1875503571550336:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619922.644189:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b34000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+8:2352:x1875503571550336:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 266us (513us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619922.644197:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 550 00002000:00100000:1.0:1788619922.644204:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (2031616->2035711) req@ffff8e77c6b34000 x1875503571550336/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:18/0 lens 488/4536 e 0 to 0 dl 1788619933 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619922.644213:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6b34000 with x1875503571550336 ext(2031616->2035711) 00002000:00010000:1.0:1788619922.644216:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6b34000 00000100:00100000:1.0:1788619922.647247:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619922.647418:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571550464 00000100:00100000:3.0:1788619922.647428:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571550464 00002000:00100000:2.0:1788619922.647612:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (2035712->2039807) req@ffff8e7900e02300 x1875503571550464/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:18/0 lens 488/0 e 0 to 0 dl 1788619933 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619922.647623:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e7900e02300 with x1875503571550464 ext(2035712->2039807) 00002000:00010000:2.0:1788619922.647629:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e7900e02300 00000100:00100000:2.0:1788619922.647702:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 551 00000100:00100000:2.0:1788619922.647712:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e02300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+8:2352:x1875503571550464:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619922.647909:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e02300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+8:2352:x1875503571550464:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 198us (675us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619922.647917:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 551 00002000:00100000:2.0:1788619922.647923:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (2035712->2039807) req@ffff8e7900e02300 x1875503571550464/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:18/0 lens 488/4536 e 0 to 0 dl 1788619933 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619922.647930:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e7900e02300 with x1875503571550464 ext(2035712->2039807) 00002000:00010000:2.0:1788619922.647933:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e7900e02300 00000100:00100000:3.0:1788619922.650412:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619922.650508:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571550592 00000100:00100000:0.0:1788619922.650516:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571550592 00002000:00100000:0.0:1788619922.650562:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (2039808->2043903) req@ffff8e78df384e00 x1875503571550592/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:18/0 lens 488/0 e 0 to 0 dl 1788619933 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619922.650573:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df384e00 with x1875503571550592 ext(2039808->2043903) 00002000:00010000:0.0:1788619922.650578:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df384e00 00000100:00100000:0.0:1788619922.650609:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 552 00000100:00100000:0.0:1788619922.650619:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df384e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+8:2352:x1875503571550592:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619922.650821:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df384e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+8:2352:x1875503571550592:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 202us (416us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619922.650830:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 552 00002000:00100000:0.0:1788619922.650837:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (2039808->2043903) req@ffff8e78df384e00 x1875503571550592/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:18/0 lens 488/4536 e 0 to 0 dl 1788619933 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619922.650846:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df384e00 with x1875503571550592 ext(2039808->2043903) 00002000:00010000:0.0:1788619922.650849:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df384e00 00000100:00100000:0.0:1788619922.653299:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619922.653400:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571550720 00000100:00100000:3.0:1788619922.653422:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571550720 00002000:00100000:3.0:1788619922.653467:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (2043904->2047999) req@ffff8e77c6b17100 x1875503571550720/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:18/0 lens 488/0 e 0 to 0 dl 1788619933 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619922.653478:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6b17100 with x1875503571550720 ext(2043904->2047999) 00002000:00010000:3.0:1788619922.653483:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6b17100 00000100:00100000:3.0:1788619922.653518:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 553 00000100:00100000:3.0:1788619922.653528:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b17100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+8:2353:x1875503571550720:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619922.653761:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b17100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+8:2353:x1875503571550720:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 234us (471us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619922.653768:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 553 00002000:00100000:3.0:1788619922.653775:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (2043904->2047999) req@ffff8e77c6b17100 x1875503571550720/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:18/0 lens 488/4536 e 0 to 0 dl 1788619933 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619922.653783:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6b17100 with x1875503571550720 ext(2043904->2047999) 00002000:00010000:3.0:1788619922.653787:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6b17100 00000100:00100000:3.0:1788619922.656895:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619922.657009:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571550848 00000100:00100000:2.0:1788619922.657019:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571550848 00002000:00100000:2.0:1788619922.657067:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (2048000->2052095) req@ffff8e78df386300 x1875503571550848/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:18/0 lens 488/0 e 0 to 0 dl 1788619933 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619922.657078:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df386300 with x1875503571550848 ext(2048000->2052095) 00002000:00010000:2.0:1788619922.657089:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df386300 00000100:00100000:2.0:1788619922.657122:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 554 00000100:00100000:2.0:1788619922.657132:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df386300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+8:2353:x1875503571550848:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619922.657378:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df386300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+8:2353:x1875503571550848:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 246us (491us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619922.657386:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 554 00002000:00100000:2.0:1788619922.657393:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (2048000->2052095) req@ffff8e78df386300 x1875503571550848/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:18/0 lens 488/4536 e 0 to 0 dl 1788619933 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619922.657402:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df386300 with x1875503571550848 ext(2048000->2052095) 00002000:00010000:2.0:1788619922.657406:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df386300 00000100:00100000:0.0:1788619922.659753:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619922.659855:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571550976 00000100:00100000:1.0:1788619922.659863:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571550976 00002000:00100000:1.0:1788619922.659907:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (2052096->2056191) req@ffff8e77c6b16a00 x1875503571550976/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:18/0 lens 488/0 e 0 to 0 dl 1788619933 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619922.659917:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6b16a00 with x1875503571550976 ext(2052096->2056191) 00002000:00010000:1.0:1788619922.659923:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6b16a00 00000100:00100000:1.0:1788619922.659963:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 555 00000100:00100000:1.0:1788619922.659973:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b16a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+8:2354:x1875503571550976:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619922.660160:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b16a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+8:2354:x1875503571550976:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 188us (414us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619922.660166:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 555 00002000:00100000:1.0:1788619922.660173:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (2052096->2056191) req@ffff8e77c6b16a00 x1875503571550976/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:18/0 lens 488/4536 e 0 to 0 dl 1788619933 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619922.660181:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6b16a00 with x1875503571550976 ext(2052096->2056191) 00002000:00010000:1.0:1788619922.660184:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6b16a00 00000100:00100000:0.0:1788619922.662425:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619922.662505:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571551104 00000100:00100000:3.0:1788619922.662523:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571551104 00002000:00100000:3.0:1788619922.662566:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (2056192->2060287) req@ffff8e77c6b16680 x1875503571551104/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:18/0 lens 488/0 e 0 to 0 dl 1788619933 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619922.662577:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6b16680 with x1875503571551104 ext(2056192->2060287) 00002000:00010000:3.0:1788619922.662581:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6b16680 00000100:00100000:3.0:1788619922.662600:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 556 00000100:00100000:3.0:1788619922.662609:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b16680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+8:2354:x1875503571551104:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619922.662801:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b16680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+8:2354:x1875503571551104:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 193us (384us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619922.662823:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 556 00002000:00100000:3.0:1788619922.662830:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (2056192->2060287) req@ffff8e77c6b16680 x1875503571551104/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:18/0 lens 488/4536 e 0 to 0 dl 1788619933 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619922.662837:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6b16680 with x1875503571551104 ext(2056192->2060287) 00002000:00010000:3.0:1788619922.662840:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6b16680 00000100:00100000:3.0:1788619922.665430:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619922.665560:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571551232 00000100:00100000:2.0:1788619922.665568:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571551232 00002000:00100000:2.0:1788619922.665613:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (2060288->2064383) req@ffff8e78fd938e00 x1875503571551232/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:18/0 lens 488/0 e 0 to 0 dl 1788619933 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619922.665624:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fd938e00 with x1875503571551232 ext(2060288->2064383) 00002000:00010000:2.0:1788619922.665630:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fd938e00 00000100:00100000:2.0:1788619922.665664:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 557 00000100:00100000:2.0:1788619922.665674:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd938e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+8:2354:x1875503571551232:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619922.665876:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd938e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+8:2354:x1875503571551232:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 202us (454us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619922.665883:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 557 00002000:00100000:2.0:1788619922.665890:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (2060288->2064383) req@ffff8e78fd938e00 x1875503571551232/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:18/0 lens 488/4536 e 0 to 0 dl 1788619933 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619922.665898:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fd938e00 with x1875503571551232 ext(2060288->2064383) 00002000:00010000:2.0:1788619922.665902:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fd938e00 00000100:00100000:0.0:1788619922.668239:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619922.668359:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571551360 00000100:00100000:1.0:1788619922.668368:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571551360 00002000:00100000:1.0:1788619922.668406:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (2064384->2068479) req@ffff8e77c6b16300 x1875503571551360/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:18/0 lens 488/0 e 0 to 0 dl 1788619933 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619922.668417:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6b16300 with x1875503571551360 ext(2064384->2068479) 00002000:00010000:1.0:1788619922.668421:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6b16300 00000100:00100000:1.0:1788619922.668449:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 558 00000100:00100000:1.0:1788619922.668457:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b16300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+8:2354:x1875503571551360:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619922.668650:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b16300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+8:2354:x1875503571551360:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 193us (419us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619922.668657:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 558 00002000:00100000:1.0:1788619922.668664:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (2064384->2068479) req@ffff8e77c6b16300 x1875503571551360/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:18/0 lens 488/4536 e 0 to 0 dl 1788619933 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619922.668672:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6b16300 with x1875503571551360 ext(2064384->2068479) 00002000:00010000:1.0:1788619922.668675:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6b16300 00000100:00100000:0.0:1788619922.671877:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619922.672405:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571551488 00000100:00100000:1.0:1788619922.672416:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571551488 00000100:00100000:1.0:1788619922.672480:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 946 00000100:00100000:1.0:1788619922.672491:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571551488:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619922.672619:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619922.672641:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd485a32/0x1cd7fe5368735690 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c951 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619922.672715:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c74aa300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589166208:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619922.672844:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7600 going to sleep for 11 seconds 00000100:00100000:1.0:1788619922.675317:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c74aa300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589166208:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619922.675421:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000cd485a32/0x1cd7fe5368735690 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c951 expref: 8 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.675445:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd485a32/0x1cd7fe5368735690 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c951 expref: 8 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.675459:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296600, rc 0) 00000100:00100000:1.0:1788619922.675610:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571551488:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3119us (3742us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619922.675619:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 946 00000100:00100000:0.0:1788619922.678622:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619922.678736:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571551616 00000100:00100000:3.0:1788619922.678746:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571551616 00000100:00100000:3.0:1788619922.678847:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 947 00000100:00100000:3.0:1788619922.678857:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7431c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2352:x1875503571551616:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619922.678875:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619922.678892:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001fc792ba/0x1cd7fe5368735697 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c958 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619922.678968:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fd93a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589166336:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619922.679055:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48ce40 going to sleep for 11 seconds 00000100:00100000:3.0:1788619922.681564:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fd93a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589166336:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619922.681601:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001fc792ba/0x1cd7fe5368735697 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c958 expref: 8 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.681623:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001fc792ba/0x1cd7fe5368735697 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c958 expref: 8 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.681636:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d800, rc 0) 00000100:00100000:3.0:1788619922.681724:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7431c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2352:x1875503571551616:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2867us (3110us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619922.681732:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 947 00000100:00100000:1.0:1788619922.685428:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619922.685553:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571551744 00000100:00100000:2.0:1788619922.685563:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571551744 00000100:00100000:2.0:1788619922.685624:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1181 00000100:00100000:2.0:1788619922.685636:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74aa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+20:14908:x1875503571551744:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619922.685663:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619922.685698:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe536873569e lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c95f expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.685732:0:9417:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678c00/0x1cd7fe5368735578 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c847 expref: 29 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.685743:0:9417:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe536873569e lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c95f expref: 21 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.685778:0:9417:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678c00/0x1cd7fe5368735578 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c847 expref: 29 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.685792:0:9417:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678c00/0x1cd7fe5368735578 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c847 expref: 29 pid: 9417 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619922.685804:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c8595c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589166464:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619922.685885:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164b40 going to sleep for 11 seconds 00000100:00100000:2.0:1788619922.689468:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c8595c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589166464:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619922.689507:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe536873569e lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c95f expref: 21 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.689530:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619922.689533:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619922.689537:0:9417:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe536873569e lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c95f expref: 21 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.689546:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6a00, rc 0) 00000100:00100000:2.0:1788619922.689611:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74aa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+21:14908:x1875503571551744:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3976us (4192us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619922.689620:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1181 00000100:00100000:0.0:1788619922.691806:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619922.692298:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571551872 00000100:00100000:2.0:1788619922.692310:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571551872 00002000:00100000:2.0:1788619922.692399:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fe000380 x1875503571551872/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:18/0 lens 488/0 e 0 to 0 dl 1788619933 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619922.692409:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fe000380 with x1875503571551872 00010000:00010000:2.0:1788619922.692418:0:5859:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678c00/0x1cd7fe5368735578 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c847 expref: 30 pid: 9417 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.692433:0:5859:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678c00/0x1cd7fe5368735578 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c847 expref: 30 pid: 9417 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788619922.692441:0:5859:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e78fe000380 00000100:00100000:2.0:1788619922.692479:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 30 00000100:00100000:2.0:1788619922.692489:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe000380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+30:2354:x1875503571551872:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:2.0:1788619922.696921:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1048576 -> 1048576 00000004:00010000:2.0:1788619922.696930:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619922 -> 1788619922 00000004:00010000:2.0:1788619922.696934:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619922 -> 1788619919 00000004:00010000:2.0:1788619922.696937:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619922 -> 1788619922 00000004:00010000:2.0:1788619922.696939:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1030 -> 1030 00000100:00100000:2.0:1788619922.697081:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe000380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+32:2354:x1875503571551872:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 4588us (5279us total) trans 4294967727 rc 0/0 00000100:00100000:2.0:1788619922.697093:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 30 00002000:00100000:2.0:1788619922.697106:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fe000380 x1875503571551872/t4294967727(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:18/0 lens 488/448 e 0 to 0 dl 1788619933 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619922.697115:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fe000380 with x1875503571551872 00010000:00010000:2.0:1788619922.697124:0:5859:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678c00/0x1cd7fe5368735578 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c847 expref: 32 pid: 9417 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.697140:0:5859:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678c00/0x1cd7fe5368735578 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c847 expref: 32 pid: 9417 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788619922.697147:0:5859:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e78fe000380 00000100:00100000:1.0:1788619922.703054:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619922.703162:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571552000 00000100:00100000:0.0:1788619922.703173:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571552000 00010000:00010000:0.0:1788619922.703212:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678c00/0x1cd7fe5368735578 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c847 expref: 32 pid: 9417 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619922.703252:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 398 00000100:00100000:0.0:1788619922.703263:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74a8000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+32:2355:x1875503571552000:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619922.703276:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619922.703302:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678c00/0x1cd7fe5368735578 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c847 expref: 32 pid: 9417 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.703315:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678c00/0x1cd7fe5368735578 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c847 expref: 32 pid: 9417 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.703361:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c7678c00/0x1cd7fe5368735578 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c847 expref: 32 pid: 9417 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.703378:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619922.703383:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe536873569e lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c95f expref: 20 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.703394:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe536873569e lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c95f expref: 20 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.703403:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe536873569e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c95f expref: 20 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.703445:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe536873569e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c95f expref: 20 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619922.703459:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78cdcc3800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589166592:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619922.703672:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb000 going to sleep for 11 seconds 00000100:00100000:0.0:1788619922.716220:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619922.716283:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619922.716432:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571552256 00000100:00100000:2.0:1788619922.716443:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571552256 00002000:00100000:2.0:1788619922.716495:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5f24e00 x1875503571552256/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:18/0 lens 488/0 e 0 to 0 dl 1788619933 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619922.716516:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5f24e00 with x1875503571552256 00000100:00100000:2.0:1788619922.716556:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 741 00000100:00100000:2.0:1788619922.716564:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f24e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+21:2354:x1875503571552256:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619922.717474:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571552128 00000100:00100000:3.0:1788619922.717481:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571552128 00002000:00100000:3.0:1788619922.717513:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1048576->2084863) req@ffff8e77c4ba1f80 x1875503571552128/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:18/0 lens 504/0 e 0 to 0 dl 1788619933 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619922.717523:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4ba1f80 with x1875503571552128 ext(1048576->2084863) 00002000:00010000:3.0:1788619922.717528:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4ba1f80 00000100:00100000:3.0:1788619922.717549:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 559 00000100:00100000:3.0:1788619922.717557:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+8:2355:x1875503571552128:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619922.719411:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78cdcc3800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589166592:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619922.719439:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619922.731418:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74a8000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+31:2355:x1875503571552000:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 28142us (28360us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619922.731434:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 398 00000100:00100000:2.0:1788619922.737860:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f24e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+21:2354:x1875503571552256:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 21283us (21563us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1788619922.737876:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 741 00002000:00100000:2.0:1788619922.737889:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5f24e00 x1875503571552256/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:18/0 lens 488/440 e 0 to 0 dl 1788619933 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619922.737900:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5f24e00 with x1875503571552256 00000100:00100000:1.0:1788619922.750767:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619922.750910:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571552384 00000100:00100000:2.0:1788619922.750921:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571552384 00000100:00100000:2.0:1788619922.750988:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1182 00000100:00100000:2.0:1788619922.750999:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74ab800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+31:2354:x1875503571552384:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619922.751029:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619922.751054:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7800/0x1cd7fe53687356a5 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c966 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.751095:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe536873569e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c95f expref: 20 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619922.751136:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c8595c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589166720:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619922.751229:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164b40 going to sleep for 11 seconds 00000004:00010000:2.0:1788619922.753461:0:5846:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619919 -> 1788619922 00000100:00100000:2.0:1788619922.753492:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c8595c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589166720:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619922.753525:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:2.0:1788619922.753543:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7800/0x1cd7fe53687356a5 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c966 expref: 32 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.753564:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619922.753567:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619922.753572:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7800/0x1cd7fe53687356a5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c966 expref: 32 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.753591:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7800, rc 0) 00000100:00100000:2.0:1788619922.753651:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74ab800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+31:2354:x1875503571552384:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2654us (2893us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619922.753660:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1182 00000100:00100000:2.0:1788619922.756150:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+8:2355:x1875503571552128:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 38584us (39932us total) trans 0 rc 991232/991232 00000100:00100000:2.0:1788619922.756163:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 559 00002000:00100000:2.0:1788619922.756176:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1048576->2084863) req@ffff8e77c4ba1f80 x1875503571552128/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:18/0 lens 504/440 e 0 to 0 dl 1788619933 ref 1 fl Complete:/600/0 rc 991232/991232 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619922.756187:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4ba1f80 with x1875503571552128 ext(1048576->2084863) 00002000:00010000:2.0:1788619922.756193:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4ba1f80 00000100:00100000:1.0:1788619922.761006:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619922.761097:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571552512 00000100:00100000:2.0:1788619922.761107:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571552512 00000100:00100000:2.0:1788619922.761164:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 948 00000100:00100000:2.0:1788619922.761176:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74a9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571552512:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619922.761205:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619922.761223:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000092f9790d/0x1cd7fe53687356ac lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c96d expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619922.761282:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c8595c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589166848:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619922.761403:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164b40 going to sleep for 11 seconds 00000100:00100000:2.0:1788619922.772635:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c8595c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589166848:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619922.772692:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000092f9790d/0x1cd7fe53687356ac lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c96d expref: 8 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.772719:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000092f9790d/0x1cd7fe53687356ac lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c96d expref: 8 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.772734:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7800, rc 0) 00000100:00100000:2.0:1788619922.772797:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74a9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571552512:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 11623us (11800us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619922.772806:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 948 00000100:00100000:3.0:1788619922.776303:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619922.776546:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571552640 00000100:00100000:1.0:1788619922.776556:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571552640 00000100:00100000:1.0:1788619922.776614:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1183 00000100:00100000:1.0:1788619922.776624:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd93a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+31:2354:x1875503571552640:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619922.776747:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619922.776770:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297c00/0x1cd7fe53687356b3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c974 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.776810:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe536873569e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c95f expref: 20 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619922.776850:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78cdcc3800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589166976:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619922.776976:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7600 going to sleep for 11 seconds 00000100:00100000:1.0:1788619922.781408:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619922.781590:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78cdcc3800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589166976:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619922.781616:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:0.0:1788619922.781635:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297c00/0x1cd7fe53687356b3 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c974 expref: 32 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.781658:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619922.781662:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619922.781668:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297c00/0x1cd7fe53687356b3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c974 expref: 32 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.781692:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297c00, rc 0) 00000100:00100000:0.0:1788619922.781753:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd93a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+31:2354:x1875503571552640:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5130us (5459us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619922.781761:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1183 00000100:00100000:2.0:1788619922.783151:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571552768 00000100:00100000:2.0:1788619922.783163:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571552768 00000100:00100000:2.0:1788619922.783220:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 949 00000100:00100000:2.0:1788619922.783230:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5019f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571552768:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619922.783245:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619922.783263:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067546236/0x1cd7fe53687356ba lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c97b expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619922.783320:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c8595c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589167104:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619922.783428:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164b40 going to sleep for 11 seconds 00000100:00100000:2.0:1788619922.786430:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c8595c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589167104:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619922.786469:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000067546236/0x1cd7fe53687356ba lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c97b expref: 8 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.786494:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067546236/0x1cd7fe53687356ba lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c97b expref: 8 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.786524:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6400, rc 0) 00000100:00100000:2.0:1788619922.786585:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5019f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571552768:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3356us (5188us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619922.786593:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 949 00000100:00100000:1.0:1788619922.791828:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619922.791872:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619922.791963:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571552896 00000100:00100000:2.0:1788619922.791973:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571552896 00000100:00100000:2.0:1788619922.792036:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1184 00000100:00100000:2.0:1788619922.792048:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e6479880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+31:2355:x1875503571552896:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619922.792069:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619922.792088:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6e00/0x1cd7fe53687356c1 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c982 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.792124:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe536873569e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c95f expref: 20 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619922.792163:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c8595c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589167232:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619922.792245:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164b40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619922.792409:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571553024 00000100:00100000:0.0:1788619922.792416:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571553024 00000100:00100000:0.0:1788619922.792449:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 950 00000100:00100000:0.0:1788619922.792457:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e6479c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571553024:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619922.792465:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619922.792476:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dfdc9508/0x1cd7fe53687356c8 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c989 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619922.792512:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f25500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589167360:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619922.792542:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619922.799237:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f25500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589167360:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619922.799276:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000dfdc9508/0x1cd7fe53687356c8 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c989 expref: 8 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.799301:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dfdc9508/0x1cd7fe53687356c8 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c989 expref: 8 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.799314:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8078200, rc 0) 00000100:00100000:0.0:1788619922.799427:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e6479c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571553024:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6969us (7552us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619922.799435:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 950 00000100:00100000:2.0:1788619922.799620:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c8595c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589167232:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619922.799635:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:2.0:1788619922.799648:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6e00/0x1cd7fe53687356c1 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c982 expref: 32 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.799662:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619922.799666:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619922.799671:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6e00/0x1cd7fe53687356c1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c982 expref: 32 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.799692:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6e00, rc 0) 00000100:00100000:2.0:1788619922.799716:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e6479880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+31:2355:x1875503571552896:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7671us (7900us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619922.799722:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1184 00000100:00100000:1.0:1788619922.803414:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619922.803547:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571553152 00000100:00100000:2.0:1788619922.803556:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571553152 00000100:00100000:2.0:1788619922.803615:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1185 00000100:00100000:2.0:1788619922.803625:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63fb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+31:2354:x1875503571553152:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619922.803645:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619922.803665:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6400/0x1cd7fe53687356cf lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c990 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.803709:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe536873569e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c95f expref: 20 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619922.803748:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c8595c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589167488:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619922.803835:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164b40 going to sleep for 11 seconds 00000100:00100000:3.0:1788619922.806277:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619922.806375:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571553280 00000100:00100000:0.0:1788619922.806383:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571553280 00000100:00100000:0.0:1788619922.806429:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 951 00000100:00100000:0.0:1788619922.806438:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd93b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571553280:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619922.806462:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619922.806477:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dfdc9508/0x1cd7fe53687356d6 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c997 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619922.806540:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f25500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589167616:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619922.806617:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619922.808816:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c8595c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589167488:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619922.808840:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:2.0:1788619922.808857:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6400/0x1cd7fe53687356cf lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c990 expref: 32 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.808876:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619922.808878:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619922.808885:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6400/0x1cd7fe53687356cf lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c990 expref: 32 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.808898:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6400, rc 0) 00000100:00100000:2.0:1788619922.808948:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63fb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+31:2354:x1875503571553152:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5324us (5542us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619922.808964:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1185 00000100:00100000:0.0:1788619922.812592:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f25500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589167616:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619922.812632:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000dfdc9508/0x1cd7fe53687356d6 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c997 expref: 8 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.812658:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dfdc9508/0x1cd7fe53687356d6 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c997 expref: 8 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.812672:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8078200, rc 0) 00000100:00100000:0.0:1788619922.812735:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd93b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571553280:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6297us (6468us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619922.812744:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 951 00000100:00100000:3.0:1788619922.821435:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619922.822871:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571553536 00000100:00100000:0.0:1788619922.822883:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571553536 00000100:00100000:0.0:1788619922.822960:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1186 00000100:00100000:0.0:1788619922.822972:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd93ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+31:14908:x1875503571553536:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:0.0:1788619922.823030:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619922.823058:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe53687355b7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c878 expref: 31 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.823079:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619922.823084:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788619922.823299:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8078200/0x1cd7fe53687356dd lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.823315:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe5368735586 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c84e expref: 20 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.823376:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8078200/0x1cd7fe53687356dd lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.823413:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe5368735586 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c84e expref: 20 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.823427:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe5368735586 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c84e expref: 20 pid: 11397 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619922.823440:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f25500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589167744:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619922.823562:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619922.826941:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f25500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589167744:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619922.826980:0:5845:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8078200/0x1cd7fe53687356dd lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619922.829059:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619922.829275:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571553664 00000100:00100000:1.0:1788619922.829283:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571553664 00010000:00010000:1.0:1788619922.829364:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe5368735586 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c84e expref: 21 pid: 11397 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619922.829407:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 399 00000100:00100000:1.0:1788619922.829417:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df965f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+21:2354:x1875503571553664:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619922.829431:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619922.829439:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe5368735586 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c84e expref: 21 pid: 11397 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.829465:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe5368735586 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c84e expref: 21 pid: 11397 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.829487:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe5368735586 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c84e expref: 21 pid: 11397 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.829509:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619922.829521:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8078200/0x1cd7fe53687356dd lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.829531:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8078200/0x1cd7fe53687356dd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.829540:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8078200/0x1cd7fe53687356dd lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.829561:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619922.829617:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df965f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+20:2354:x1875503571553664:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 201us (565us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619922.829625:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 399 00010000:00010000:0.0:1788619922.830090:0:5845:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8078200/0x1cd7fe53687356dd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.830100:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8078200/0x1cd7fe53687356dd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788619922.830208:0:5845:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967728 00000004:00010000:0.0:1788619922.830256:0:5845:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1048576 -> 1048576 00000004:00010000:0.0:1788619922.830259:0:5845:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619922 -> 1788619922 00000004:00010000:0.0:1788619922.830261:0:5845:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619922 -> 1788619919 00000004:00010000:0.0:1788619922.830263:0:5845:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619922 -> 1788619922 00000004:00010000:0.0:1788619922.830265:0:5845:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1030 -> 1030 00000004:00010000:0.0:1788619922.830272:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000003625e1ce reply state 000000005e2068ab transno 4294967729 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8078200/0x1cd7fe53687356dd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619922.830280:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788619922.830385:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd93ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+33:14908:x1875503571553536:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 7415us (8958us total) trans 4294967729 rc 0/0 00000100:00100000:0.0:1788619922.830393:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1186 00000020:00080000:0.0:1788619922.830397:0:5845:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1788619922.836143:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00010000:00010000:2.0:1788619922.836242:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8078200/0x1cd7fe53687356dd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.836255:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8078200/0x1cd7fe53687356dd lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.836262:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8078200/0x1cd7fe53687356dd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.836268:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78c8078200 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:2.0:1788619922.836272:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8078200/0x1cd7fe53687356dd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.836280:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8078200/0x1cd7fe53687356dd lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.836295:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619922.836298:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788619922.836303:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8078200/0x1cd7fe53687356dd lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.836309:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8078200/0x1cd7fe53687356dd lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619922.837103:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571553792 00000100:00100000:1.0:1788619922.837114:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571553792 00000100:00100000:1.0:1788619922.837176:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1187 00000100:00100000:1.0:1788619922.837185:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df966a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+32:14908:x1875503571553792:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619922.837207:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619922.837225:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe53687356e4 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c99e expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.837257:0:9417:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe536873569e lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c95f expref: 19 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.837267:0:9417:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe53687356e4 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c99e expref: 33 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.837305:0:9417:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe536873569e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c95f expref: 19 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.837318:0:9417:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe536873569e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c95f expref: 19 pid: 9417 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619922.837358:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63f8700 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589167872:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619922.837439:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e70c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619922.845570:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63f8700 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589167872:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619922.845607:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe53687356e4 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c99e expref: 33 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.845628:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619922.845631:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619922.845636:0:9417:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe53687356e4 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c99e expref: 33 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.845646:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297000, rc 0) 00000100:00100000:1.0:1788619922.845715:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df966a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+33:14908:x1875503571553792:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8530us (9582us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619922.845723:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1187 00000100:00100000:2.0:1788619922.851413:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619922.851797:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571553920 00000100:00100000:3.0:1788619922.851808:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571553920 00002000:00100000:3.0:1788619922.851858:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c8595c00 x1875503571553920/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:18/0 lens 488/0 e 0 to 0 dl 1788619933 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619922.851869:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c8595c00 with x1875503571553920 00010000:00010000:3.0:1788619922.851878:0:5859:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe536873569e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c95f expref: 20 pid: 9417 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.851892:0:5859:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe536873569e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c95f expref: 20 pid: 9417 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788619922.851917:0:5859:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e78c8595c00 00000100:00100000:3.0:1788619922.851960:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 31 00000100:00100000:3.0:1788619922.851970:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8595c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+20:2354:x1875503571553920:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:0.0:1788619922.855818:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1048576 -> 1048576 00000004:00010000:0.0:1788619922.855829:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619922 -> 1788619922 00000004:00010000:0.0:1788619922.855832:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619919 -> 1788619919 00000004:00010000:0.0:1788619922.855834:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619922 -> 1788619922 00000004:00010000:0.0:1788619922.855837:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1030 -> 1030 00000100:00100000:0.0:1788619922.857238:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8595c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+22:2354:x1875503571553920:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 5259us (5826us total) trans 4294967730 rc 0/0 00000100:00100000:0.0:1788619922.857255:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 31 00002000:00100000:0.0:1788619922.857268:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c8595c00 x1875503571553920/t4294967730(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:18/0 lens 488/448 e 0 to 0 dl 1788619933 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619922.857278:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c8595c00 with x1875503571553920 00010000:00010000:0.0:1788619922.857289:0:5859:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe536873569e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c95f expref: 22 pid: 9417 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.857309:0:5859:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe536873569e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c95f expref: 22 pid: 9417 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788619922.857316:0:5859:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e78c8595c00 00000100:00100000:2.0:1788619922.865135:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619922.865238:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571554048 00000100:00100000:3.0:1788619922.865249:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571554048 00010000:00010000:3.0:1788619922.865305:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe536873569e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c95f expref: 22 pid: 9417 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619922.865383:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 400 00000100:00100000:3.0:1788619922.865394:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8595f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+22:2352:x1875503571554048:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619922.865408:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619922.865433:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe536873569e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c95f expref: 22 pid: 9417 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.865448:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe536873569e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c95f expref: 22 pid: 9417 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.865468:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe536873569e lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c95f expref: 22 pid: 9417 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.865483:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619922.865489:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe53687356e4 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c99e expref: 32 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.865499:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe53687356e4 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c99e expref: 32 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.865536:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe53687356e4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c99e expref: 32 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.865576:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe53687356e4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c99e expref: 32 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619922.865589:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df967800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589168000:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619922.865670:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48ce40 going to sleep for 11 seconds 00000100:00100000:3.0:1788619922.871704:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619922.871804:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571554176 00000100:00100000:2.0:1788619922.871814:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571554176 00000100:00100000:2.0:1788619922.871877:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 952 00000100:00100000:2.0:1788619922.871888:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+7:14908:x1875503571554176:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619922.871919:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619922.871937:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d2409dc/0x1cd7fe53687356eb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c9a5 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.871977:0:7654:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe5368735643 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1351680->1355775) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c904 expref: 7 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.871987:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(11 7) 00010000:00010000:2.0:1788619922.871991:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(12 7) 00010000:00010000:2.0:1788619922.871997:0:7654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d2409dc/0x1cd7fe53687356eb lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c9a5 expref: 8 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.872032:0:7654:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe5368735643 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1351680->1355775) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c904 expref: 7 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.872047:0:7654:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe5368735643 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1351680->1355775) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c904 expref: 7 pid: 6696 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619922.872062:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c8596300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589168128:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619922.872145:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164b40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619922.872393:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df967800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589168000:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619922.872415:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619922.872445:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8595f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+21:2352:x1875503571554048:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 7052us (7319us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619922.872455:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 400 00000100:00100000:2.0:1788619922.878179:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619922.878305:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c8596300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589168128:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619922.878377:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000d2409dc/0x1cd7fe53687356eb lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c9a5 expref: 8 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.878401:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6000, rc 0) 00000100:00100000:1.0:1788619922.878458:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+8:14908:x1875503571554176:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6572us (6763us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619922.878468:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 952 00000100:00100000:3.0:1788619922.878749:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571554304 00000100:00100000:3.0:1788619922.878756:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571554304 00010000:00010000:3.0:1788619922.878789:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe5368735643 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1351680->1355775) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c904 expref: 8 pid: 6696 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619922.878823:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 401 00000100:00100000:3.0:1788619922.878831:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8595180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+8:2355:x1875503571554304:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619922.878843:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619922.878865:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe5368735643 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1351680->1355775) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c904 expref: 8 pid: 6696 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.878876:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe5368735643 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1351680->1355775) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c904 expref: 8 pid: 6696 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.878898:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe5368735643 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1351680->1355775) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c904 expref: 8 pid: 6696 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.878931:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(12 7) 00010000:00010000:3.0:1788619922.878935:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(12 7) 00010000:00010000:3.0:1788619922.878942:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000000d2409dc/0x1cd7fe53687356eb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c9a5 expref: 7 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.878951:0:5829:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d2409dc/0x1cd7fe53687356eb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c9a5 expref: 7 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.878995:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000000d2409dc/0x1cd7fe53687356eb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c9a5 expref: 7 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619922.879010:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63ff100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1875503589168256:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619922.879065:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c780 going to sleep for 11 seconds 00000100:00100000:3.0:1788619922.884046:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63ff100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1875503589168256:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619922.884071:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619922.884127:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8595180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571554304:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 5295us (5957us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619922.884138:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 401 00000100:00100000:0.0:1788619922.888582:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619922.888627:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619922.888732:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571554560 00000100:00100000:1.0:1788619922.888742:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571554560 00002000:00010000:1.0:1788619922.888781:0:6701:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:1.0:1788619922.888786:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5f27480 with x1875503571554560 ext(0->18446744073709551615) 00002000:00010000:1.0:1788619922.888797:0:6701:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5f27480. 00000100:00100000:1.0:1788619922.888831:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 560 00000100:00100000:1.0:1788619922.888840:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f27480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571554560:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:1.0:1788619922.889142:0:6701:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 2084447 -> 0 00002000:00010000:1.0:1788619922.889147:0:6701:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619922 -> 1788619922 00002000:00010000:1.0:1788619922.889149:0:6701:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619922 -> 1788619922 00002000:00010000:1.0:1788619922.889151:0:6701:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619922 -> 1788619922 00002000:00010000:1.0:1788619922.889153:0:6701:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 263 -> 263 00000100:00100000:1.0:1788619922.889217:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f27480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2355:x1875503571554560:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 378us (587us total) trans 4294967460 rc 0/0 00000100:00100000:1.0:1788619922.889224:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 560 00002000:00010000:1.0:1788619922.889228:0:6701:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:1.0:1788619922.889231:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5f27480 with x1875503571554560 ext(0->18446744073709551615) 00002000:00010000:1.0:1788619922.889235:0:6701:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5f27480. 00000100:00100000:2.0:1788619922.889367:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571554432 00000100:00100000:2.0:1788619922.889372:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571554432 00002000:00010000:2.0:1788619922.889395:0:5858:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:2.0:1788619922.889399:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5f25500 with x1875503571554432 00000100:00100000:2.0:1788619922.889416:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 742 00000100:00100000:2.0:1788619922.889423:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f25500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+33:2355:x1875503571554432:12345-192.168.201.5@tcp:10:fsx.0 00000004:00010000:2.0:1788619922.889672:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1048576 -> 123008 00000004:00010000:2.0:1788619922.889676:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619922 -> 1788619922 00000004:00010000:2.0:1788619922.889694:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619919 -> 1788619919 00000004:00010000:2.0:1788619922.889697:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619922 -> 1788619922 00000004:00010000:2.0:1788619922.889699:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1030 -> 1030 00000100:00100000:2.0:1788619922.889729:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f25500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+34:2355:x1875503571554432:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 306us (1157us total) trans 4294967731 rc 0/0 00000100:00100000:2.0:1788619922.889735:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 742 00002000:00010000:2.0:1788619922.889738:0:5858:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:2.0:1788619922.889741:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5f25500 with x1875503571554432 00000100:00100000:2.0:1788619922.893145:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619922.893311:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571554688 00000100:00100000:0.0:1788619922.893321:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571554688 00000100:00100000:0.0:1788619922.893590:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1188 00000100:00100000:0.0:1788619922.893600:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8596680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+34:14908:x1875503571554688:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619922.893624:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619922.893644:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe53687356f2 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c9ac expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.893762:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe53687356f9 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.893776:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe53687356f9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.893787:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe53687356f9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619922.893812:0:5846:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 123008 00000004:00200000:0.0:1788619922.893817:0:5846:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619922.893834:0:5846:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe53687356f9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.893861:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe53687356f2 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c9ac expref: 36 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.893879:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe53687356f9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c9ac expref: 35 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.893887:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619922.893890:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619922.893894:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079000, rc 0) 00000100:00100000:0.0:1788619922.894016:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8596680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+35:14908:x1875503571554688:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 417us (882us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619922.894023:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1188 00000100:00100000:2.0:1788619922.900858:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619922.901035:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571554816 00000100:00100000:0.0:1788619922.901046:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571554816 00000100:00100000:0.0:1788619922.901148:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1189 00000100:00100000:0.0:1788619922.901161:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8597100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+21:14908:x1875503571554816:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619922.901186:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619922.901208:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079600/0x1cd7fe5368735700 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c9b3 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.901306:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8078200/0x1cd7fe5368735707 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.901321:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8078200/0x1cd7fe5368735707 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.901360:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8078200/0x1cd7fe5368735707 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619922.901388:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 123008 00000004:00200000:0.0:1788619922.901393:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619922.901411:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8078200/0x1cd7fe5368735707 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.901436:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079600/0x1cd7fe5368735700 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c9b3 expref: 23 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.901463:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8078200/0x1cd7fe5368735707 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c9b3 expref: 22 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.901470:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619922.901473:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619922.901477:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8078200, rc 0) 00000100:00100000:0.0:1788619922.901544:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8597100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+22:14908:x1875503571554816:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 386us (698us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619922.901551:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1189 00000100:00100000:1.0:1788619922.910371:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619922.910422:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619922.910456:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571555072 00000100:00100000:1.0:1788619922.910467:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571555072 00000100:00100000:0.0:1788619922.910634:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571554944 00000100:00100000:0.0:1788619922.910638:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571554944 00000100:00100000:0.0:1788619922.910678:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1190 00000100:00100000:0.0:1788619922.910698:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63f8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+22:2355:x1875503571554944:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619922.910718:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619922.910737:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079800/0x1cd7fe536873570e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c9ba expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.910771:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe53687356e4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c99e expref: 34 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619922.910808:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f25180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589168384:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619922.910884:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebb40 going to sleep for 11 seconds 00000100:00100000:1.0:1788619922.911316:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 953 00000100:00100000:1.0:1788619922.911346:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63f8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571555072:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619922.911355:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619922.911368:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff40cdc2/0x1cd7fe5368735715 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c9c1 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619922.911404:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63f8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589168512:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619922.911439:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e70c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619922.917601:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63f8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589168512:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619922.917640:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ff40cdc2/0x1cd7fe5368735715 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c9c1 expref: 8 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.917667:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff40cdc2/0x1cd7fe5368735715 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c9c1 expref: 8 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.917696:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297a00, rc 0) 00000100:00100000:2.0:1788619922.917805:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63f8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571555072:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6481us (7382us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619922.917815:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 953 00000004:00010000:1.0:1788619922.917850:0:9417:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619919 -> 1788619922 00000100:00100000:1.0:1788619922.917871:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f25180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589168384:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619922.917885:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 123008 00010000:00010000:1.0:1788619922.917898:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079800/0x1cd7fe536873570e lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c9ba expref: 23 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.917912:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619922.917915:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619922.917919:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079800/0x1cd7fe536873570e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c9ba expref: 23 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.917930:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079800, rc 0) 00000100:00100000:1.0:1788619922.917954:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63f8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+22:2355:x1875503571554944:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7259us (7595us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619922.917961:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1190 00000100:00100000:2.0:1788619922.921756:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619922.921794:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619922.921847:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571555200 00000100:00100000:3.0:1788619922.921857:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571555200 00000100:00100000:3.0:1788619922.921915:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1191 00000100:00100000:3.0:1788619922.921925:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8597800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+22:2355:x1875503571555200:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619922.921945:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619922.921985:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1494200/0x1cd7fe536873571c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c9c8 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.922020:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe53687356e4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c99e expref: 34 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619922.922057:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63ff100 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589168640:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619922.922130:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571555328 00000100:00100000:0.0:1788619922.922133:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571555328 00000100:00100000:0.0:1788619922.922165:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 954 00000100:00100000:0.0:1788619922.922172:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8596300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571555328:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619922.922179:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619922.922189:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000855993/0x1cd7fe5368735723 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c9cf expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619922.922221:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f27800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589168768:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619922.922738:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c780 going to sleep for 11 seconds 00000100:00100000:0.0:1788619922.922811:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619922.925289:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f27800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589168768:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619922.925619:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000000855993/0x1cd7fe5368735723 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c9cf expref: 8 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.925646:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000855993/0x1cd7fe5368735723 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c9cf expref: 8 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.925658:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079c00, rc 0) 00000100:00100000:0.0:1788619922.925772:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8596300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571555328:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3599us (3976us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619922.925781:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 954 00000100:00100000:2.0:1788619922.926217:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63ff100 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589168640:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619922.926234:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 123008 00010000:00010000:2.0:1788619922.926247:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1494200/0x1cd7fe536873571c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c9c8 expref: 23 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.926260:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619922.926264:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619922.926268:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1494200/0x1cd7fe536873571c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c9c8 expref: 23 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.926280:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c1494200, rc 0) 00000100:00100000:2.0:1788619922.926668:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8597800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+22:2355:x1875503571555200:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4743us (4919us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619922.926676:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1191 00000100:00100000:1.0:1788619922.931061:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619922.931384:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619922.937002:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571555456 00000100:00100000:0.0:1788619922.937016:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571555456 00000100:00100000:0.0:1788619922.937083:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1192 00000100:00100000:0.0:1788619922.937094:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63f9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+22:2355:x1875503571555456:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619922.937115:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619922.937137:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079600/0x1cd7fe536873572a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c9d6 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619922.937175:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe53687356e4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c99e expref: 34 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619922.937218:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f27800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589168896:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619922.937344:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619922.938191:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571555584 00000100:00100000:3.0:1788619922.938198:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571555584 00000100:00100000:3.0:1788619922.938248:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 955 00000100:00100000:3.0:1788619922.938257:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63faa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571555584:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619922.938268:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619922.938281:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000064837316/0x1cd7fe5368735731 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c9dd expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619922.938363:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63fc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589169024:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619922.938409:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c780 going to sleep for 11 seconds 00000100:00100000:3.0:1788619922.944105:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63fc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589169024:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619922.944142:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000064837316/0x1cd7fe5368735731 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c9dd expref: 8 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.944167:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000064837316/0x1cd7fe5368735731 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c9dd expref: 8 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619922.944179:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c1495a00, rc 0) 00000100:00100000:3.0:1788619922.944250:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63faa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571555584:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5994us (12866us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619922.944259:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 955 00000100:00100000:2.0:1788619922.945239:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f27800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589168896:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619922.945259:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 123008 00010000:00010000:2.0:1788619922.945273:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079600/0x1cd7fe536873572a lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c9d6 expref: 23 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.945296:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619922.945299:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619922.945303:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079600/0x1cd7fe536873572a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c9d6 expref: 23 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.945313:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079600, rc 0) 00000100:00100000:2.0:1788619922.945400:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63f9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+22:2355:x1875503571555456:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8307us (14347us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619922.945407:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1192 00000100:00100000:0.0:1788619922.948885:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619922.948938:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619922.949055:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571555840 00000100:00100000:2.0:1788619922.949066:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571555840 00000100:00100000:2.0:1788619922.949128:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 956 00000100:00100000:2.0:1788619922.949139:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f26d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571555840:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619922.949157:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619922.949175:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001dc9de0/0x1cd7fe5368735738 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c9eb expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619922.949234:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5026d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589169152:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619922.949319:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164e40 going to sleep for 11 seconds 00000100:00100000:1.0:1788619922.950070:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571555712 00000100:00100000:1.0:1788619922.950077:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571555712 00000100:00100000:1.0:1788619922.950125:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1193 00000100:00100000:1.0:1788619922.950133:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f24a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+22:2355:x1875503571555712:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619922.950146:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619922.950161:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296a00/0x1cd7fe536873573f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c9e4 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.950190:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe53687356e4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c99e expref: 34 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619922.950218:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63fbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589169280:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619922.950281:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7780 going to sleep for 11 seconds 00000100:00100000:2.0:1788619922.955695:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5026d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589169152:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619922.955734:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000001dc9de0/0x1cd7fe5368735738 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c9eb expref: 8 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.955758:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001dc9de0/0x1cd7fe5368735738 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c9eb expref: 8 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.955771:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6a00, rc 0) 00000100:00100000:2.0:1788619922.955825:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f26d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571555840:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6688us (6885us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619922.955833:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 956 00000100:00100000:1.0:1788619922.963715:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63fbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589169280:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619922.963742:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 123008 00010000:00010000:1.0:1788619922.963761:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296a00/0x1cd7fe536873573f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c9e4 expref: 23 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.963780:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619922.963784:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619922.963790:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296a00/0x1cd7fe536873573f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c9e4 expref: 23 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.963805:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296a00, rc 0) 00000100:00100000:1.0:1788619922.963868:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f24a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+22:2355:x1875503571555712:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 13734us (14991us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619922.963876:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1193 00000100:00100000:3.0:1788619922.970194:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619922.970273:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571555968 00000100:00100000:1.0:1788619922.970283:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571555968 00002000:00100000:1.0:1788619922.970366:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c63fc700 x1875503571555968/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:18/0 lens 488/0 e 0 to 0 dl 1788619933 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619922.970377:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c63fc700 with x1875503571555968 00000100:00100000:1.0:1788619922.970412:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 743 00000100:00100000:1.0:1788619922.970421:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63fc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+35:2355:x1875503571555968:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619922.973580:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63fc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+35:2355:x1875503571555968:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 3149us (3384us total) trans 0 rc 126976/126976 00000100:00100000:1.0:1788619922.973595:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 743 00002000:00100000:1.0:1788619922.973608:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c63fc700 x1875503571555968/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:18/0 lens 488/440 e 0 to 0 dl 1788619933 ref 1 fl Complete:/600/0 rc 126976/126976 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619922.973619:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c63fc700 with x1875503571555968 00000100:00100000:3.0:1788619922.977827:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619922.977931:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571556096 00000100:00100000:2.0:1788619922.977942:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571556096 00000100:00100000:2.0:1788619922.978018:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1194 00000100:00100000:2.0:1788619922.978029:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63fc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+22:2354:x1875503571556096:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619922.978050:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619922.978072:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe5368735746 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c9f2 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619922.978110:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe53687356e4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c99e expref: 34 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619922.978153:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5026d80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589169408:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619922.978244:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164e40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619922.981220:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619922.982775:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571556224 00000100:00100000:2.0:1788619922.982787:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571556224 00000100:00100000:2.0:1788619922.982858:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 957 00000100:00100000:2.0:1788619922.982868:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f26300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571556224:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619922.982885:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619922.982913:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003a2d9d44/0x1cd7fe536873574d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1c9f9 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619922.982993:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c75f7100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589169536:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619922.983087:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164480 going to sleep for 11 seconds 00000100:00100000:1.0:1788619922.983761:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5026d80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589169408:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619922.983784:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 123008 00010000:00010000:1.0:1788619922.983802:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe5368735746 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c9f2 expref: 23 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.983820:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619922.983823:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619922.983830:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe5368735746 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c9f2 expref: 23 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.983845:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6a00, rc 0) 00000100:00100000:1.0:1788619922.983892:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63fc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+22:2354:x1875503571556096:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5865us (6075us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619922.983914:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1194 00000100:00100000:1.0:1788619922.987543:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c75f7100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589169536:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619922.987585:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003a2d9d44/0x1cd7fe536873574d lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c9f9 expref: 8 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.987610:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a2d9d44/0x1cd7fe536873574d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c9f9 expref: 8 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619922.987622:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7600, rc 0) 00000100:00100000:1.0:1788619922.987695:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f26300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571556224:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4827us (6484us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619922.987704:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 957 00000100:00100000:0.0:1788619922.999108:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619922.999164:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619922.999181:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571556480 00000100:00100000:1.0:1788619922.999192:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571556480 00000100:00100000:1.0:1788619922.999252:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 958 00000100:00100000:1.0:1788619922.999264:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f25f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571556480:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619922.999285:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619922.999303:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006901c1d2/0x1cd7fe5368735754 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ca07 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619922.999381:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63fbb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589169664:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619922.999469:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7780 going to sleep for 11 seconds 00000100:00100000:1.0:1788619923.006021:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63fbb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589169664:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619923.006061:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000006901c1d2/0x1cd7fe5368735754 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca07 expref: 8 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.006087:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006901c1d2/0x1cd7fe5368735754 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca07 expref: 8 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.006100:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296a00, rc 0) 00000100:00100000:1.0:1788619923.006208:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f25f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571556480:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6945us (7108us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619923.006217:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 958 00000100:00100000:2.0:1788619923.006706:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571556352 00000100:00100000:2.0:1788619923.006715:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571556352 00000100:00100000:2.0:1788619923.006770:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1195 00000100:00100000:2.0:1788619923.006779:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f27100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+22:2355:x1875503571556352:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619923.006797:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619923.006815:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6e00/0x1cd7fe536873575b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ca00 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.006848:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe53687356e4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c99e expref: 34 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619923.006888:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c75f7100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589169792:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619923.007084:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81649c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619923.019282:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c75f7100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589169792:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619923.019311:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 123008 00010000:00010000:1.0:1788619923.019361:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6e00/0x1cd7fe536873575b lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca00 expref: 23 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.019381:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619923.019384:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619923.019389:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6e00/0x1cd7fe536873575b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca00 expref: 23 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.019404:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6e00, rc 0) 00000100:00100000:1.0:1788619923.019465:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f27100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+22:2355:x1875503571556352:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 12687us (20300us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619923.019474:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1195 00000100:00100000:1.0:1788619923.023153:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619923.023253:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571556608 00000100:00100000:0.0:1788619923.023266:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571556608 00000100:00100000:0.0:1788619923.023362:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1196 00000100:00100000:0.0:1788619923.023373:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63fbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+22:14908:x1875503571556608:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619923.023395:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619923.023416:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079800/0x1cd7fe5368735762 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ca0e expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619923.023450:0:9417:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe53687356e4 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c99e expref: 34 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619923.023460:0:9417:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079800/0x1cd7fe5368735762 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca0e expref: 23 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619923.023496:0:9417:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe53687356e4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c99e expref: 34 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619923.023523:0:9417:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe53687356e4 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c99e expref: 34 pid: 9417 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619923.023539:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f24700 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589169920:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619923.023638:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619923.027531:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619923.027638:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f24700 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589169920:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619923.027697:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079800/0x1cd7fe5368735762 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca0e expref: 23 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619923.027722:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619923.027725:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619923.027729:0:9417:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079800/0x1cd7fe5368735762 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca0e expref: 23 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619923.027739:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079800, rc 0) 00000100:00100000:0.0:1788619923.027799:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63fbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+23:14908:x1875503571556608:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4427us (4656us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619923.027807:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1196 00000100:00100000:1.0:1788619923.028141:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571556736 00000100:00100000:1.0:1788619923.028148:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571556736 00010000:00010000:1.0:1788619923.028180:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe53687356e4 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c99e expref: 35 pid: 9417 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619923.028205:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 402 00000100:00100000:1.0:1788619923.028214:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63fc380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+35:2355:x1875503571556736:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619923.028227:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619923.028251:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe53687356e4 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c99e expref: 35 pid: 9417 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.028263:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe53687356e4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c99e expref: 35 pid: 9417 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.028282:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe53687356e4 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c99e expref: 35 pid: 9417 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.028295:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619923.028301:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079800/0x1cd7fe5368735762 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca0e expref: 22 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.028310:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079800/0x1cd7fe5368735762 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca0e expref: 22 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.028319:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079800/0x1cd7fe5368735762 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca0e expref: 22 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.028378:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079800/0x1cd7fe5368735762 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca0e expref: 22 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619923.028395:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63fa680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589170048:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619923.028594:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7780 going to sleep for 11 seconds 00000100:00100000:3.0:1788619923.032265:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619923.032433:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63fa680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589170048:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619923.032455:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619923.032501:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63fc380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+34:2355:x1875503571556736:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 4287us (4989us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619923.032512:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 402 00000100:00100000:0.0:1788619923.033581:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571556864 00000100:00100000:0.0:1788619923.033590:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571556864 00000100:00100000:0.0:1788619923.033747:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 744 00000100:00100000:0.0:1788619923.033758:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63fd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+23:2355:x1875503571556864:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:0.0:1788619923.034050:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 123008 -> 123008 00000004:00010000:0.0:1788619923.034055:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619922 -> 1788619922 00000004:00010000:0.0:1788619923.034058:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619922 -> 1788619922 00000004:00010000:0.0:1788619923.034060:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619922 -> 1788619922 00000004:00010000:0.0:1788619923.034063:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1030 -> 1030 00000100:00100000:0.0:1788619923.034129:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63fd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+24:2355:x1875503571556864:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 373us (1874us total) trans 4294967732 rc 0/0 00000100:00100000:0.0:1788619923.034137:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 744 00000100:00100000:2.0:1788619923.044675:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619923.044758:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571556992 00000100:00100000:3.0:1788619923.044768:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571556992 00000100:00100000:3.0:1788619923.044824:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1197 00000100:00100000:3.0:1788619923.044834:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+34:2355:x1875503571556992:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619923.044856:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619923.044876:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368735769 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ca15 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.044913:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079800/0x1cd7fe5368735762 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca0e expref: 23 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619923.044966:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63fea00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589170176:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619923.045039:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c780 going to sleep for 11 seconds 00000100:00100000:0.0:1788619923.050387:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63fea00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589170176:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619923.050416:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 123008 00010000:00010000:0.0:1788619923.050436:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368735769 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca15 expref: 35 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619923.050458:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619923.050461:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619923.050467:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368735769 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca15 expref: 35 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619923.050482:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d000, rc 0) 00000100:00100000:0.0:1788619923.050534:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+34:2355:x1875503571556992:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5701us (5866us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619923.050543:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1197 00000100:00100000:3.0:1788619923.055389:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619923.055535:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571557120 00000100:00100000:1.0:1788619923.055545:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571557120 00000100:00100000:1.0:1788619923.055609:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1198 00000100:00100000:1.0:1788619923.055620:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63ffb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+34:2354:x1875503571557120:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619923.055652:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619923.055674:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe5368735770 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ca1c expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.055724:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079800/0x1cd7fe5368735762 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca0e expref: 23 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619923.055766:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63fa680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589170304:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619923.055921:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7780 going to sleep for 11 seconds 00000100:00100000:1.0:1788619923.059756:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63fa680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589170304:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619923.059782:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 123008 00010000:00010000:1.0:1788619923.059799:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe5368735770 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca1c expref: 35 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.059824:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619923.059827:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619923.059833:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe5368735770 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca1c expref: 35 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.059849:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297000, rc 0) 00000100:00100000:1.0:1788619923.059909:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63ffb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+34:2354:x1875503571557120:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4291us (4531us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619923.059918:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1198 00000100:00100000:3.0:1788619923.063063:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619923.063317:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571557376 00000100:00100000:0.0:1788619923.063363:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571557376 00000100:00100000:0.0:1788619923.063427:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1199 00000100:00100000:0.0:1788619923.063438:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63fe680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+24:14908:x1875503571557376:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:0.0:1788619923.063497:0:9417:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619923.063523:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8078200/0x1cd7fe5368735707 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c9b3 expref: 24 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619923.063543:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619923.063548:0:9417:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788619923.063868:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368735777 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619923.063884:0:9417:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe53687356f9 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c9ac expref: 33 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619923.063893:0:9417:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368735777 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619923.063926:0:9417:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe53687356f9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c9ac expref: 33 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619923.063941:0:9417:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe53687356f9 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c9ac expref: 33 pid: 5846 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619923.064023:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f24700 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589170432:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619923.064115:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebf00 going to sleep for 11 seconds 00000100:00100000:0.0:1788619923.065650:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f24700 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589170432:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619923.065688:0:9417:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368735777 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619923.066170:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619923.066471:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571557504 00000100:00100000:2.0:1788619923.066478:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571557504 00010000:00010000:2.0:1788619923.066515:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe53687356f9 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c9ac expref: 34 pid: 5846 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619923.066548:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 403 00000100:00100000:2.0:1788619923.066557:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+34:2355:x1875503571557504:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619923.066572:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619923.066579:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe53687356f9 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c9ac expref: 34 pid: 5846 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.066591:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe53687356f9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c9ac expref: 34 pid: 5846 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.066611:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe53687356f9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c9ac expref: 34 pid: 5846 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.066627:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619923.066631:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368735777 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.066641:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368735777 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.066648:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368735777 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.068912:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619923.069047:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+33:2355:x1875503571557504:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 2483us (2869us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619923.069061:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 403 00010000:00010000:1.0:1788619923.073430:0:9417:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368735777 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.073451:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368735777 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788619923.073703:0:9417:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967733 00000004:00010000:3.0:1788619923.073774:0:9417:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 123008 -> 123008 00000004:00010000:3.0:1788619923.073777:0:9417:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619922 -> 1788619922 00000004:00010000:3.0:1788619923.073780:0:9417:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619922 -> 1788619922 00000004:00010000:3.0:1788619923.073782:0:9417:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619922 -> 1788619922 00000004:00010000:3.0:1788619923.073784:0:9417:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1030 -> 1030 00000004:00010000:3.0:1788619923.073791:0:9417:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000002acba989 reply state 000000007201b94d transno 4294967734 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368735777 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619923.073800:0:9417:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788619923.075024:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63fe680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+26:14908:x1875503571557376:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 11579us (11961us total) trans 4294967734 rc 0/0 00000100:00100000:3.0:1788619923.075035:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1199 00000020:00080000:3.0:1788619923.075042:0:9417:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788619923.081361:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368735777 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619923.081381:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368735777 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619923.081390:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368735777 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619923.081396:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78c8079400 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788619923.081401:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368735777 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619923.081410:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368735777 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619923.081426:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619923.081429:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788619923.081434:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368735777 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619923.081441:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368735777 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619923.087961:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619923.088206:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571557632 00000100:00100000:3.0:1788619923.088217:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571557632 00000100:00100000:3.0:1788619923.088317:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 959 00000100:00100000:3.0:1788619923.088361:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:14908:x1875503571557632:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619923.088381:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619923.088401:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe536873577e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ca23 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.088434:0:6698:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000000d2409dc/0x1cd7fe53687356eb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c9a5 expref: 8 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.088444:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(12 7) 00010000:00010000:3.0:1788619923.088448:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(13 7) 00010000:00010000:3.0:1788619923.088455:0:6698:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe536873577e lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca23 expref: 8 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.088512:0:6698:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000000d2409dc/0x1cd7fe53687356eb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c9a5 expref: 8 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.088527:0:6698:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000000d2409dc/0x1cd7fe53687356eb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c9a5 expref: 8 pid: 7654 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619923.088541:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63fce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589170560:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619923.088625:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48ce40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619923.091819:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63fce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589170560:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619923.091854:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe536873577e lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca23 expref: 8 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619923.091876:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683ca00, rc 0) 00000100:00100000:0.0:1788619923.091936:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+8:14908:x1875503571557632:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3608us (3985us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619923.091944:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 959 00000100:00100000:3.0:1788619923.093190:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619923.093271:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571557760 00000100:00100000:2.0:1788619923.093279:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571557760 00010000:00010000:2.0:1788619923.093316:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000000d2409dc/0x1cd7fe53687356eb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c9a5 expref: 9 pid: 7654 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619923.093389:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 404 00000100:00100000:2.0:1788619923.093399:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8558050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571557760:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619923.093414:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619923.093437:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000000d2409dc/0x1cd7fe53687356eb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c9a5 expref: 9 pid: 7654 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.093451:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000000d2409dc/0x1cd7fe53687356eb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c9a5 expref: 9 pid: 7654 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.093475:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d2409dc/0x1cd7fe53687356eb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1c9a5 expref: 9 pid: 7654 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.093490:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(13 7) 00010000:00010000:2.0:1788619923.093494:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(13 7) 00010000:00010000:2.0:1788619923.093502:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe536873577e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca23 expref: 7 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.093512:0:5831:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe536873577e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca23 expref: 7 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.093548:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe536873577e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca23 expref: 7 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619923.093562:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cd8a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5831:1875503589170688:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619923.093625:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164240 going to sleep for 11 seconds 00000100:00100000:0.0:1788619923.102838:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cd8a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5831:1875503589170688:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619923.102867:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619923.102922:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8558050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571557760:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 9524us (9738us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619923.102933:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 404 00000100:00100000:3.0:1788619923.103157:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619923.103244:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619923.103363:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571557888 00000100:00100000:1.0:1788619923.103370:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571557888 00002000:00010000:1.0:1788619923.103406:0:5859:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:1.0:1788619923.103412:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c63ff480 with x1875503571557888 00000100:00100000:1.0:1788619923.103435:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 745 00000100:00100000:1.0:1788619923.103444:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63ff480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+25:2355:x1875503571557888:12345-192.168.201.5@tcp:10:fsx.0 00000004:00010000:1.0:1788619923.103631:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 123008 -> 678955 00000004:00010000:1.0:1788619923.103635:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619922 -> 1788619922 00000004:00010000:1.0:1788619923.103637:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619922 -> 1788619922 00000004:00010000:1.0:1788619923.103639:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619922 -> 1788619922 00000004:00010000:1.0:1788619923.103642:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1030 -> 1030 00000100:00100000:1.0:1788619923.103694:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63ff480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+26:2355:x1875503571557888:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 250us (449us total) trans 4294967735 rc 0/0 00000100:00100000:1.0:1788619923.103703:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 745 00002000:00010000:1.0:1788619923.103706:0:5859:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:1.0:1788619923.103708:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c63ff480 with x1875503571557888 00000100:00100000:2.0:1788619923.103805:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571558016 00000100:00100000:2.0:1788619923.103809:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571558016 00002000:00010000:2.0:1788619923.103828:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:2.0:1788619923.103832:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c63fd500 with x1875503571558016 ext(0->18446744073709551615) 00002000:00010000:2.0:1788619923.103840:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c63fd500. 00000100:00100000:2.0:1788619923.103859:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 561 00000100:00100000:2.0:1788619923.103865:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63fd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+8:2355:x1875503571558016:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:2.0:1788619923.104010:0:6703:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:2.0:1788619923.104013:0:6703:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619922 -> 1788619922 00002000:00010000:2.0:1788619923.104015:0:6703:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619922 -> 1788619922 00002000:00010000:2.0:1788619923.104017:0:6703:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619922 -> 1788619922 00002000:00010000:2.0:1788619923.104019:0:6703:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 263 -> 263 00000100:00100000:2.0:1788619923.104041:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63fd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571558016:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 177us (886us total) trans 4294967461 rc 0/0 00000100:00100000:2.0:1788619923.104048:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 561 00002000:00010000:2.0:1788619923.104050:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:2.0:1788619923.104053:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c63fd500 with x1875503571558016 ext(0->18446744073709551615) 00002000:00010000:2.0:1788619923.104057:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c63fd500. 00000100:00100000:3.0:1788619923.109706:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619923.109784:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571558144 00000100:00100000:0.0:1788619923.109794:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571558144 00000100:00100000:0.0:1788619923.109856:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1200 00000100:00100000:0.0:1788619923.109865:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63fd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+26:14908:x1875503571558144:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619923.109889:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619923.109909:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079600/0x1cd7fe5368735785 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ca2a expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619923.110011:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe536873578c lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619923.110027:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe536873578c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619923.110039:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe536873578c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619923.110066:0:11397:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 678955 00000004:00200000:0.0:1788619923.110070:0:11397:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619923.110088:0:11397:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe536873578c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619923.110113:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079600/0x1cd7fe5368735785 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca2a expref: 28 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619923.110130:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe536873578c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca2a expref: 27 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619923.110138:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619923.110141:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619923.110145:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079000, rc 0) 00000100:00100000:0.0:1788619923.110254:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63fd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+27:14908:x1875503571558144:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 389us (557us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619923.110261:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1200 00000100:00100000:0.0:1788619923.124144:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619923.124211:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571558272 00000100:00100000:1.0:1788619923.124221:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571558272 00000100:00100000:1.0:1788619923.124276:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1201 00000100:00100000:1.0:1788619923.124286:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f24700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+33:14908:x1875503571558272:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619923.124307:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619923.124353:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe5368735793 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ca31 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.124444:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297800/0x1cd7fe536873579a lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.124458:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297800/0x1cd7fe536873579a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.124470:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297800/0x1cd7fe536873579a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788619923.124497:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 678955 00000004:00200000:1.0:1788619923.124502:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788619923.124519:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297800/0x1cd7fe536873579a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.124543:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe5368735793 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca31 expref: 35 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.124563:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297800/0x1cd7fe536873579a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca31 expref: 34 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.124571:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619923.124573:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788619923.124576:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297800, rc 0) 00000100:00100000:1.0:1788619923.124634:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f24700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+34:14908:x1875503571558272:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 349us (499us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619923.124642:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1201 00000100:00100000:3.0:1788619923.128098:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619923.128543:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571558400 00000100:00100000:0.0:1788619923.128553:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571558400 00000100:00100000:0.0:1788619923.128757:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1202 00000100:00100000:0.0:1788619923.128769:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63ff800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+34:2355:x1875503571558400:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619923.128790:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619923.128810:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079600/0x1cd7fe53687357a1 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ca38 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619923.128847:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079800/0x1cd7fe5368735762 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca0e expref: 26 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619923.128889:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f24000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589170816:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619923.128987:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb540 going to sleep for 11 seconds 00000100:00100000:0.0:1788619923.130773:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619923.130923:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571558528 00000100:00100000:3.0:1788619923.130931:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571558528 00000100:00100000:3.0:1788619923.131082:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 960 00000100:00100000:3.0:1788619923.131092:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f24380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571558528:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619923.131108:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619923.131124:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe53687357a8 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ca3f expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619923.131179:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63fca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589170944:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619923.131257:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48cd80 going to sleep for 11 seconds 00000100:00100000:0.0:1788619923.135276:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f24000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589170816:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619923.135305:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 678955 00010000:00010000:0.0:1788619923.135366:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079600/0x1cd7fe53687357a1 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca38 expref: 35 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619923.135388:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619923.135391:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619923.135397:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079600/0x1cd7fe53687357a1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca38 expref: 35 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619923.135411:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079600, rc 0) 00000100:00100000:0.0:1788619923.135596:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63ff800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+34:2355:x1875503571558400:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6828us (7507us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619923.135605:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1202 00000100:00100000:3.0:1788619923.140627:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63fca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589170944:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619923.140703:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe53687357a8 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca3f expref: 9 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.140729:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe53687357a8 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca3f expref: 9 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.140740:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c200, rc 0) 00000100:00100000:3.0:1788619923.140841:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f24380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571558528:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 9750us (10076us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619923.140850:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 960 00000100:00100000:3.0:1788619923.146551:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619923.146597:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619923.146745:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571558784 00000100:00100000:2.0:1788619923.146756:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571558784 00000100:00100000:2.0:1788619923.146823:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 961 00000100:00100000:2.0:1788619923.146833:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571558784:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619923.146851:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619923.146871:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d2409dc/0x1cd7fe53687357af lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ca4d expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619923.146931:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cd8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589171072:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619923.147015:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164d80 going to sleep for 11 seconds 00000100:00100000:1.0:1788619923.147062:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571558656 00000100:00100000:1.0:1788619923.147067:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571558656 00000100:00100000:1.0:1788619923.147104:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1203 00000100:00100000:1.0:1788619923.147112:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63fca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+34:2353:x1875503571558656:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619923.147122:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619923.147134:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296e00/0x1cd7fe53687357b6 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ca46 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.147161:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079800/0x1cd7fe5368735762 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca0e expref: 26 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619923.147186:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63fa680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589171200:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619923.147212:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7780 going to sleep for 11 seconds 00000100:00100000:2.0:1788619923.150145:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cd8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589171072:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619923.150186:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000d2409dc/0x1cd7fe53687357af lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca4d expref: 9 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.150210:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d2409dc/0x1cd7fe53687357af lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca4d expref: 9 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.150221:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6000, rc 0) 00000100:00100000:2.0:1788619923.150288:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571558784:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3456us (3690us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619923.150296:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 961 00000100:00100000:1.0:1788619923.150418:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63fa680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589171200:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619923.150435:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 678955 00010000:00010000:1.0:1788619923.150448:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296e00/0x1cd7fe53687357b6 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca46 expref: 35 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.150464:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619923.150468:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619923.150473:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296e00/0x1cd7fe53687357b6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca46 expref: 35 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.150485:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296e00, rc 0) 00000100:00100000:1.0:1788619923.150520:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63fca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+34:2353:x1875503571558656:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3409us (3979us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619923.150527:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1203 00000100:00100000:3.0:1788619923.154629:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619923.155762:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619923.155898:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571559040 00000100:00100000:1.0:1788619923.155919:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571559040 00000100:00100000:1.0:1788619923.155981:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 962 00000100:00100000:1.0:1788619923.155992:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63fdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571559040:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619923.156010:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619923.156027:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6252eaa/0x1cd7fe53687357bd lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ca5b expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619923.156103:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63fa680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589171328:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619923.156181:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7780 going to sleep for 11 seconds 00000100:00100000:2.0:1788619923.157546:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571558912 00000100:00100000:2.0:1788619923.157558:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571558912 00000100:00100000:2.0:1788619923.159366:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1204 00000100:00100000:2.0:1788619923.159388:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63fe300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+34:2353:x1875503571558912:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619923.159408:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619923.159428:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe53687357c4 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ca54 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.159467:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079800/0x1cd7fe5368735762 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca0e expref: 26 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619923.159506:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cd8a80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589171456:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619923.159592:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164d80 going to sleep for 11 seconds 00000100:00100000:2.0:1788619923.161976:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63fa680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589171328:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619923.162017:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a6252eaa/0x1cd7fe53687357bd lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca5b expref: 9 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.162048:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6252eaa/0x1cd7fe53687357bd lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca5b expref: 9 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.162072:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297600, rc 0) 00000100:00100000:2.0:1788619923.162126:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63fdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571559040:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6136us (6369us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619923.162134:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 962 00000100:00100000:3.0:1788619923.164663:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cd8a80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589171456:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619923.164703:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 678955 00010000:00010000:3.0:1788619923.164724:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe53687357c4 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca54 expref: 35 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.164755:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619923.164758:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619923.164764:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe53687357c4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca54 expref: 35 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.164786:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6000, rc 0) 00000100:00100000:3.0:1788619923.164847:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63fe300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+34:2353:x1875503571558912:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5467us (10227us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619923.164856:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1204 00000100:00100000:0.0:1788619923.167755:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619923.168270:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571559168 00000100:00100000:3.0:1788619923.168281:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571559168 00000100:00100000:3.0:1788619923.168366:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1205 00000100:00100000:3.0:1788619923.168378:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f24000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+34:14908:x1875503571559168:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619923.168400:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619923.168422:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687357cb lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ca62 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.168451:0:9417:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079800/0x1cd7fe5368735762 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca0e expref: 26 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.168461:0:9417:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687357cb lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca62 expref: 35 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.168495:0:9417:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079800/0x1cd7fe5368735762 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca0e expref: 26 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.168603:0:9417:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079800/0x1cd7fe5368735762 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca0e expref: 26 pid: 9417 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619923.168616:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63fc380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589171584:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619923.168709:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48cd80 going to sleep for 11 seconds 00000100:00100000:0.0:1788619923.171023:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619923.171491:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63fc380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589171584:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619923.171523:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687357cb lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca62 expref: 35 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.171545:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619923.171548:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619923.171553:0:9417:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687357cb lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca62 expref: 35 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.171563:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683da00, rc 0) 00000100:00100000:3.0:1788619923.171608:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f24000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+35:14908:x1875503571559168:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3233us (3872us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619923.171616:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1205 00000100:00100000:1.0:1788619923.172511:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571559296 00000100:00100000:1.0:1788619923.172520:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571559296 00010000:00010000:1.0:1788619923.172554:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079800/0x1cd7fe5368735762 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca0e expref: 27 pid: 9417 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619923.172621:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 405 00000100:00100000:1.0:1788619923.172631:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f25880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+27:2355:x1875503571559296:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619923.172646:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619923.172670:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079800/0x1cd7fe5368735762 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca0e expref: 27 pid: 9417 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.172696:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079800/0x1cd7fe5368735762 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca0e expref: 27 pid: 9417 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.172719:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079800/0x1cd7fe5368735762 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca0e expref: 27 pid: 9417 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.172734:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619923.172748:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687357cb lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca62 expref: 34 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.172760:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687357cb lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca62 expref: 34 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.172769:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687357cb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca62 expref: 34 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.172871:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687357cb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca62 expref: 34 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619923.172886:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63f9c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589171712:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619923.172961:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7f00 going to sleep for 11 seconds 00000100:00100000:0.0:1788619923.178144:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619923.178254:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63f9c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589171712:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619923.178291:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619923.178373:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f25880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+26:2355:x1875503571559296:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 5743us (7357us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619923.178384:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 405 00000100:00100000:2.0:1788619923.181203:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571559424 00000100:00100000:2.0:1788619923.181218:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571559424 00002000:00100000:2.0:1788619923.181555:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5f26a00 x1875503571559424/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:19/0 lens 488/0 e 0 to 0 dl 1788619934 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619923.181565:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5f26a00 with x1875503571559424 00000100:00100000:2.0:1788619923.181634:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 746 00000100:00100000:2.0:1788619923.181645:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f26a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+35:2355:x1875503571559424:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619923.195197:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f26a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+35:2355:x1875503571559424:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 13543us (17051us total) trans 0 rc 552960/552960 00000100:00100000:2.0:1788619923.195212:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 746 00002000:00100000:2.0:1788619923.195225:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5f26a00 x1875503571559424/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:19/0 lens 488/440 e 0 to 0 dl 1788619934 ref 1 fl Complete:/600/0 rc 552960/552960 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619923.195234:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5f26a00 with x1875503571559424 00000100:00100000:3.0:1788619923.201397:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619923.201576:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571559552 00000100:00100000:0.0:1788619923.201586:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571559552 00000100:00100000:0.0:1788619923.201648:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 963 00000100:00100000:0.0:1788619923.201658:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63fc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571559552:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619923.201691:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619923.201710:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b7120d83/0x1cd7fe53687357d2 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ca69 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619923.201787:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f27b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589171840:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619923.201882:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb540 going to sleep for 11 seconds 00000100:00100000:3.0:1788619923.204598:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f27b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589171840:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619923.204640:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b7120d83/0x1cd7fe53687357d2 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca69 expref: 9 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.204666:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b7120d83/0x1cd7fe53687357d2 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca69 expref: 9 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.204692:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f50400, rc 0) 00000100:00100000:3.0:1788619923.204751:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63fc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571559552:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3094us (3461us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619923.205649:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 963 00000100:00100000:3.0:1788619923.206978:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619923.207135:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571559680 00000100:00100000:2.0:1788619923.207143:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571559680 00000100:00100000:2.0:1788619923.207195:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 964 00000100:00100000:2.0:1788619923.207203:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571559680:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619923.207217:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619923.207231:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3b293bd/0x1cd7fe53687357d9 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ca70 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619923.207283:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cd8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589171968:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619923.207403:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164000 going to sleep for 11 seconds 00000100:00100000:3.0:1788619923.210273:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cd8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589171968:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619923.210313:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f3b293bd/0x1cd7fe53687357d9 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca70 expref: 9 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.210424:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3b293bd/0x1cd7fe53687357d9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca70 expref: 9 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.210437:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7e00, rc 0) 00000100:00100000:2.0:1788619923.210555:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571559680:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3349us (3581us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619923.210565:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 964 00000100:00100000:0.0:1788619923.215581:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619923.215662:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571559808 00000100:00100000:1.0:1788619923.215686:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571559808 00000100:00100000:1.0:1788619923.215760:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1206 00000100:00100000:1.0:1788619923.215769:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f26680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+26:2352:x1875503571559808:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619923.215921:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619923.215939:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296e00/0x1cd7fe53687357e0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ca77 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.215971:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296e00/0x1cd7fe53687357e0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca77 expref: 27 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788619923.215981:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 678955 00010000:00010000:1.0:1788619923.215986:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296e00/0x1cd7fe53687357e0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca77 expref: 27 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.215996:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619923.215999:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619923.216001:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296e00, rc 0) 00000100:00100000:1.0:1788619923.216061:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f26680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+27:2352:x1875503571559808:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 293us (491us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619923.216066:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1206 00000100:00100000:3.0:1788619923.226237:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619923.226644:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571559936 00000100:00100000:2.0:1788619923.226654:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571559936 00000100:00100000:2.0:1788619923.227234:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1207 00000100:00100000:2.0:1788619923.227246:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63fce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+27:14908:x1875503571559936:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619923.227267:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619923.227270:0:5844:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1788619923.227614:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296e00/0x1cd7fe53687357e0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca77 expref: 27 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.227631:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619923.227635:0:5844:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788619923.227646:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6200/0x1cd7fe53687357e7 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ca7e expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.227669:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687357cb lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca62 expref: 34 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.227692:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6200/0x1cd7fe53687357e7 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca7e expref: 27 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.227732:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687357cb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca62 expref: 34 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.227746:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687357cb lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca62 expref: 34 pid: 9417 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619923.227761:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cd8a80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589172096:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619923.229460:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164a80 going to sleep for 11 seconds 00000100:00100000:2.0:1788619923.233251:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cd8a80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589172096:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619923.233290:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6200/0x1cd7fe53687357e7 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca7e expref: 27 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.233317:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619923.233321:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619923.233362:0:5844:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6200/0x1cd7fe53687357e7 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca7e expref: 27 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.233375:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6200, rc 0) 00000100:00100000:2.0:1788619923.233439:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63fce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+27:14908:x1875503571559936:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6196us (7213us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619923.233449:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1207 00000100:00100000:0.0:1788619923.233761:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619923.234003:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571560064 00000100:00100000:3.0:1788619923.234011:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571560064 00010000:00010000:3.0:1788619923.234046:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687357cb lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca62 expref: 35 pid: 9417 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619923.234091:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 406 00000100:00100000:3.0:1788619923.234101:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f25c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+35:2354:x1875503571560064:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619923.234116:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619923.234144:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687357cb lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca62 expref: 35 pid: 9417 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.234157:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687357cb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca62 expref: 35 pid: 9417 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.234179:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687357cb lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca62 expref: 35 pid: 9417 timeout: 515 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.234194:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619923.234198:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6200/0x1cd7fe53687357e7 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca7e expref: 26 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.234212:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6200/0x1cd7fe53687357e7 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca7e expref: 26 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.234221:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6200/0x1cd7fe53687357e7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca7e expref: 26 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.234254:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6200/0x1cd7fe53687357e7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca7e expref: 26 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619923.234267:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63fea00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589172224:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619923.234392:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48cd80 going to sleep for 11 seconds 00010000:00010000:3.0:1788619923.234707:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788619923.237244:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63fea00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589172224:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619923.237271:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619923.237705:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f25c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+34:2354:x1875503571560064:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3602us (3942us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619923.237718:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 406 00000100:00100000:0.0:1788619923.240273:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619923.240354:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571560192 00000100:00100000:3.0:1788619923.240365:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571560192 00000100:00100000:3.0:1788619923.240436:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 747 00000100:00100000:3.0:1788619923.240446:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f27b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+27:2355:x1875503571560192:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:3.0:1788619923.240668:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 678955 -> 678955 00000004:00010000:3.0:1788619923.240672:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619922 -> 1788619922 00000004:00010000:3.0:1788619923.240702:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619922 -> 1788619922 00000004:00010000:3.0:1788619923.240707:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619922 -> 1788619922 00000004:00010000:3.0:1788619923.240710:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1030 -> 1030 00000100:00100000:3.0:1788619923.240997:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f27b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+28:2355:x1875503571560192:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 551us (732us total) trans 4294967736 rc 0/0 00000100:00100000:3.0:1788619923.241006:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 747 00000100:00100000:1.0:1788619923.244164:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619923.244252:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571560320 00000100:00100000:3.0:1788619923.244261:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571560320 00000100:00100000:3.0:1788619923.244356:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1208 00000100:00100000:3.0:1788619923.244367:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63f9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+34:2352:x1875503571560320:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619923.244388:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619923.244407:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687357ee lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ca85 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.244444:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6200/0x1cd7fe53687357e7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca7e expref: 27 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619923.244484:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63fea00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589172352:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619923.244568:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48cd80 going to sleep for 11 seconds 00000100:00100000:0.0:1788619923.249587:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619923.250095:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571560448 00000100:00100000:2.0:1788619923.250104:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571560448 00000100:00100000:2.0:1788619923.250153:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 965 00000100:00100000:2.0:1788619923.250160:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f27800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2352:x1875503571560448:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619923.250176:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619923.250190:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5ec8983/0x1cd7fe53687357f5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ca8c expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619923.250239:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cd8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589172480:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619923.250321:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164a80 going to sleep for 11 seconds 00000100:00100000:3.0:1788619923.251626:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63fea00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589172352:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619923.251645:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 678955 00010000:00010000:3.0:1788619923.251662:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687357ee lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca85 expref: 35 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.251688:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619923.251690:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619923.251695:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687357ee lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca85 expref: 35 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.251705:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683da00, rc 0) 00000100:00100000:3.0:1788619923.251743:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63f9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+34:2352:x1875503571560320:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7379us (7591us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619923.251750:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1208 00000100:00100000:2.0:1788619923.255804:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cd8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589172480:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619923.255973:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a5ec8983/0x1cd7fe53687357f5 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca8c expref: 9 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.255998:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5ec8983/0x1cd7fe53687357f5 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca8c expref: 9 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.256012:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6600, rc 0) 00000100:00100000:2.0:1788619923.256306:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f27800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2352:x1875503571560448:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6143us (6726us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619923.256314:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 965 00000100:00100000:0.0:1788619923.261784:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619923.261968:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619923.262202:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571560704 00000100:00100000:2.0:1788619923.262213:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571560704 00000100:00100000:2.0:1788619923.262281:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 966 00000100:00100000:2.0:1788619923.262291:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74a8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571560704:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619923.262309:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619923.262362:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000092f9790d/0x1cd7fe53687357fc lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ca9a expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619923.262420:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cd8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589172608:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619923.262509:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164a80 going to sleep for 11 seconds 00000100:00100000:1.0:1788619923.262728:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571560576 00000100:00100000:1.0:1788619923.262734:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571560576 00000100:00100000:1.0:1788619923.262801:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1209 00000100:00100000:1.0:1788619923.262817:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f25180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+34:2355:x1875503571560576:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619923.262828:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619923.262841:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296e00/0x1cd7fe5368735803 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ca93 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.262870:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6200/0x1cd7fe53687357e7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca7e expref: 27 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619923.262894:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63f8e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589172736:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619923.263118:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7f00 going to sleep for 11 seconds 00000100:00100000:3.0:1788619923.265318:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63f8e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589172736:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619923.265377:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 678955 00010000:00010000:3.0:1788619923.265397:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296e00/0x1cd7fe5368735803 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca93 expref: 35 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.265418:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619923.265421:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619923.265427:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296e00/0x1cd7fe5368735803 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca93 expref: 35 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.265442:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296e00, rc 0) 00000100:00100000:3.0:1788619923.265491:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f25180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+34:2355:x1875503571560576:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2682us (3715us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619923.265511:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1209 00000100:00100000:1.0:1788619923.266478:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cd8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589172608:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619923.266525:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000092f9790d/0x1cd7fe53687357fc lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca9a expref: 9 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.266546:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000092f9790d/0x1cd7fe53687357fc lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca9a expref: 9 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.266558:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7800, rc 0) 00000100:00100000:1.0:1788619923.266609:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74a8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571560704:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4318us (4639us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619923.266617:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 966 00000100:00100000:0.0:1788619923.276063:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619923.276134:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619923.276282:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571560960 00000100:00100000:1.0:1788619923.276291:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571560960 00000100:00100000:1.0:1788619923.276374:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 967 00000100:00100000:1.0:1788619923.276386:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8595180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571560960:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619923.276403:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619923.276422:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6252eaa/0x1cd7fe536873580a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1caa8 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619923.276480:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63f9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589172864:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619923.276567:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7b40 going to sleep for 11 seconds 00000100:00100000:2.0:1788619923.276734:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571560832 00000100:00100000:2.0:1788619923.276740:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571560832 00000100:00100000:2.0:1788619923.276778:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1210 00000100:00100000:2.0:1788619923.276786:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8595f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+34:2355:x1875503571560832:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619923.276796:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619923.276818:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6600/0x1cd7fe5368735811 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1caa1 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.276847:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6200/0x1cd7fe53687357e7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca7e expref: 27 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619923.276873:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cd8a80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589172992:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619923.276900:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81640c0 going to sleep for 11 seconds 00000004:00010000:2.0:1788619923.280306:0:5844:0:(mdt_lvb.c:183:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime: 1788619922 -> 1788619923 00000004:00010000:2.0:1788619923.280317:0:5844:0:(mdt_lvb.c:195:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime: 1788619922 -> 1788619923 00000100:00100000:2.0:1788619923.280384:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cd8a80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589172992:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619923.280410:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 678955 00010000:00010000:2.0:1788619923.280432:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6600/0x1cd7fe5368735811 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1caa1 expref: 35 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.280454:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619923.280457:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619923.280464:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6600/0x1cd7fe5368735811 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1caa1 expref: 35 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.280477:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6600, rc 0) 00000100:00100000:2.0:1788619923.280539:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8595f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+34:2355:x1875503571560832:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3754us (4487us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619923.280548:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1210 00002000:00010000:1.0:1788619923.283394:0:6698:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size: 0 -> 1859521 00002000:00010000:1.0:1788619923.283403:0:6698:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime: 1788619922 -> 1788619923 00002000:00010000:1.0:1788619923.283406:0:6698:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime: 1788619922 -> 1788619923 00000100:00100000:1.0:1788619923.283430:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63f9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589172864:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619923.283464:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a6252eaa/0x1cd7fe536873580a lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1caa8 expref: 9 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.283488:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6252eaa/0x1cd7fe536873580a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1caa8 expref: 9 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.283509:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297600, rc 0) 00000100:00100000:1.0:1788619923.283569:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8595180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571560960:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7184us (7434us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619923.283579:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 967 00000100:00100000:0.0:1788619923.289990:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619923.290034:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619923.290116:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571561088 00000100:00100000:1.0:1788619923.290126:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571561088 00000100:00100000:1.0:1788619923.290186:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1211 00000100:00100000:1.0:1788619923.290196:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df967800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+34:2354:x1875503571561088:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619923.290220:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619923.290241:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe5368735818 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1caaf expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.290278:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6200/0x1cd7fe53687357e7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca7e expref: 27 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619923.290320:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63f9500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589173120:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619923.290429:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7b40 going to sleep for 11 seconds 00000100:00100000:3.0:1788619923.291002:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571561216 00000100:00100000:3.0:1788619923.291006:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571561216 00000100:00100000:3.0:1788619923.291048:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 968 00000100:00100000:3.0:1788619923.291055:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df965f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571561216:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619923.291066:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619923.291078:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001391204f/0x1cd7fe536873581f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cab6 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619923.291184:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63fea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589173248:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619923.291233:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48cd80 going to sleep for 11 seconds 00000100:00100000:1.0:1788619923.295608:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63f9500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589173120:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619923.295636:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 678955 00010000:00010000:1.0:1788619923.295654:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe5368735818 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1caaf expref: 35 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.295684:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619923.295688:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619923.295694:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe5368735818 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1caaf expref: 35 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.295709:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297000, rc 0) 00000100:00100000:1.0:1788619923.295770:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df967800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+34:2354:x1875503571561088:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5574us (5735us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619923.295779:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1211 00000100:00100000:3.0:1788619923.296278:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63fea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589173248:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619923.296300:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001391204f/0x1cd7fe536873581f lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cab6 expref: 9 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.296318:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001391204f/0x1cd7fe536873581f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cab6 expref: 9 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.296367:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683da00, rc 0) 00000100:00100000:3.0:1788619923.296396:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df965f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571561216:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5342us (6417us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619923.296402:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 968 00000100:00100000:0.0:1788619923.309311:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619923.309427:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571561344 00000100:00100000:2.0:1788619923.309439:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571561344 00002000:00100000:2.0:1788619923.309486:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78cdcc3800 x1875503571561344/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:19/0 lens 488/0 e 0 to 0 dl 1788619934 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619923.309497:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78cdcc3800 with x1875503571561344 00000100:00100000:2.0:1788619923.309523:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 748 00000100:00100000:2.0:1788619923.309532:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+28:2352:x1875503571561344:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619923.317283:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619923.317430:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571561472 00000100:00100000:1.0:1788619923.317442:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571561472 00002000:00100000:1.0:1788619923.317498:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1826815) req@ffff8e77c5f56d80 x1875503571561472/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:19/0 lens 488/0 e 0 to 0 dl 1788619934 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619923.317514:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5f56d80 with x1875503571561472 ext(1048576->1826815) 00002000:00010000:1.0:1788619923.317520:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5f56d80 00000100:00100000:1.0:1788619923.317560:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 562 00000100:00100000:1.0:1788619923.317570:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f56d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571561472:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619923.323278:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f56d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571561472:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 5698us (5995us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619923.323294:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 562 00002000:00100000:1.0:1788619923.323307:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1826815) req@ffff8e77c5f56d80 x1875503571561472/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:19/0 lens 488/440 e 0 to 0 dl 1788619934 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619923.323318:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5f56d80 with x1875503571561472 ext(1048576->1826815) 00002000:00010000:1.0:1788619923.323344:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5f56d80 00000100:00100000:2.0:1788619923.323638:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+28:2352:x1875503571561344:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 14106us (14338us total) trans 0 rc 679936/679936 00000100:00100000:2.0:1788619923.323647:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 748 00002000:00100000:2.0:1788619923.323654:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78cdcc3800 x1875503571561344/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:19/0 lens 488/440 e 0 to 0 dl 1788619934 ref 1 fl Complete:/600/0 rc 679936/679936 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619923.323663:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78cdcc3800 with x1875503571561344 00000100:00100000:3.0:1788619923.334124:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619923.334304:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571561600 00000100:00100000:2.0:1788619923.334314:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571561600 00002000:00100000:2.0:1788619923.334398:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c63fea00 x1875503571561600/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:19/0 lens 488/0 e 0 to 0 dl 1788619934 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619923.334409:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c63fea00 with x1875503571561600 00000100:00100000:2.0:1788619923.334448:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 749 00000100:00100000:2.0:1788619923.334458:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63fea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+28:2353:x1875503571561600:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:2.0:1788619923.344201:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 678955 -> 678955 00000004:00010000:2.0:1788619923.344211:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619923 -> 1788619923 00000004:00010000:2.0:1788619923.344214:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619922 -> 1788619922 00000004:00010000:2.0:1788619923.344215:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619923 -> 1788619923 00000004:00010000:2.0:1788619923.344217:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1030 -> 1030 00000100:00100000:2.0:1788619923.344305:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63fea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+30:2353:x1875503571561600:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 9843us (10184us total) trans 4294967737 rc 0/0 00000100:00100000:2.0:1788619923.344318:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 749 00002000:00100000:2.0:1788619923.344378:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c63fea00 x1875503571561600/t4294967737(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:19/0 lens 488/448 e 0 to 0 dl 1788619934 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619923.344388:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c63fea00 with x1875503571561600 00000100:00100000:0.0:1788619923.348935:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619923.349030:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571561728 00000100:00100000:3.0:1788619923.349040:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571561728 00000100:00100000:3.0:1788619923.349108:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1212 00000100:00100000:3.0:1788619923.349119:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f56300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+30:2354:x1875503571561728:12345-192.168.201.5@tcp:16:fsx.0 00000001:00080000:0.0:1788619923.375369:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967733 is committed 00000001:00080000:0.0:1788619923.375392:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967729 is committed 00000001:00080000:0.0:1788619923.375408:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967735 is committed 00000001:00080000:0.0:1788619923.375429:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967737 is committed 00000001:00080000:0.0:1788619923.375435:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967731 is committed 00000100:00100000:1.0:1788619923.375703:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f56300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571561728:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 26579us (26772us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619923.375716:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1212 00000100:00100000:1.0:1788619923.382487:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619923.382639:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571561856 00000100:00100000:2.0:1788619923.382649:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571561856 00000100:00100000:2.0:1788619923.382719:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1213 00000100:00100000:2.0:1788619923.382729:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63f9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571561856:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619923.382749:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619923.382770:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6600/0x1cd7fe5368735826 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cabd expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.382911:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6200/0x1cd7fe53687357e7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca7e expref: 9 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619923.383024:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cd8a80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589173376:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619923.383236:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81640c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619923.387632:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000004:00010000:2.0:1788619923.387835:0:5844:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619922 -> 1788619923 00000100:00100000:2.0:1788619923.387861:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cd8a80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589173376:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619923.387885:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 678955 00010000:00010000:2.0:1788619923.387903:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6600/0x1cd7fe5368735826 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cabd expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.387922:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619923.387925:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619923.387930:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6600/0x1cd7fe5368735826 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cabd expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.388066:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6600, rc 0) 00000100:00100000:2.0:1788619923.388197:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63f9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571561856:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5468us (5717us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619923.388207:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1213 00000100:00100000:0.0:1788619923.390240:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571561984 00000100:00100000:0.0:1788619923.390254:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571561984 00000100:00100000:1.0:1788619923.390405:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 969 00000100:00100000:1.0:1788619923.390417:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571561984:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619923.390435:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619923.390452:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6252eaa/0x1cd7fe536873582d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cac4 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619923.390620:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63f9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589173504:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619923.390719:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7d80 going to sleep for 11 seconds 00002000:00010000:2.0:1788619923.393864:0:6697:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619922 -> 1788619923 00000100:00100000:2.0:1788619923.393896:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63f9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589173504:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619923.393949:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a6252eaa/0x1cd7fe536873582d lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cac4 expref: 9 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.393972:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6252eaa/0x1cd7fe536873582d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cac4 expref: 9 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.393989:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297600, rc 0) 00000100:00100000:2.0:1788619923.394048:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571561984:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3633us (6424us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619923.394056:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 969 00000100:00100000:0.0:1788619923.397560:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619923.397710:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571562112 00000100:00100000:1.0:1788619923.397719:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571562112 00000100:00100000:1.0:1788619923.397777:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1214 00000100:00100000:1.0:1788619923.397786:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f55f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571562112:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619923.397816:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619923.397836:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296a00/0x1cd7fe5368735834 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cacb expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.397874:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6200/0x1cd7fe53687357e7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca7e expref: 9 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619923.397923:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63fb800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589173632:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619923.398013:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7f00 going to sleep for 11 seconds 00000100:00100000:1.0:1788619923.402201:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619923.402289:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571562240 00000100:00100000:2.0:1788619923.402299:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571562240 00000100:00100000:2.0:1788619923.402388:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 970 00000100:00100000:2.0:1788619923.402398:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63f8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571562240:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619923.402415:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619923.402432:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5ec8983/0x1cd7fe536873583b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cad2 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619923.402489:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cd8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589173760:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619923.403443:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81640c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619923.405907:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cd8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589173760:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619923.405959:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a5ec8983/0x1cd7fe536873583b lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cad2 expref: 9 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.405984:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5ec8983/0x1cd7fe536873583b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cad2 expref: 9 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.405996:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6600, rc 0) 00000100:00100000:2.0:1788619923.406059:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63f8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571562240:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3662us (3865us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619923.406067:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 970 00000100:00100000:1.0:1788619923.406785:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63fb800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589173632:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619923.406801:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 678955 00010000:00010000:1.0:1788619923.406825:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296a00/0x1cd7fe5368735834 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cacb expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.406839:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619923.406842:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619923.406846:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296a00/0x1cd7fe5368735834 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cacb expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.406859:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296a00, rc 0) 00000100:00100000:1.0:1788619923.406945:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f55f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571562112:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 9152us (9386us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619923.406956:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1214 00000100:00100000:0.0:1788619923.414458:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619923.414552:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571562496 00000100:00100000:3.0:1788619923.414562:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571562496 00000100:00100000:3.0:1788619923.414625:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1215 00000100:00100000:3.0:1788619923.414635:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f57100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:14908:x1875503571562496:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:3.0:1788619923.414700:0:5846:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619923.414728:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297800/0x1cd7fe536873579a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca31 expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.414747:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619923.414752:0:5846:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788619923.415025:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe5368735842 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.415041:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe536873578c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca2a expref: 9 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.415049:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe5368735842 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.415096:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe536873578c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca2a expref: 9 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.415110:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe536873578c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca2a expref: 9 pid: 11397 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619923.415123:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63ff100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589173888:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619923.415210:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48cd80 going to sleep for 11 seconds 00000100:00100000:3.0:1788619923.418397:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63ff100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589173888:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619923.418429:0:5846:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe5368735842 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619923.420159:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619923.420300:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571562624 00000100:00100000:3.0:1788619923.420309:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571562624 00010000:00010000:3.0:1788619923.420373:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe536873578c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca2a expref: 10 pid: 11397 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619923.420416:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 407 00000100:00100000:3.0:1788619923.420426:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f54700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571562624:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619923.420441:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619923.420448:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe536873578c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca2a expref: 10 pid: 11397 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.420460:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe536873578c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca2a expref: 10 pid: 11397 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.420481:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe536873578c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca2a expref: 10 pid: 11397 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.420505:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619923.420510:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe5368735842 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.420520:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe5368735842 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.420528:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe5368735842 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.420554:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619923.420616:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f54700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571562624:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 191us (464us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619923.420624:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 407 00010000:00010000:0.0:1788619923.421237:0:5846:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe5368735842 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619923.421246:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe5368735842 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788619923.421395:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967738 00000004:00010000:0.0:1788619923.421445:0:5846:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 678955 -> 678955 00000004:00010000:0.0:1788619923.421448:0:5846:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619923 -> 1788619923 00000004:00010000:0.0:1788619923.421451:0:5846:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619923 -> 1788619922 00000004:00010000:0.0:1788619923.421453:0:5846:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619923 -> 1788619923 00000004:00010000:0.0:1788619923.421455:0:5846:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1030 -> 390 00000004:00010000:0.0:1788619923.421462:0:5846:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000019aadad1 reply state 00000000e5b2ce0b transno 4294967739 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe5368735842 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619923.421470:0:5846:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788619923.421562:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f57100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571562496:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 6928us (7113us total) trans 4294967739 rc 0/0 00000100:00100000:0.0:1788619923.421569:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1215 00000020:00080000:0.0:1788619923.421573:0:5846:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:2.0:1788619923.425593:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe5368735842 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.425609:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe5368735842 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.425616:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe5368735842 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.425621:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c683da00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:2.0:1788619923.425626:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe5368735842 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.425634:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe5368735842 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.425651:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619923.425654:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788619923.425658:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe5368735842 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.425664:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe5368735842 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619923.427160:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619923.427288:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571562752 00000100:00100000:3.0:1788619923.427298:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571562752 00000100:00100000:3.0:1788619923.427391:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1216 00000100:00100000:3.0:1788619923.427402:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571562752:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619923.427425:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619923.427447:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368735849 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cad9 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.427477:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6200/0x1cd7fe53687357e7 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca7e expref: 8 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.427488:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368735849 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cad9 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.427533:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6200/0x1cd7fe53687357e7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca7e expref: 8 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.427547:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6200/0x1cd7fe53687357e7 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca7e expref: 8 pid: 5844 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619923.427560:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63ff100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589174016:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619923.427640:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48cd80 going to sleep for 11 seconds 00000100:00100000:3.0:1788619923.430953:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63ff100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589174016:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619923.430993:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368735849 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cad9 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.431012:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619923.431015:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619923.431019:0:5844:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368735849 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cad9 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.431029:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d200, rc 0) 00000100:00100000:3.0:1788619923.431090:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571562752:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3689us (3938us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619923.431099:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1216 00000100:00100000:0.0:1788619923.435253:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619923.435361:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571562880 00000100:00100000:2.0:1788619923.435371:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571562880 00010000:00010000:2.0:1788619923.435412:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6200/0x1cd7fe53687357e7 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca7e expref: 9 pid: 5844 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619923.435455:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 408 00000100:00100000:2.0:1788619923.435463:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f54000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571562880:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619923.435475:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619923.435503:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6200/0x1cd7fe53687357e7 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca7e expref: 9 pid: 5844 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.435513:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6200/0x1cd7fe53687357e7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca7e expref: 9 pid: 5844 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.435530:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6200/0x1cd7fe53687357e7 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca7e expref: 9 pid: 5844 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.435542:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619923.435547:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368735849 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cad9 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.435554:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368735849 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cad9 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.435560:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368735849 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cad9 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.435588:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368735849 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cad9 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619923.435601:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5d5ca80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589174144:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619923.435668:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81640c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619923.438065:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5d5ca80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589174144:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619923.438086:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619923.438126:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f54000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+8:2352:x1875503571562880:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 2665us (2882us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619923.438133:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 408 00000100:00100000:0.0:1788619923.439388:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619923.439456:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571563008 00000100:00100000:3.0:1788619923.439464:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571563008 00000100:00100000:3.0:1788619923.439516:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 971 00000100:00100000:3.0:1788619923.439525:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f54380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:14908:x1875503571563008:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619923.439542:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619923.439558:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe5368735850 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cae0 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.439588:0:6698:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe536873577e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca23 expref: 8 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.439597:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=1s v=(13 7) 00010000:00010000:3.0:1788619923.439601:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(1 13) 00010000:00010000:3.0:1788619923.439607:0:6698:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe5368735850 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cae0 expref: 9 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.439638:0:6698:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe536873577e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca23 expref: 8 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.439654:0:6698:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe536873577e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca23 expref: 8 pid: 6698 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619923.439668:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63ff100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589174272:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619923.439748:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48cd80 going to sleep for 11 seconds 00000100:00100000:3.0:1788619923.445181:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619923.445378:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63ff100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589174272:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619923.445412:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe5368735850 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cae0 expref: 9 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.445433:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c200, rc 0) 00000100:00100000:1.0:1788619923.445505:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f54380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+9:14908:x1875503571563008:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5973us (6113us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619923.445514:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 971 00000100:00100000:2.0:1788619923.446226:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571563136 00000100:00100000:2.0:1788619923.446236:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571563136 00002000:00100000:2.0:1788619923.446277:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1826816->1859520) req@ffff8e77c6a5ad80 x1875503571563136/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:19/0 lens 488/0 e 0 to 0 dl 1788619934 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619923.446288:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a5ad80 with x1875503571563136 ext(1826816->1859520) 00010000:00010000:2.0:1788619923.446295:0:6701:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe536873577e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca23 expref: 9 pid: 6698 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.446310:0:6701:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe536873577e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca23 expref: 9 pid: 6698 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788619923.446318:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff8e77c6a5ad80 00000100:00100000:2.0:1788619923.446377:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 17 00000100:00100000:2.0:1788619923.446388:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571563136:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:2.0:1788619923.452439:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571563136:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 6044us (7259us total) trans 4294967462 rc 0/0 00000100:00100000:2.0:1788619923.452450:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 17 00002000:00100000:2.0:1788619923.452462:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1826816->1859520) req@ffff8e77c6a5ad80 x1875503571563136/t4294967462(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:19/0 lens 488/448 e 0 to 0 dl 1788619934 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619923.452469:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a5ad80 with x1875503571563136 ext(1826816->1859520) 00010000:00010000:2.0:1788619923.452479:0:6701:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe536873577e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca23 expref: 11 pid: 6698 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.452494:0:6701:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe536873577e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca23 expref: 11 pid: 6698 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788619923.452499:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff8e77c6a5ad80 00000100:00100000:0.0:1788619923.455768:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619923.455916:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571563264 00000100:00100000:3.0:1788619923.455926:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571563264 00010000:00010000:3.0:1788619923.455980:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe536873577e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca23 expref: 11 pid: 6698 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619923.456024:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 409 00000100:00100000:3.0:1788619923.456033:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f55880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571563264:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619923.456047:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619923.456067:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe536873577e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca23 expref: 11 pid: 6698 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.456080:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe536873577e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca23 expref: 11 pid: 6698 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.456104:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe536873577e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ca23 expref: 11 pid: 6698 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.456119:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(1 13) 00010000:00010000:3.0:1788619923.456123:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(1 13) 00010000:00010000:3.0:1788619923.456131:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe5368735850 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cae0 expref: 8 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.456141:0:13330:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe5368735850 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cae0 expref: 8 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.456181:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe5368735850 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cae0 expref: 8 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619923.456195:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a5a300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589174400:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619923.456276:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c240 going to sleep for 11 seconds 00000100:00100000:2.0:1788619923.458751:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a5a300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589174400:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619923.458775:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619923.458900:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f55880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571563264:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 2867us (3140us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619923.458921:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 409 00000100:00100000:0.0:1788619923.462104:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619923.462179:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571563392 00000100:00100000:1.0:1788619923.462188:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571563392 00002000:00010000:1.0:1788619923.462221:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1697912->18446744073709551615) 00002000:00100000:1.0:1788619923.462225:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5f56a00 with x1875503571563392 ext(1697912->18446744073709551615) 00002000:00010000:1.0:1788619923.462235:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5f56a00. 00000100:00100000:1.0:1788619923.462260:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 563 00000100:00100000:1.0:1788619923.462267:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f56a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2353:x1875503571563392:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:1.0:1788619923.462563:0:6702:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1859521 -> 1697912 00002000:00010000:1.0:1788619923.462566:0:6702:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619923 -> 1788619923 00002000:00010000:1.0:1788619923.462568:0:6702:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619923 -> 1788619922 00002000:00010000:1.0:1788619923.462570:0:6702:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619923 -> 1788619923 00002000:00010000:1.0:1788619923.462572:0:6702:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 263 -> 263 00000100:00100000:1.0:1788619923.462620:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f56a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+10:2353:x1875503571563392:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 354us (525us total) trans 4294967463 rc 0/0 00000100:00100000:1.0:1788619923.462625:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 563 00002000:00010000:1.0:1788619923.462628:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1697912->18446744073709551615) 00002000:00100000:1.0:1788619923.462630:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5f56a00 with x1875503571563392 ext(1697912->18446744073709551615) 00002000:00010000:1.0:1788619923.462633:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5f56a00. 00000100:00100000:0.0:1788619923.464860:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619923.464926:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571563520 00000100:00100000:1.0:1788619923.464932:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571563520 00000100:00100000:1.0:1788619923.465067:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1217 00000100:00100000:1.0:1788619923.465076:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f57b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571563520:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619923.465095:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619923.465113:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297200/0x1cd7fe5368735857 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cae7 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.465186:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297400/0x1cd7fe536873585e lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.465197:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297400/0x1cd7fe536873585e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.465205:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297400/0x1cd7fe536873585e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788619923.465224:0:9417:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1697912 00000004:00200000:1.0:1788619923.465227:0:9417:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788619923.465240:0:9417:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297400/0x1cd7fe536873585e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.465258:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297200/0x1cd7fe5368735857 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cae7 expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.465271:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297400/0x1cd7fe536873585e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cae7 expref: 15 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.465282:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619923.465284:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788619923.465287:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297400, rc 0) 00000100:00100000:1.0:1788619923.465381:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f57b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:14908:x1875503571563520:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 306us (526us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619923.465387:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1217 00000100:00100000:3.0:1788619923.472741:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619923.472852:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619923.473114:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571563648 00000100:00100000:1.0:1788619923.473125:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571563648 00002000:00100000:1.0:1788619923.473175:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1699839) req@ffff8e77c6a5b100 x1875503571563648/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:19/0 lens 488/0 e 0 to 0 dl 1788619934 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619923.473187:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a5b100 with x1875503571563648 ext(1048576->1699839) 00002000:00010000:1.0:1788619923.473193:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a5b100 00000100:00100000:1.0:1788619923.473229:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 564 00000100:00100000:1.0:1788619923.473239:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+10:2355:x1875503571563648:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619923.477923:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571563776 00000100:00100000:2.0:1788619923.478000:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571563776 00002000:00100000:2.0:1788619923.478063:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6a59f80 x1875503571563776/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:19/0 lens 488/0 e 0 to 0 dl 1788619934 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619923.478076:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6a59f80 with x1875503571563776 00000100:00100000:2.0:1788619923.478118:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 750 00000100:00100000:2.0:1788619923.478129:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a59f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571563776:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619923.480602:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a59f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571563776:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 2464us (7861us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619923.480616:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 750 00002000:00100000:2.0:1788619923.480628:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6a59f80 x1875503571563776/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:19/0 lens 488/440 e 0 to 0 dl 1788619934 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619923.480639:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6a59f80 with x1875503571563776 00000100:00100000:2.0:1788619923.481545:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+10:2355:x1875503571563648:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 8303us (8689us total) trans 0 rc 651264/651264 00000100:00100000:2.0:1788619923.481555:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 564 00002000:00100000:2.0:1788619923.481564:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1699839) req@ffff8e77c6a5b100 x1875503571563648/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:19/0 lens 488/440 e 0 to 0 dl 1788619934 ref 1 fl Complete:/600/0 rc 651264/651264 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619923.481575:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a5b100 with x1875503571563648 ext(1048576->1699839) 00002000:00010000:2.0:1788619923.481579:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a5b100 00000100:00100000:1.0:1788619923.486765:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619923.487164:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571563904 00000100:00100000:0.0:1788619923.487175:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571563904 00002000:00100000:0.0:1788619923.487229:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1204224->1232895) req@ffff8e77c63fb800 x1875503571563904/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:19/0 lens 488/0 e 0 to 0 dl 1788619934 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619923.487241:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c63fb800 with x1875503571563904 ext(1204224->1232895) 00002000:00010000:0.0:1788619923.487247:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c63fb800 00000100:00100000:0.0:1788619923.487321:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 565 00000100:00100000:0.0:1788619923.487374:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63fb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+10:2355:x1875503571563904:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:0.0:1788619923.490238:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63fb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2355:x1875503571563904:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 2856us (3470us total) trans 4294967464 rc 0/0 00000100:00100000:0.0:1788619923.490252:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 565 00002000:00100000:0.0:1788619923.490263:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1204224->1232895) req@ffff8e77c63fb800 x1875503571563904/t4294967464(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:19/0 lens 488/448 e 0 to 0 dl 1788619934 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619923.490275:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c63fb800 with x1875503571563904 ext(1204224->1232895) 00002000:00010000:0.0:1788619923.490281:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c63fb800 00000100:00100000:3.0:1788619923.493694:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619923.493793:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571564032 00000100:00100000:1.0:1788619923.493813:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571564032 00000100:00100000:1.0:1788619923.493872:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 972 00000100:00100000:1.0:1788619923.493883:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a58700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2355:x1875503571564032:12345-192.168.201.5@tcp:16:fsx.0 00000001:00080000:3.0:1788619923.528163:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967464 is committed 00000001:00080000:3.0:1788619923.528193:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967459 is committed 00000001:00080000:3.0:1788619923.528200:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967462 is committed 00000100:00100000:1.0:1788619923.528368:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a58700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571564032:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 34481us (34679us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619923.528380:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 972 00000100:00100000:3.0:1788619923.531237:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619923.531372:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571564160 00000100:00100000:1.0:1788619923.531382:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571564160 00000100:00100000:1.0:1788619923.531438:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1218 00000100:00100000:1.0:1788619923.531448:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a58000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+8:14908:x1875503571564160:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619923.531468:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619923.531489:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297200/0x1cd7fe5368735865 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1caee expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.531703:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe536873586c lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.531720:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe536873586c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.531733:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe536873586c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788619923.531760:0:5846:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1697912 00000004:00200000:1.0:1788619923.531764:0:5846:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788619923.531782:0:5846:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe536873586c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.531882:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297200/0x1cd7fe5368735865 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1caee expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.531901:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe536873586c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1caee expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.531922:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619923.531925:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788619923.531929:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297a00, rc 0) 00000100:00100000:1.0:1788619923.532026:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a58000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571564160:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 579us (796us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619923.532033:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1218 00000100:00100000:2.0:1788619923.538121:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619923.538199:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619923.538261:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571564288 00000100:00100000:3.0:1788619923.538270:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571564288 00000100:00100000:3.0:1788619923.538318:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1219 00000100:00100000:3.0:1788619923.538345:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6532d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571564288:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619923.538365:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619923.538385:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368735873 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1caf5 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.538421:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368735849 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cad9 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619923.538457:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a58380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589174528:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619923.538650:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619923.538749:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571564416 00000100:00100000:1.0:1788619923.538754:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571564416 00000100:00100000:1.0:1788619923.538818:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 973 00000100:00100000:1.0:1788619923.538826:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d5ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571564416:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619923.538834:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619923.538844:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d990b60e/0x1cd7fe536873587a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cafc expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619923.538878:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63fad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589174656:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619923.538902:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7f00 going to sleep for 11 seconds 00002000:00010000:1.0:1788619923.541466:0:6697:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619922 -> 1788619923 00000100:00100000:1.0:1788619923.541512:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63fad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589174656:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619923.541548:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d990b60e/0x1cd7fe536873587a lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cafc expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.541570:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d990b60e/0x1cd7fe536873587a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cafc expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.541582:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297200, rc 0) 00000100:00100000:1.0:1788619923.541636:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d5ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571564416:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2810us (3523us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619923.541644:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 973 00000004:00010000:3.0:1788619923.542064:0:5844:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619922 -> 1788619923 00000100:00100000:3.0:1788619923.542083:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a58380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589174528:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619923.542097:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 678955 00010000:00010000:3.0:1788619923.542109:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368735873 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1caf5 expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.542123:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619923.542126:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619923.542130:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368735873 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1caf5 expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.542141:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d000, rc 0) 00000100:00100000:3.0:1788619923.542173:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6532d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571564288:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3847us (3974us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619923.542179:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1219 00000100:00100000:0.0:1788619923.546157:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619923.546203:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619923.546290:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571564672 00000100:00100000:2.0:1788619923.546300:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571564672 00000100:00100000:2.0:1788619923.546388:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 974 00000100:00100000:2.0:1788619923.546398:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f55c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571564672:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619923.546417:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619923.546436:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001dc9de0/0x1cd7fe5368735881 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cb0a expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619923.546494:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6533480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589174784:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619923.546573:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164480 going to sleep for 11 seconds 00000100:00100000:1.0:1788619923.546717:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571564544 00000100:00100000:1.0:1788619923.546722:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571564544 00000100:00100000:1.0:1788619923.546761:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1220 00000100:00100000:1.0:1788619923.546767:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f56680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571564544:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619923.546777:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619923.546790:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297c00/0x1cd7fe5368735888 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cb03 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.546816:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368735849 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cad9 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619923.546838:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63fad80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589174912:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619923.546865:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7f00 going to sleep for 11 seconds 00000100:00100000:1.0:1788619923.549841:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63fad80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589174912:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619923.549865:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 678955 00010000:00010000:1.0:1788619923.549884:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297c00/0x1cd7fe5368735888 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb03 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.549904:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619923.549907:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619923.549912:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297c00/0x1cd7fe5368735888 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb03 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.549926:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297c00, rc 0) 00000100:00100000:1.0:1788619923.549993:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f56680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571564544:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3224us (3845us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619923.550002:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1220 00000100:00100000:2.0:1788619923.550659:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6533480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589174784:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619923.550694:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000001dc9de0/0x1cd7fe5368735881 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb0a expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.550713:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001dc9de0/0x1cd7fe5368735881 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb0a expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.550729:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6a00, rc 0) 00000100:00100000:2.0:1788619923.550770:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f55c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571564672:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4374us (4567us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619923.550777:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 974 00000100:00100000:0.0:1788619923.555252:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619923.555358:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571564800 00000100:00100000:2.0:1788619923.555368:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571564800 00002000:00100000:2.0:1788619923.555419:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5f54e00 x1875503571564800/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:19/0 lens 488/0 e 0 to 0 dl 1788619934 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619923.555429:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5f54e00 with x1875503571564800 00000100:00100000:2.0:1788619923.555458:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 751 00000100:00100000:2.0:1788619923.555466:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f54e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571564800:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619923.555721:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f54e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571564800:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 254us (477us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619923.555729:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 751 00002000:00100000:2.0:1788619923.555736:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5f54e00 x1875503571564800/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:19/0 lens 488/4536 e 0 to 0 dl 1788619934 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619923.555743:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5f54e00 with x1875503571564800 00000100:00100000:3.0:1788619923.558968:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619923.559037:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571564928 00000100:00100000:2.0:1788619923.559047:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571564928 00002000:00100000:2.0:1788619923.559092:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6a58380 x1875503571564928/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:19/0 lens 488/0 e 0 to 0 dl 1788619934 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619923.559103:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6a58380 with x1875503571564928 00000100:00100000:2.0:1788619923.559139:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 752 00000100:00100000:2.0:1788619923.559147:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a58380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2352:x1875503571564928:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619923.559414:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a58380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2352:x1875503571564928:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 266us (455us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619923.559421:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 752 00002000:00100000:2.0:1788619923.559428:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6a58380 x1875503571564928/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:19/0 lens 488/4536 e 0 to 0 dl 1788619934 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619923.559436:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6a58380 with x1875503571564928 00000100:00100000:0.0:1788619923.562515:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619923.562583:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571565056 00000100:00100000:1.0:1788619923.562591:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571565056 00000100:00100000:1.0:1788619923.562633:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1221 00000100:00100000:1.0:1788619923.562640:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f55500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571565056:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619923.562657:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619923.562686:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297c00/0x1cd7fe536873588f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cb11 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.562729:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368735849 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cad9 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619923.562769:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63fad80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589175040:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619923.562900:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7f00 going to sleep for 11 seconds 00000100:00100000:0.0:1788619923.564149:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619923.564724:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571565184 00000100:00100000:1.0:1788619923.564732:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571565184 00000100:00100000:1.0:1788619923.564785:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 975 00000100:00100000:1.0:1788619923.564795:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f57480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571565184:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619923.564810:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619923.564825:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d990b60e/0x1cd7fe5368735896 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cb18 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619923.564873:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63fa300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589175168:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619923.565039:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7780 going to sleep for 11 seconds 00000100:00100000:3.0:1788619923.566881:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63fa300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589175168:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619923.566916:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d990b60e/0x1cd7fe5368735896 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb18 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.566947:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d990b60e/0x1cd7fe5368735896 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb18 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.566960:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297200, rc 0) 00000100:00100000:3.0:1788619923.567049:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f57480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571565184:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2256us (2904us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619923.567058:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 975 00000100:00100000:1.0:1788619923.567390:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63fad80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589175040:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619923.567400:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 678955 00010000:00010000:1.0:1788619923.567410:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297c00/0x1cd7fe536873588f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb11 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.567424:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619923.567426:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619923.567429:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297c00/0x1cd7fe536873588f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb11 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.567436:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297c00, rc 0) 00000100:00100000:1.0:1788619923.567454:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f55500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571565056:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4815us (4948us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619923.567458:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1221 00000100:00100000:0.0:1788619923.569606:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619923.569637:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619923.569647:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571565312 00000100:00100000:3.0:1788619923.569656:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571565312 00000100:00100000:3.0:1788619923.569723:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1222 00000100:00100000:3.0:1788619923.569734:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f55180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571565312:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619923.569752:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619923.569772:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe536873589d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cb1f expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619923.569791:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571565440 00000100:00100000:1.0:1788619923.569796:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571565440 00010000:00010000:3.0:1788619923.569808:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368735849 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cad9 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619923.569846:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a59880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589175296:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619923.569979:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619923.570123:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 976 00000100:00100000:1.0:1788619923.570130:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f57800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571565440:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619923.570136:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619923.570145:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002abacdb6/0x1cd7fe53687358a4 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cb26 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619923.570170:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63fad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589175424:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619923.570188:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7f00 going to sleep for 11 seconds 00000100:00100000:1.0:1788619923.571621:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63fad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589175424:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619923.571657:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002abacdb6/0x1cd7fe53687358a4 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb26 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.571690:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002abacdb6/0x1cd7fe53687358a4 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb26 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.571703:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296200, rc 0) 00000100:00100000:1.0:1788619923.571748:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f57800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571565440:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 1619us (2110us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619923.571755:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 976 00000100:00100000:3.0:1788619923.574190:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a59880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589175296:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619923.574210:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 678955 00010000:00010000:3.0:1788619923.574225:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe536873589d lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb1f expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.574240:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619923.574242:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619923.574246:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe536873589d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb1f expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.574256:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683da00, rc 0) 00000100:00100000:3.0:1788619923.574310:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f55180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571565312:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4577us (4709us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619923.574319:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1222 00000100:00100000:0.0:1788619923.576539:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619923.576616:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571565568 00000100:00100000:1.0:1788619923.576625:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571565568 00000100:00100000:1.0:1788619923.576688:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1223 00000100:00100000:1.0:1788619923.576699:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f54a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571565568:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619923.576718:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619923.576746:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297c00/0x1cd7fe53687358ab lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cb2d expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.576779:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368735849 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cad9 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619923.576866:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63fad80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589175552:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619923.576935:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7f00 going to sleep for 11 seconds 00000100:00100000:0.0:1788619923.580422:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619923.580631:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63fad80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589175552:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619923.580649:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 678955 00010000:00010000:1.0:1788619923.580663:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297c00/0x1cd7fe53687358ab lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb2d expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.580694:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619923.580697:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619923.580704:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297c00/0x1cd7fe53687358ab lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb2d expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.580717:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297c00, rc 0) 00000100:00100000:2.0:1788619923.580739:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571565696 00000100:00100000:2.0:1788619923.580746:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571565696 00000100:00100000:1.0:1788619923.580776:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f54a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571565568:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4079us (4245us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619923.580784:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1223 00000100:00100000:2.0:1788619923.580798:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 977 00000100:00100000:2.0:1788619923.580807:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571565696:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619923.580819:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619923.580834:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001dc9de0/0x1cd7fe53687358b2 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cb34 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619923.580885:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6533480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589175680:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619923.580935:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164300 going to sleep for 11 seconds 00000100:00100000:2.0:1788619923.583901:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6533480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589175680:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619923.583961:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000001dc9de0/0x1cd7fe53687358b2 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb34 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.583984:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001dc9de0/0x1cd7fe53687358b2 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb34 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.583998:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6a00, rc 0) 00000100:00100000:2.0:1788619923.584044:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571565696:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3239us (3631us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619923.584051:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 977 00000100:00100000:0.0:1788619923.587203:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619923.587523:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571565824 00000100:00100000:1.0:1788619923.587532:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571565824 00000100:00100000:1.0:1788619923.587583:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1224 00000100:00100000:1.0:1788619923.587592:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571565824:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619923.587611:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619923.587630:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297c00/0x1cd7fe53687358b9 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cb3b expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.587662:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368735849 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cad9 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619923.587710:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63fad80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589175808:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619923.587780:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7f00 going to sleep for 11 seconds 00000100:00100000:0.0:1788619923.589119:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619923.589182:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571565952 00000100:00100000:2.0:1788619923.589188:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571565952 00000100:00100000:2.0:1788619923.589233:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 978 00000100:00100000:2.0:1788619923.589240:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571565952:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619923.589254:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619923.589268:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041aa2bf0/0x1cd7fe53687358c0 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cb42 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619923.589315:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6533480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589175936:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619923.589396:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164300 going to sleep for 11 seconds 00000100:00100000:2.0:1788619923.592145:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6533480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589175936:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619923.592186:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000041aa2bf0/0x1cd7fe53687358c0 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb42 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.592212:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041aa2bf0/0x1cd7fe53687358c0 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb42 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.592224:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7a00, rc 0) 00000100:00100000:2.0:1788619923.592285:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571565952:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3045us (3173us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619923.592294:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 978 00000100:00100000:1.0:1788619923.593000:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63fad80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589175808:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619923.593015:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 678955 00010000:00010000:1.0:1788619923.593029:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297c00/0x1cd7fe53687358b9 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb3b expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.593044:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619923.593047:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619923.593052:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297c00/0x1cd7fe53687358b9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb3b expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.593063:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297c00, rc 0) 00000100:00100000:1.0:1788619923.593106:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571565824:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5515us (5911us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619923.593113:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1224 00000100:00100000:0.0:1788619923.597179:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619923.597486:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571566080 00000100:00100000:3.0:1788619923.597507:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571566080 00000100:00100000:3.0:1788619923.597565:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1225 00000100:00100000:3.0:1788619923.597575:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571566080:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619923.597595:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619923.597615:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687358c7 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cb49 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.597645:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368735849 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cad9 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.597656:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687358c7 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb49 expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.597701:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368735849 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cad9 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.597714:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368735849 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cad9 expref: 14 pid: 5844 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619923.597727:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a59880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589176064:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619923.597817:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619923.600749:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619923.600882:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a59880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589176064:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619923.600927:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687358c7 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb49 expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.600958:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619923.600961:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619923.600965:0:5844:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687358c7 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb49 expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.600975:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683da00, rc 0) 00000100:00100000:3.0:1788619923.601036:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:14908:x1875503571566080:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3463us (3867us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619923.601045:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1225 00000100:00100000:1.0:1788619923.602140:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571566208 00000100:00100000:1.0:1788619923.602149:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571566208 00002000:00100000:1.0:1788619923.602196:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6a5f800 x1875503571566208/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:19/0 lens 488/0 e 0 to 0 dl 1788619934 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619923.602206:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6a5f800 with x1875503571566208 00010000:00010000:1.0:1788619923.602215:0:5859:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368735849 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cad9 expref: 15 pid: 5844 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.602227:0:5859:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368735849 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cad9 expref: 15 pid: 5844 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788619923.602235:0:5859:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e77c6a5f800 00000100:00100000:1.0:1788619923.602316:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 32 00000100:00100000:1.0:1788619923.602352:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571566208:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:3.0:1788619923.606773:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 678955 -> 678955 00000004:00010000:3.0:1788619923.606784:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619923 -> 1788619923 00000004:00010000:3.0:1788619923.606787:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619923 -> 1788619922 00000004:00010000:3.0:1788619923.606789:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619923 -> 1788619923 00000004:00010000:3.0:1788619923.606792:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 390 -> 390 00000100:00100000:3.0:1788619923.606870:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2354:x1875503571566208:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 4540us (6123us total) trans 4294967740 rc 0/0 00000100:00100000:3.0:1788619923.606882:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 32 00002000:00100000:3.0:1788619923.606895:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6a5f800 x1875503571566208/t4294967740(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:19/0 lens 488/448 e 0 to 0 dl 1788619934 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619923.606904:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6a5f800 with x1875503571566208 00010000:00010000:3.0:1788619923.606912:0:5859:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368735849 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cad9 expref: 17 pid: 5844 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.606928:0:5859:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368735849 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cad9 expref: 17 pid: 5844 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788619923.606943:0:5859:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e77c6a5f800 00000100:00100000:0.0:1788619923.611210:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619923.611320:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571566336 00000100:00100000:1.0:1788619923.611348:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571566336 00010000:00010000:1.0:1788619923.611388:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368735849 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cad9 expref: 17 pid: 5844 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619923.611467:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 410 00000100:00100000:1.0:1788619923.611478:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5ca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2354:x1875503571566336:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619923.611560:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619923.611587:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368735849 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cad9 expref: 17 pid: 5844 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.611601:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368735849 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cad9 expref: 17 pid: 5844 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.611621:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368735849 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cad9 expref: 17 pid: 5844 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.611638:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619923.611644:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687358c7 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb49 expref: 9 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.611655:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687358c7 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb49 expref: 9 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.611665:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687358c7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb49 expref: 9 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.611715:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687358c7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb49 expref: 9 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619923.611729:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63fad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589176192:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619923.611872:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7f00 going to sleep for 11 seconds 00000100:00100000:0.0:1788619923.617662:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619923.617769:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63fad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589176192:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619923.617795:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619923.617852:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5ca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571566336:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 6376us (6653us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619923.617864:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 410 00000100:00100000:3.0:1788619923.617959:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571566464 00000100:00100000:3.0:1788619923.617966:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571566464 00000100:00100000:3.0:1788619923.618028:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 753 00000100:00100000:3.0:1788619923.618036:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571566464:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:3.0:1788619923.618221:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 678955 -> 678955 00000004:00010000:3.0:1788619923.618225:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619923 -> 1788619923 00000004:00010000:3.0:1788619923.618228:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619922 -> 1788619922 00000004:00010000:3.0:1788619923.618230:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619923 -> 1788619923 00000004:00010000:3.0:1788619923.618232:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 390 -> 390 00000100:00100000:3.0:1788619923.618267:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571566464:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 231us (617us total) trans 4294967741 rc 0/0 00000100:00100000:3.0:1788619923.618274:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 753 00000100:00100000:2.0:1788619923.620832:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619923.621053:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571566592 00000100:00100000:1.0:1788619923.621062:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571566592 00000100:00100000:1.0:1788619923.621119:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1226 00000100:00100000:1.0:1788619923.621129:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6533480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2352:x1875503571566592:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619923.621150:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619923.621169:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297c00/0x1cd7fe53687358ce lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cb50 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.621207:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687358c7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb49 expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619923.621246:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63fad80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589176320:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619923.621360:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7f00 going to sleep for 11 seconds 00000004:00010000:1.0:1788619923.623767:0:9417:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619922 -> 1788619923 00000100:00100000:1.0:1788619923.623807:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63fad80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589176320:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619923.623828:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 678955 00010000:00010000:1.0:1788619923.623846:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297c00/0x1cd7fe53687358ce lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb50 expref: 17 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.623865:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619923.623868:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619923.623873:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297c00/0x1cd7fe53687358ce lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb50 expref: 17 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.623893:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297c00, rc 0) 00000100:00100000:1.0:1788619923.623956:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6533480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2352:x1875503571566592:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2828us (3132us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619923.623965:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1226 00000100:00100000:0.0:1788619923.626391:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619923.626479:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571566720 00000100:00100000:1.0:1788619923.626488:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571566720 00000100:00100000:1.0:1788619923.626552:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1227 00000100:00100000:1.0:1788619923.626563:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2352:x1875503571566720:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619923.626583:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619923.626605:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296200/0x1cd7fe53687358d5 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cb57 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.626639:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687358c7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb49 expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619923.626686:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63fad80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589176448:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619923.626747:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7f00 going to sleep for 11 seconds 00000100:00100000:1.0:1788619923.629225:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63fad80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589176448:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619923.629250:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 678955 00010000:00010000:1.0:1788619923.629268:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296200/0x1cd7fe53687358d5 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb57 expref: 17 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.629287:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619923.629290:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619923.629295:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296200/0x1cd7fe53687358d5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb57 expref: 17 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.629311:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296200, rc 0) 00000100:00100000:1.0:1788619923.629530:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2352:x1875503571566720:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2969us (3147us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619923.629539:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1227 00000100:00100000:3.0:1788619923.632851:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619923.633162:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571566848 00000100:00100000:1.0:1788619923.633173:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571566848 00000100:00100000:1.0:1788619923.633220:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 979 00000100:00100000:1.0:1788619923.633230:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a59880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571566848:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619923.633249:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619923.633265:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005b4a2f71/0x1cd7fe53687358dc lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cb5e expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619923.633351:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63fad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589176576:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619923.633429:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7f00 going to sleep for 11 seconds 00000100:00100000:1.0:1788619923.635636:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63fad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589176576:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619923.635684:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005b4a2f71/0x1cd7fe53687358dc lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb5e expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.635706:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005b4a2f71/0x1cd7fe53687358dc lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb5e expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.635719:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297c00, rc 0) 00000100:00100000:1.0:1788619923.635767:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a59880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571566848:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2539us (2927us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619923.635779:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 979 00000100:00100000:3.0:1788619923.638866:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619923.638964:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571566976 00000100:00100000:1.0:1788619923.638975:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571566976 00000100:00100000:1.0:1788619923.639034:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 980 00000100:00100000:1.0:1788619923.639044:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571566976:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619923.639061:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619923.639077:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009166796c/0x1cd7fe53687358e3 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cb65 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619923.639133:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63fad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589176704:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619923.639205:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7f00 going to sleep for 11 seconds 00000100:00100000:1.0:1788619923.641246:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63fad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589176704:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619923.641281:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009166796c/0x1cd7fe53687358e3 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb65 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.641302:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009166796c/0x1cd7fe53687358e3 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb65 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.641314:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297e00, rc 0) 00000100:00100000:1.0:1788619923.641400:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571566976:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2357us (2542us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619923.641409:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 980 00000100:00100000:3.0:1788619923.644557:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619923.644645:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571567104 00000100:00100000:2.0:1788619923.644655:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571567104 00000100:00100000:2.0:1788619923.644711:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 981 00000100:00100000:2.0:1788619923.644720:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571567104:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619923.644737:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619923.644752:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000860f3b75/0x1cd7fe53687358ea lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cb6c expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619923.644814:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5dca680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589176832:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619923.644887:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164300 going to sleep for 11 seconds 00000100:00100000:2.0:1788619923.647415:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5dca680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589176832:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619923.647453:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000860f3b75/0x1cd7fe53687358ea lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb6c expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.647474:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000860f3b75/0x1cd7fe53687358ea lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb6c expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.647488:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7200, rc 0) 00000100:00100000:2.0:1788619923.647549:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571567104:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2830us (3000us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619923.647558:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 981 00000100:00100000:3.0:1788619923.653462:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619923.653557:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571567232 00000100:00100000:1.0:1788619923.653567:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571567232 00002000:00100000:1.0:1788619923.653615:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6a5a680 x1875503571567232/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:19/0 lens 504/0 e 0 to 0 dl 1788619934 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619923.653626:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6a5a680 with x1875503571567232 00000100:00100000:1.0:1788619923.653664:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 754 00000100:00100000:1.0:1788619923.653683:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571567232:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619923.663055:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571567232:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 9363us (9592us total) trans 0 rc 679936/679936 00000100:00100000:1.0:1788619923.663070:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 754 00002000:00100000:1.0:1788619923.663082:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6a5a680 x1875503571567232/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:19/0 lens 504/440 e 0 to 0 dl 1788619934 ref 1 fl Complete:/600/0 rc 679936/679936 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619923.663092:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6a5a680 with x1875503571567232 00000100:00100000:0.0:1788619923.666595:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619923.666698:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571567360 00000100:00100000:2.0:1788619923.666708:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571567360 00002000:00100000:2.0:1788619923.666756:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6a5df80 x1875503571567360/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:19/0 lens 488/0 e 0 to 0 dl 1788619934 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619923.666767:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6a5df80 with x1875503571567360 00000100:00100000:2.0:1788619923.666905:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 755 00000100:00100000:2.0:1788619923.666916:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2352:x1875503571567360:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619923.667219:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2352:x1875503571567360:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 304us (634us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619923.667228:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 755 00002000:00100000:2.0:1788619923.667235:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6a5df80 x1875503571567360/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:19/0 lens 488/440 e 0 to 0 dl 1788619934 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619923.667243:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6a5df80 with x1875503571567360 00000100:00100000:2.0:1788619923.669866:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619923.670039:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571567488 00000100:00100000:1.0:1788619923.670049:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571567488 00002000:00100000:1.0:1788619923.670092:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5dca680 x1875503571567488/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:19/0 lens 488/0 e 0 to 0 dl 1788619934 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619923.670104:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5dca680 with x1875503571567488 00000100:00100000:1.0:1788619923.670137:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 756 00000100:00100000:1.0:1788619923.670146:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dca680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571567488:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619923.670419:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dca680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571567488:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 273us (561us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619923.670427:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 756 00002000:00100000:1.0:1788619923.670435:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5dca680 x1875503571567488/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:19/0 lens 488/440 e 0 to 0 dl 1788619934 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619923.670442:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5dca680 with x1875503571567488 00000100:00100000:0.0:1788619923.672981:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619923.673245:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571567616 00000100:00100000:2.0:1788619923.673252:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571567616 00002000:00100000:2.0:1788619923.673293:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6a5f100 x1875503571567616/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:19/0 lens 488/0 e 0 to 0 dl 1788619934 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619923.673304:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6a5f100 with x1875503571567616 00000100:00100000:2.0:1788619923.673360:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 757 00000100:00100000:2.0:1788619923.673368:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571567616:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619923.673596:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571567616:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 228us (623us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619923.673603:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 757 00002000:00100000:2.0:1788619923.673609:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6a5f100 x1875503571567616/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:19/0 lens 488/440 e 0 to 0 dl 1788619934 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619923.673622:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6a5f100 with x1875503571567616 00000100:00100000:0.0:1788619923.676174:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619923.676255:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571567744 00000100:00100000:3.0:1788619923.676264:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571567744 00002000:00100000:3.0:1788619923.676307:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6a5c700 x1875503571567744/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:19/0 lens 488/0 e 0 to 0 dl 1788619934 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619923.676319:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6a5c700 with x1875503571567744 00000100:00100000:3.0:1788619923.676378:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 758 00000100:00100000:3.0:1788619923.676386:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571567744:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619923.676601:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571567744:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 215us (434us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619923.676607:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 758 00002000:00100000:3.0:1788619923.676613:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6a5c700 x1875503571567744/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:19/0 lens 488/440 e 0 to 0 dl 1788619934 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619923.676620:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6a5c700 with x1875503571567744 00000100:00100000:3.0:1788619923.679134:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619923.679236:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571567872 00000100:00100000:1.0:1788619923.679245:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571567872 00002000:00100000:1.0:1788619923.679285:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6a59c00 x1875503571567872/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:19/0 lens 488/0 e 0 to 0 dl 1788619934 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619923.679296:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6a59c00 with x1875503571567872 00000100:00100000:1.0:1788619923.679399:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 759 00000100:00100000:1.0:1788619923.679410:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a59c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571567872:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619923.679902:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a59c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571567872:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 493us (775us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619923.679910:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 759 00002000:00100000:1.0:1788619923.679917:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6a59c00 x1875503571567872/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:19/0 lens 488/440 e 0 to 0 dl 1788619934 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619923.679924:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6a59c00 with x1875503571567872 00000100:00100000:0.0:1788619923.681738:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619923.681826:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571568000 00000100:00100000:2.0:1788619923.681833:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571568000 00000100:00100000:2.0:1788619923.681879:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 982 00000100:00100000:2.0:1788619923.681888:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:14908:x1875503571568000:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619923.681903:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619923.681919:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3b293bd/0x1cd7fe53687358f1 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cb73 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.681960:0:6696:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe5368735850 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cae0 expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.681970:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(1 13) 00010000:00010000:2.0:1788619923.681975:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 13) 00010000:00010000:2.0:1788619923.681981:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3b293bd/0x1cd7fe53687358f1 lrc: 4/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->1064959] (req 1048576->1064959) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb73 expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.682013:0:6696:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe5368735850 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cae0 expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.682027:0:6696:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe5368735850 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cae0 expref: 5 pid: 6698 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619923.682041:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5dcad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589176960:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619923.682109:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164b40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619923.685302:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619923.685418:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571568128 00000100:00100000:1.0:1788619923.685427:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571568128 00010000:00010000:1.0:1788619923.685463:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe5368735850 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cae0 expref: 6 pid: 6698 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619923.685504:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 411 00000100:00100000:1.0:1788619923.685512:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c855a850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571568128:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619923.685526:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619923.685548:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe5368735850 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cae0 expref: 6 pid: 6698 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.685562:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe5368735850 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cae0 expref: 6 pid: 6698 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.685589:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 13) 00010000:00010000:1.0:1788619923.685593:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 13) 00010000:00010000:1.0:1788619923.685602:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000f3b293bd/0x1cd7fe53687358f1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1064959) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb73 expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.685612:0:5829:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f3b293bd/0x1cd7fe53687358f1 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1064959) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb73 expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.685648:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000f3b293bd/0x1cd7fe53687358f1 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1064959) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb73 expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619923.685663:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63fad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1875503589177088:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619923.685739:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7f00 going to sleep for 11 seconds 00010000:00010000:2.0:1788619923.686426:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe5368735850 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cae0 expref: 6 pid: 6698 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619923.686442:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5dcad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589176960:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619923.686464:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f3b293bd/0x1cd7fe53687358f1 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1064959) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb73 expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.686475:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7e00, rc 0) 00000100:00100000:2.0:1788619923.686523:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:14908:x1875503571568000:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4636us (4790us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619923.686531:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 982 00000100:00100000:1.0:1788619923.690100:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63fad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1875503589177088:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619923.690125:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619923.690180:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c855a850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571568128:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 4667us (4885us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619923.690192:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 411 00000100:00100000:3.0:1788619923.696756:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619923.696813:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619923.697076:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571568384 00000100:00100000:0.0:1788619923.697087:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571568384 00002000:00100000:0.0:1788619923.697137:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6a59500 x1875503571568384/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:19/0 lens 488/0 e 0 to 0 dl 1788619934 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619923.697149:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6a59500 with x1875503571568384 00000100:00100000:0.0:1788619923.697185:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 760 00000100:00100000:0.0:1788619923.697195:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a59500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571568384:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619923.698295:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571568256 00000100:00100000:2.0:1788619923.698304:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571568256 00002000:00100000:2.0:1788619923.698369:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1699839) req@ffff8e77c6a58e00 x1875503571568256/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:19/0 lens 488/0 e 0 to 0 dl 1788619934 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619923.698380:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a58e00 with x1875503571568256 ext(1048576->1699839) 00002000:00010000:2.0:1788619923.698385:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a58e00 00000100:00100000:2.0:1788619923.698405:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 566 00000100:00100000:2.0:1788619923.698414:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a58e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571568256:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619923.701420:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a59500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571568384:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 4215us (4595us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619923.701434:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 760 00002000:00100000:2.0:1788619923.701448:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6a59500 x1875503571568384/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:19/0 lens 488/440 e 0 to 0 dl 1788619934 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619923.701458:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6a59500 with x1875503571568384 00000100:00100000:2.0:1788619923.709200:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a58e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571568256:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 10776us (12442us total) trans 0 rc 651264/651264 00000100:00100000:2.0:1788619923.709214:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 566 00002000:00100000:2.0:1788619923.709227:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1699839) req@ffff8e77c6a58e00 x1875503571568256/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:19/0 lens 488/440 e 0 to 0 dl 1788619934 ref 1 fl Complete:/600/0 rc 651264/651264 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619923.709238:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a58e00 with x1875503571568256 ext(1048576->1699839) 00002000:00010000:2.0:1788619923.709244:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a58e00 00000100:00100000:1.0:1788619923.713431:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619923.713524:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571568512 00000100:00100000:3.0:1788619923.713534:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571568512 00000100:00100000:3.0:1788619923.713596:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1228 00000100:00100000:3.0:1788619923.713606:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63fad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2353:x1875503571568512:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619923.713626:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619923.713645:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe53687358f8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cb7a expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.713693:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687358c7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb49 expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619923.713733:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a5b480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589177216:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619923.713925:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619923.715159:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619923.715242:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571568640 00000100:00100000:1.0:1788619923.715249:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571568640 00000100:00100000:1.0:1788619923.715294:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 983 00000100:00100000:1.0:1788619923.715302:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a59180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571568640:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619923.715315:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619923.715362:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005b4a2f71/0x1cd7fe53687358ff lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cb81 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.715385:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 13) 00010000:00010000:1.0:1788619923.715390:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 13) 00010000:00010000:1.0:1788619923.715399:0:6698:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005b4a2f71/0x1cd7fe53687358ff lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb81 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.715416:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005b4a2f71/0x1cd7fe53687358ff lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb81 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.715428:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297c00, rc 0) 00000100:00100000:1.0:1788619923.715467:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a59180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571568640:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 166us (312us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619923.715474:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 983 00000100:00100000:3.0:1788619923.717632:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a5b480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589177216:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619923.717658:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 678955 00010000:00010000:3.0:1788619923.717688:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe53687358f8 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb7a expref: 17 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.717708:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619923.717710:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619923.717717:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe53687358f8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb7a expref: 17 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.717732:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d000, rc 0) 00000100:00100000:3.0:1788619923.717788:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63fad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2353:x1875503571568512:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4183us (4366us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619923.717796:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1228 00000100:00100000:0.0:1788619923.720566:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619923.720664:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571568768 00000100:00100000:1.0:1788619923.720682:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571568768 00000100:00100000:1.0:1788619923.720745:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1229 00000100:00100000:1.0:1788619923.720756:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2352:x1875503571568768:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619923.720777:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619923.720809:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296200/0x1cd7fe5368735906 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cb88 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.720845:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687358c7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb49 expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619923.720883:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63f9f80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589177344:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619923.721168:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7780 going to sleep for 11 seconds 00000100:00100000:1.0:1788619923.724407:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63f9f80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589177344:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619923.724435:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 678955 00010000:00010000:1.0:1788619923.724454:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296200/0x1cd7fe5368735906 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb88 expref: 17 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.724473:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619923.724477:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619923.724482:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296200/0x1cd7fe5368735906 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb88 expref: 17 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.724567:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296200, rc 0) 00000100:00100000:1.0:1788619923.724681:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2352:x1875503571568768:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3925us (4122us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619923.724690:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1229 00000100:00100000:2.0:1788619923.728210:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619923.728310:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571568896 00000100:00100000:1.0:1788619923.728319:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571568896 00000100:00100000:1.0:1788619923.728551:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1230 00000100:00100000:1.0:1788619923.728561:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dcad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2352:x1875503571568896:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619923.728582:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619923.728605:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe536873590d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cb8f expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.728641:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687358c7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb49 expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619923.728692:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63f9f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589177472:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619923.728776:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7780 going to sleep for 11 seconds 00000100:00100000:1.0:1788619923.731307:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63f9f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589177472:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619923.731360:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 678955 00010000:00010000:1.0:1788619923.731378:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe536873590d lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb8f expref: 17 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.731397:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619923.731400:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619923.731405:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe536873590d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb8f expref: 17 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.731419:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297e00, rc 0) 00000100:00100000:1.0:1788619923.731469:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dcad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2352:x1875503571568896:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2909us (3270us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619923.731477:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1230 00000100:00100000:2.0:1788619923.739578:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619923.739644:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571569024 00000100:00100000:1.0:1788619923.739654:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571569024 00000100:00100000:1.0:1788619923.739718:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 984 00000100:00100000:1.0:1788619923.739728:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe139450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+6:14908:x1875503571569024:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619923.739747:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619923.739768:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002abacdb6/0x1cd7fe5368735914 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cb96 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.739810:0:7654:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000f3b293bd/0x1cd7fe53687358f1 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1064959) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb73 expref: 5 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.739820:0:7654:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000005b4a2f71/0x1cd7fe53687358ff lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb81 expref: 7 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.739828:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 13) 00010000:00010000:1.0:1788619923.739832:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 13) 00010000:00010000:1.0:1788619923.739837:0:7654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002abacdb6/0x1cd7fe5368735914 lrc: 5/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [1261568->1265663] (req 1261568->1265663) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb96 expref: 7 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.739871:0:7654:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000005b4a2f71/0x1cd7fe53687358ff lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb81 expref: 7 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.739885:0:7654:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000005b4a2f71/0x1cd7fe53687358ff lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb81 expref: 7 pid: 6698 timeout: 516 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619923.739899:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63f9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589177600:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619923.740009:0:7654:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000f3b293bd/0x1cd7fe53687358f1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1064959) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb73 expref: 5 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.740019:0:7654:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000f3b293bd/0x1cd7fe53687358f1 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1064959) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb73 expref: 5 pid: 6696 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619923.740028:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63fa300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589177728:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619923.740053:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7780 going to sleep for 11 seconds 00000100:00100000:2.0:1788619923.743854:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619923.743907:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619923.744080:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63f9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589177600:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619923.744090:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63fa300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589177728:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619923.744122:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002abacdb6/0x1cd7fe5368735914 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [1261568->1265663] (req 1261568->1265663) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb96 expref: 7 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.744142:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296200, rc 0) 00000100:00100000:1.0:1788619923.744194:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe139450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+7:14908:x1875503571569024:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4468us (4627us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619923.744202:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 984 00000100:00100000:3.0:1788619923.744437:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571569152 00000100:00100000:3.0:1788619923.744444:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571569152 00010000:00010000:3.0:1788619923.744476:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000005b4a2f71/0x1cd7fe53687358ff lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb81 expref: 7 pid: 6698 timeout: 516 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619923.744506:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571569280 00000100:00100000:3.0:1788619923.744508:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571569280 00010000:00010000:3.0:1788619923.744518:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000f3b293bd/0x1cd7fe53687358f1 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1064959) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb73 expref: 6 pid: 6696 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619923.744535:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 412 00000100:00100000:3.0:1788619923.744542:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df96a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571569152:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619923.744554:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619923.744573:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000005b4a2f71/0x1cd7fe53687358ff lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb81 expref: 7 pid: 6698 timeout: 516 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.744590:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000005b4a2f71/0x1cd7fe53687358ff lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb81 expref: 7 pid: 6698 timeout: 516 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.744611:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005b4a2f71/0x1cd7fe53687358ff lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb81 expref: 7 pid: 6698 timeout: 516 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.744623:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619923.744640:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df96a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571569152:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 99us (796us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619923.744646:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 412 00000100:00100000:3.0:1788619923.744657:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 413 00000100:00100000:3.0:1788619923.744663:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69f8a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571569280:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619923.744681:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619923.744688:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000f3b293bd/0x1cd7fe53687358f1 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1064959) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb73 expref: 6 pid: 6696 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.744699:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000f3b293bd/0x1cd7fe53687358f1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1064959) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb73 expref: 6 pid: 6696 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.744712:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3b293bd/0x1cd7fe53687358f1 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1064959) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb73 expref: 6 pid: 6696 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.744721:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 13) 00010000:00010000:3.0:1788619923.744725:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 13) 00010000:00010000:3.0:1788619923.744733:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000002abacdb6/0x1cd7fe5368735914 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1261568->1265663) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb96 expref: 5 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.744743:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002abacdb6/0x1cd7fe5368735914 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1261568->1265663) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb96 expref: 5 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.744770:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000002abacdb6/0x1cd7fe5368735914 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1261568->1265663) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb96 expref: 5 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619923.744782:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a5b480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589177856:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619923.744832:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619923.748795:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619923.749094:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a5b480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589177856:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619923.749116:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619923.749168:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69f8a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571569280:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 4505us (5265us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619923.749178:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 413 00000100:00100000:2.0:1788619923.749659:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571569408 00000100:00100000:2.0:1788619923.749676:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571569408 00002000:00100000:2.0:1788619923.749715:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1261568->1265663) req@ffff8e77c6a5d880 x1875503571569408/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:19/0 lens 488/0 e 0 to 0 dl 1788619934 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619923.749726:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a5d880 with x1875503571569408 ext(1261568->1265663) 00002000:00010000:2.0:1788619923.749731:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a5d880 00000100:00100000:2.0:1788619923.749755:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 567 00000100:00100000:2.0:1788619923.749763:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571569408:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619923.750246:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571569408:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 482us (1460us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619923.750254:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 567 00002000:00100000:2.0:1788619923.750260:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1261568->1265663) req@ffff8e77c6a5d880 x1875503571569408/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:19/0 lens 488/4536 e 0 to 0 dl 1788619934 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619923.750268:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a5d880 with x1875503571569408 ext(1261568->1265663) 00002000:00010000:2.0:1788619923.750272:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a5d880 00000100:00100000:2.0:1788619923.757016:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619923.757169:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571569536 00000100:00100000:1.0:1788619923.757178:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571569536 00002000:00100000:1.0:1788619923.757223:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1265664->1269759) req@ffff8e77c69fad80 x1875503571569536/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:19/0 lens 488/0 e 0 to 0 dl 1788619934 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619923.757235:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c69fad80 with x1875503571569536 ext(1265664->1269759) 00002000:00010000:1.0:1788619923.757240:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c69fad80 00000100:00100000:1.0:1788619923.757276:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 568 00000100:00100000:1.0:1788619923.757286:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571569536:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619923.757596:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571569536:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 311us (588us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619923.757604:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 568 00002000:00100000:1.0:1788619923.757611:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1265664->1269759) req@ffff8e77c69fad80 x1875503571569536/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:19/0 lens 488/4536 e 0 to 0 dl 1788619934 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619923.757619:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c69fad80 with x1875503571569536 ext(1265664->1269759) 00002000:00010000:1.0:1788619923.757623:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c69fad80 00000100:00100000:3.0:1788619923.760779:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619923.761356:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571569664 00000100:00100000:2.0:1788619923.761366:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571569664 00002000:00100000:2.0:1788619923.761416:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1269760->1273855) req@ffff8e77c6a5b480 x1875503571569664/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:19/0 lens 488/0 e 0 to 0 dl 1788619934 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619923.761428:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a5b480 with x1875503571569664 ext(1269760->1273855) 00002000:00010000:2.0:1788619923.761434:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a5b480 00000100:00100000:2.0:1788619923.761471:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 569 00000100:00100000:2.0:1788619923.761481:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571569664:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619923.761751:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571569664:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 270us (982us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619923.761759:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 569 00002000:00100000:2.0:1788619923.761766:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1269760->1273855) req@ffff8e77c6a5b480 x1875503571569664/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:19/0 lens 488/4536 e 0 to 0 dl 1788619934 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619923.761774:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a5b480 with x1875503571569664 ext(1269760->1273855) 00002000:00010000:2.0:1788619923.761778:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a5b480 00000100:00100000:2.0:1788619923.766108:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619923.766259:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571569792 00000100:00100000:0.0:1788619923.766268:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571569792 00002000:00100000:0.0:1788619923.766315:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1273856->1277951) req@ffff8e77c69fa300 x1875503571569792/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:19/0 lens 488/0 e 0 to 0 dl 1788619934 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619923.766999:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c69fa300 with x1875503571569792 ext(1273856->1277951) 00002000:00010000:0.0:1788619923.767007:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c69fa300 00000100:00100000:0.0:1788619923.767037:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 570 00000100:00100000:0.0:1788619923.767106:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571569792:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619923.767430:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571569792:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 328us (1330us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619923.767437:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 570 00002000:00100000:0.0:1788619923.767445:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1273856->1277951) req@ffff8e77c69fa300 x1875503571569792/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:19/0 lens 488/4536 e 0 to 0 dl 1788619934 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619923.767453:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c69fa300 with x1875503571569792 ext(1273856->1277951) 00002000:00010000:0.0:1788619923.767457:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c69fa300 00000100:00100000:0.0:1788619923.772213:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619923.772308:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571569920 00000100:00100000:3.0:1788619923.772318:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571569920 00002000:00100000:3.0:1788619923.772381:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1277952->1282047) req@ffff8e77c6a5ea00 x1875503571569920/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:19/0 lens 488/0 e 0 to 0 dl 1788619934 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619923.772394:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a5ea00 with x1875503571569920 ext(1277952->1282047) 00002000:00010000:3.0:1788619923.772399:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a5ea00 00000100:00100000:3.0:1788619923.772470:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 571 00000100:00100000:3.0:1788619923.772481:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571569920:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619923.773375:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571569920:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 454us (730us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619923.773385:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 571 00002000:00100000:3.0:1788619923.773394:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1277952->1282047) req@ffff8e77c6a5ea00 x1875503571569920/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:19/0 lens 488/4536 e 0 to 0 dl 1788619934 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619923.773403:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a5ea00 with x1875503571569920 ext(1277952->1282047) 00002000:00010000:3.0:1788619923.773408:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a5ea00 00000100:00100000:0.0:1788619923.776754:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619923.777092:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571570048 00000100:00100000:2.0:1788619923.777102:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571570048 00002000:00100000:2.0:1788619923.777155:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1282048->1286143) req@ffff8e77c6a5fb80 x1875503571570048/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:19/0 lens 488/0 e 0 to 0 dl 1788619934 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619923.777167:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a5fb80 with x1875503571570048 ext(1282048->1286143) 00002000:00010000:2.0:1788619923.777172:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a5fb80 00000100:00100000:2.0:1788619923.777207:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 572 00000100:00100000:2.0:1788619923.777219:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571570048:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619923.777513:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571570048:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 295us (768us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619923.777521:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 572 00002000:00100000:2.0:1788619923.777528:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1282048->1286143) req@ffff8e77c6a5fb80 x1875503571570048/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:19/0 lens 488/4536 e 0 to 0 dl 1788619934 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619923.777535:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a5fb80 with x1875503571570048 ext(1282048->1286143) 00002000:00010000:2.0:1788619923.777538:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a5fb80 00000100:00100000:2.0:1788619923.782255:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619923.782381:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571570176 00000100:00100000:3.0:1788619923.782390:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571570176 00002000:00100000:3.0:1788619923.782441:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1261568->1286143) req@ffff8e77c69f9f80 x1875503571570176/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:19/0 lens 488/0 e 0 to 0 dl 1788619934 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619923.782452:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c69f9f80 with x1875503571570176 ext(1261568->1286143) 00002000:00010000:3.0:1788619923.782458:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c69f9f80 00000100:00100000:3.0:1788619923.782492:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 573 00000100:00100000:3.0:1788619923.782502:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571570176:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:2.0:1788619923.788480:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2352:x1875503571570176:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 5968us (6223us total) trans 4294967465 rc 0/0 00000100:00100000:2.0:1788619923.788506:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 573 00002000:00100000:2.0:1788619923.788521:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1261568->1286143) req@ffff8e77c69f9f80 x1875503571570176/t4294967465(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:19/0 lens 488/448 e 0 to 0 dl 1788619934 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619923.788531:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c69f9f80 with x1875503571570176 ext(1261568->1286143) 00002000:00010000:2.0:1788619923.788539:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c69f9f80 00000100:00100000:2.0:1788619923.792021:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619923.792151:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571570304 00000100:00100000:1.0:1788619923.792161:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571570304 00000100:00100000:1.0:1788619923.792229:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 985 00000100:00100000:1.0:1788619923.792240:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2352:x1875503571570304:12345-192.168.201.5@tcp:16:fsx.0 00000001:00080000:1.0:1788619923.832401:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967465 is committed 00000100:00100000:2.0:1788619923.833516:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571570304:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 41262us (41488us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619923.833530:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 985 00000100:00100000:0.0:1788619923.838313:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619923.838554:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571570432 00000100:00100000:2.0:1788619923.838565:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571570432 00000100:00100000:2.0:1788619923.838625:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 986 00000100:00100000:2.0:1788619923.838636:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571570432:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619923.838656:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619923.838684:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001dc9de0/0x1cd7fe536873591b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cb9d expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619923.838743:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69f8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589177984:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619923.839140:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164f00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619923.842158:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69f8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589177984:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619923.842196:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000001dc9de0/0x1cd7fe536873591b lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb9d expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.842222:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001dc9de0/0x1cd7fe536873591b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb9d expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.842234:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6a00, rc 0) 00000100:00100000:2.0:1788619923.842369:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571570432:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3733us (4063us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619923.842378:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 986 00000100:00100000:2.0:1788619923.847316:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619923.847396:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571570560 00000100:00100000:3.0:1788619923.847405:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571570560 00000100:00100000:3.0:1788619923.847463:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 987 00000100:00100000:3.0:1788619923.847474:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571570560:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619923.847504:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619923.847522:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062c34eb6/0x1cd7fe5368735922 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cba4 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619923.847577:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a5b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589178112:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619923.847889:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c300 going to sleep for 11 seconds 00000100:00100000:3.0:1788619923.851223:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a5b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589178112:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619923.851263:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000062c34eb6/0x1cd7fe5368735922 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cba4 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.851288:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062c34eb6/0x1cd7fe5368735922 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cba4 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.851300:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c800, rc 0) 00000100:00100000:2.0:1788619923.851724:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571570560:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4249us (4416us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619923.851732:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 987 00000100:00100000:0.0:1788619923.855608:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619923.855770:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571570688 00000100:00100000:2.0:1788619923.855780:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571570688 00000100:00100000:2.0:1788619923.855852:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 988 00000100:00100000:2.0:1788619923.855863:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571570688:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619923.855882:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619923.855899:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001dc9de0/0x1cd7fe5368735929 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cbab expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619923.855978:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69f8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589178240:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619923.856464:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164f00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619923.859416:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69f8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589178240:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619923.859457:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000001dc9de0/0x1cd7fe5368735929 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cbab expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.859483:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001dc9de0/0x1cd7fe5368735929 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cbab expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.859497:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6a00, rc 0) 00000100:00100000:2.0:1788619923.859558:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571570688:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3696us (3958us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619923.859568:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 988 00000100:00100000:0.0:1788619923.863697:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619923.864169:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571570816 00000100:00100000:1.0:1788619923.864179:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571570816 00000100:00100000:1.0:1788619923.864242:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 989 00000100:00100000:1.0:1788619923.864253:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571570816:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619923.864271:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619923.864288:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006901c1d2/0x1cd7fe5368735930 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cbb2 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619923.864373:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63f8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589178368:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619923.864458:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e70c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619923.869438:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63f8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589178368:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619923.869480:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000006901c1d2/0x1cd7fe5368735930 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cbb2 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.869521:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006901c1d2/0x1cd7fe5368735930 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cbb2 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.869536:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296a00, rc 0) 00000100:00100000:1.0:1788619923.869616:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571570816:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5365us (5928us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619923.869625:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 989 00000100:00100000:0.0:1788619923.874653:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619923.875427:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571570944 00000100:00100000:3.0:1788619923.875438:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571570944 00000100:00100000:3.0:1788619923.875502:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 990 00000100:00100000:3.0:1788619923.875512:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571570944:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619923.875530:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619923.875548:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062c34eb6/0x1cd7fe5368735937 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cbb9 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619923.875607:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a5b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589178496:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619923.875703:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c300 going to sleep for 11 seconds 00000100:00100000:3.0:1788619923.879073:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a5b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589178496:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619923.879114:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000062c34eb6/0x1cd7fe5368735937 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cbb9 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.879140:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062c34eb6/0x1cd7fe5368735937 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cbb9 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.879154:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c800, rc 0) 00000100:00100000:3.0:1788619923.879218:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571570944:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3707us (4574us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619923.879227:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 990 00000100:00100000:0.0:1788619923.883761:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619923.884314:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571571072 00000100:00100000:2.0:1788619923.884373:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571571072 00000100:00100000:2.0:1788619923.884477:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 991 00000100:00100000:2.0:1788619923.884486:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571571072:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619923.884505:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619923.884525:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001dc9de0/0x1cd7fe536873593e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cbc0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619923.884586:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69f8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589178624:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619923.884654:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164f00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619923.886654:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69f8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589178624:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619923.886699:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000001dc9de0/0x1cd7fe536873593e lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cbc0 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.888713:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001dc9de0/0x1cd7fe536873593e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cbc0 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.888729:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6a00, rc 0) 00000100:00100000:2.0:1788619923.889137:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571571072:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4649us (5383us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619923.889146:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 991 00000100:00100000:0.0:1788619923.894762:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619923.894858:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571571200 00000100:00100000:2.0:1788619923.894867:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571571200 00000100:00100000:2.0:1788619923.894918:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 992 00000100:00100000:2.0:1788619923.894928:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571571200:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619923.894946:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619923.894963:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001dc9de0/0x1cd7fe5368735945 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cbc7 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619923.895021:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69f8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589178752:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619923.895162:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164f00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619923.897776:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69f8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589178752:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619923.897886:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000001dc9de0/0x1cd7fe5368735945 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cbc7 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.897913:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001dc9de0/0x1cd7fe5368735945 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cbc7 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.897925:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6a00, rc 0) 00000100:00100000:2.0:1788619923.898133:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571571200:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3205us (3378us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619923.898143:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 992 00000100:00100000:0.0:1788619923.901768:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619923.901875:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571571328 00000100:00100000:1.0:1788619923.901884:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571571328 00000100:00100000:1.0:1788619923.901966:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 993 00000100:00100000:1.0:1788619923.901977:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7431c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571571328:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619923.901995:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619923.902013:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006901c1d2/0x1cd7fe536873594c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cbce expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619923.902075:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63f8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589178880:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619923.902159:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e70c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619923.904777:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63f8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589178880:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619923.904812:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000006901c1d2/0x1cd7fe536873594c lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cbce expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.904836:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006901c1d2/0x1cd7fe536873594c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cbce expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619923.904849:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296a00, rc 0) 00000100:00100000:1.0:1788619923.904895:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7431c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571571328:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2922us (3139us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619923.904902:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 993 00000100:00100000:0.0:1788619923.907282:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619923.907392:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571571456 00000100:00100000:3.0:1788619923.907401:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571571456 00000100:00100000:3.0:1788619923.907460:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 994 00000100:00100000:3.0:1788619923.907471:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74a9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571571456:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619923.907496:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619923.907513:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062c34eb6/0x1cd7fe5368735953 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cbd5 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619923.907561:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a5b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589179008:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619923.907625:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c300 going to sleep for 11 seconds 00000100:00100000:3.0:1788619923.909799:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a5b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589179008:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619923.909837:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000062c34eb6/0x1cd7fe5368735953 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cbd5 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.909863:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062c34eb6/0x1cd7fe5368735953 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cbd5 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.909877:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c800, rc 0) 00000100:00100000:3.0:1788619923.909970:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74a9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571571456:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2501us (2695us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619923.909980:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 994 00000100:00100000:0.0:1788619923.912792:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619923.913024:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571571584 00000100:00100000:2.0:1788619923.913034:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571571584 00000100:00100000:2.0:1788619923.913120:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 995 00000100:00100000:2.0:1788619923.913132:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74a8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571571584:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619923.913149:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619923.913166:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001dc9de0/0x1cd7fe536873595a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cbdc expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619923.913222:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69f8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589179136:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619923.913314:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164f00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619923.916753:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69f8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589179136:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619923.916791:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000001dc9de0/0x1cd7fe536873595a lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cbdc expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.916822:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001dc9de0/0x1cd7fe536873595a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cbdc expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.916836:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6a00, rc 0) 00000100:00100000:2.0:1788619923.916932:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74a8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571571584:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3801us (4148us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619923.916940:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 995 00000100:00100000:0.0:1788619923.926054:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619923.926649:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571571712 00000100:00100000:2.0:1788619923.926717:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571571712 00000100:00100000:2.0:1788619923.926836:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 996 00000100:00100000:2.0:1788619923.926848:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571571712:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619923.926866:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619923.926884:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001dc9de0/0x1cd7fe5368735961 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cbe3 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619923.926955:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69f8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589179264:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619923.927049:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164f00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619923.931510:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69f8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589179264:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619923.931549:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000001dc9de0/0x1cd7fe5368735961 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cbe3 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.931576:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001dc9de0/0x1cd7fe5368735961 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cbe3 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.931596:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6a00, rc 0) 00000100:00100000:2.0:1788619923.931656:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571571712:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4810us (5611us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619923.932938:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 996 00000100:00100000:0.0:1788619923.936629:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619923.936733:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571571840 00000100:00100000:2.0:1788619923.936777:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571571840 00000100:00100000:2.0:1788619923.936846:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 997 00000100:00100000:2.0:1788619923.936857:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571571840:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619923.936875:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619923.936892:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000073be6ea/0x1cd7fe5368735968 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cbea expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619923.936963:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69f8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589179392:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619923.937053:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164f00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619923.941405:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69f8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589179392:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619923.941445:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000073be6ea/0x1cd7fe5368735968 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cbea expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.941468:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000073be6ea/0x1cd7fe5368735968 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cbea expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.941480:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7c00, rc 0) 00000100:00100000:2.0:1788619923.941545:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571571840:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4690us (4926us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619923.941555:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 997 00000100:00100000:0.0:1788619923.949211:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619923.949359:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571571968 00000100:00100000:3.0:1788619923.949369:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571571968 00000100:00100000:3.0:1788619923.949426:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 998 00000100:00100000:3.0:1788619923.949436:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd93b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571571968:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619923.949455:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619923.949474:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062c34eb6/0x1cd7fe536873596f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cbf1 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619923.949543:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a5b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589179520:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619923.949626:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c300 going to sleep for 11 seconds 00000100:00100000:3.0:1788619923.953537:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a5b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589179520:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619923.953578:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000062c34eb6/0x1cd7fe536873596f lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cbf1 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.953604:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062c34eb6/0x1cd7fe536873596f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cbf1 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.953619:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c800, rc 0) 00000100:00100000:3.0:1788619923.953758:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd93b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571571968:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4322us (4554us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619923.953766:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 998 00000100:00100000:0.0:1788619923.959417:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619923.959706:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571572096 00000100:00100000:3.0:1788619923.959717:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571572096 00000100:00100000:3.0:1788619923.959834:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 999 00000100:00100000:3.0:1788619923.959845:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e6479c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571572096:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619923.959864:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619923.959881:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062c34eb6/0x1cd7fe5368735976 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cbf8 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619923.959958:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a5b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589179648:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619923.960052:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c300 going to sleep for 11 seconds 00000100:00100000:3.0:1788619923.963636:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a5b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589179648:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619923.963683:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000062c34eb6/0x1cd7fe5368735976 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cbf8 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.963709:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062c34eb6/0x1cd7fe5368735976 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cbf8 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.963721:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c800, rc 0) 00000100:00100000:3.0:1788619923.963779:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e6479c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571572096:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3936us (4371us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619923.963788:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 999 00000100:00100000:0.0:1788619923.973423:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619923.973772:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571572224 00000100:00100000:2.0:1788619923.973782:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571572224 00000100:00100000:2.0:1788619923.973834:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1000 00000100:00100000:2.0:1788619923.973845:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5019f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571572224:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619923.973863:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619923.973880:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000073be6ea/0x1cd7fe536873597d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cbff expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619923.973949:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69f8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589179776:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619923.974035:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164f00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619923.978265:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69f8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589179776:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619923.978305:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000073be6ea/0x1cd7fe536873597d lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cbff expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.978368:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000073be6ea/0x1cd7fe536873597d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cbff expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619923.978383:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7c00, rc 0) 00000100:00100000:2.0:1788619923.978435:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5019f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571572224:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4591us (5020us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619923.978444:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1000 00000100:00100000:0.0:1788619923.982783:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619923.983155:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571572352 00000100:00100000:2.0:1788619923.983165:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571572352 00000100:00100000:3.0:1788619923.983301:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1001 00000100:00100000:3.0:1788619923.983314:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8596300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571572352:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619923.983357:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619923.983377:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062c34eb6/0x1cd7fe5368735984 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cc06 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619923.983444:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a5b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589179904:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619923.983528:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c300 going to sleep for 11 seconds 00000100:00100000:3.0:1788619923.986078:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a5b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589179904:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619923.986116:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000062c34eb6/0x1cd7fe5368735984 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc06 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.986141:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062c34eb6/0x1cd7fe5368735984 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc06 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619923.986155:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c800, rc 0) 00000100:00100000:3.0:1788619923.986216:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8596300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571572352:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2904us (3443us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619923.986234:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1001 00000100:00100000:0.0:1788619923.998133:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619923.998210:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571572480 00000100:00100000:3.0:1788619923.998219:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571572480 00000100:00100000:3.0:1788619923.998272:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1002 00000100:00100000:3.0:1788619923.998281:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8595180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571572480:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619923.998297:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619923.998314:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062c34eb6/0x1cd7fe536873598b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cc0d expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619923.998405:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a5b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589180032:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619923.998476:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c300 going to sleep for 11 seconds 00000100:00100000:3.0:1788619924.000994:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a5b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589180032:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619924.001034:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000062c34eb6/0x1cd7fe536873598b lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc0d expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.001060:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062c34eb6/0x1cd7fe536873598b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc0d expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.001073:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c800, rc 0) 00000100:00100000:3.0:1788619924.001131:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8595180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571572480:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2850us (3007us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619924.001140:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1002 00000100:00100000:0.0:1788619924.005643:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619924.005829:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571572608 00000100:00100000:2.0:1788619924.005838:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571572608 00000100:00100000:2.0:1788619924.005912:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1003 00000100:00100000:2.0:1788619924.005932:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f26d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571572608:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619924.005965:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619924.005983:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000073be6ea/0x1cd7fe5368735992 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cc14 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619924.006039:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69f8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589180160:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619924.006120:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164f00 going to sleep for 11 seconds 00000100:00080000:3.0:1788619924.006706:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1788619924.009686:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69f8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589180160:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619924.009725:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000073be6ea/0x1cd7fe5368735992 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc14 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.009749:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000073be6ea/0x1cd7fe5368735992 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc14 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.009762:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7c00, rc 0) 00000100:00100000:2.0:1788619924.009827:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f26d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571572608:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3906us (4194us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619924.009836:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1003 00000100:00100000:1.0:1788619924.016837:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619924.017516:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571572736 00000100:00100000:3.0:1788619924.017603:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571572736 00002000:00100000:3.0:1788619924.017741:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c63f8000 x1875503571572736/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:20/0 lens 488/0 e 0 to 0 dl 1788619935 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619924.017753:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c63f8000 with x1875503571572736 00000100:00100000:3.0:1788619924.017794:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 761 00000100:00100000:3.0:1788619924.017805:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63f8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571572736:12345-192.168.201.5@tcp:4:fsx.0 00000100:00080000:2.0:1788619924.018805:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000004:00010000:0.0:1788619924.024519:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 678955 -> 987136 00000004:00010000:0.0:1788619924.024600:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619923 -> 1788619923 00000004:00010000:0.0:1788619924.024603:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619923 -> 1788619922 00000004:00010000:0.0:1788619924.024605:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619923 -> 1788619923 00000004:00010000:0.0:1788619924.024607:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 390 -> 390 00000100:00100000:0.0:1788619924.024778:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63f8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+13:2355:x1875503571572736:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 6969us (7945us total) trans 4294967742 rc 0/0 00000100:00100000:0.0:1788619924.024790:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 761 00002000:00100000:0.0:1788619924.024803:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c63f8000 x1875503571572736/t4294967742(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:20/0 lens 488/448 e 0 to 0 dl 1788619935 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619924.024813:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c63f8000 with x1875503571572736 00000100:00100000:1.0:1788619924.027890:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619924.028053:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571572864 00000100:00100000:3.0:1788619924.028064:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571572864 00000100:00100000:3.0:1788619924.028173:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1231 00000100:00100000:3.0:1788619924.028184:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63faa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+13:2355:x1875503571572864:12345-192.168.201.5@tcp:16:fsx.0 00000100:00100000:3.0:1788619924.066597:0:3319:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a5b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:3319:1875503589180288:0@lo:400:kworker.0 00000100:00100000:3.0:1788619924.066679:0:3319:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619924.066716:0:3319:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a5aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3319:1875503589180800:0@lo:400:kworker.0 00000100:00100000:3.0:1788619924.066744:0:3319:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619924.066770:0:3319:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1788619924.066776:0:3319:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a59880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3319:1875503589180672:0@lo:400:kworker.0 00000100:00100000:3.0:1788619924.066800:0:3319:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619924.066826:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a58a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3316:1875503589180416:0@lo:400:kworker.0 00000100:00100000:3.0:1788619924.066852:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619924.066869:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a5bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589180928:0@lo:400:kworker.0 00000100:00100000:3.0:1788619924.066915:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619924.066948:0:3316:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1788619924.066954:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a58700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3316:1875503589180544:0@lo:400:kworker.0 00000100:00100000:3.0:1788619924.066976:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619924.067005:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589180288 00000100:00100000:3.0:1788619924.067013:0:5836:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589180288 00000100:00100000:3.0:1788619924.067057:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 292 00000100:00100000:3.0:1788619924.067065:0:5836:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5ce3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+19:3319:x1875503589180288:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788619924.067111:0:5836:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5ce3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+19:3319:x1875503589180288:12345-0@lo:400:kworker.0 Request processed in 47us (432us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619924.067118:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 292 00000100:00100000:3.0:1788619924.067148:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589180800 00000100:00100000:3.0:1788619924.067152:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589180800 00000100:00100000:3.0:1788619924.067183:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589180416 00000100:00100000:3.0:1788619924.067186:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589180416 00000100:00100000:3.0:1788619924.067198:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589180544 00000100:00100000:3.0:1788619924.067200:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589180544 00000100:00100000:3.0:1788619924.067215:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 189 00000100:00100000:3.0:1788619924.067221:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a59180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3319:x1875503589180800:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788619924.067253:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a59180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3319:x1875503589180800:12345-0@lo:400:kworker.0 Request processed in 32us (508us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619924.067259:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 189 00000100:00100000:3.0:1788619924.067272:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 190 00000100:00100000:3.0:1788619924.067278:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f54380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3316:x1875503589180416:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788619924.067308:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f54380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3316:x1875503589180416:12345-0@lo:400:kworker.0 Request processed in 31us (455us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619924.067314:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 190 00000100:00100000:3.0:1788619924.067346:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 191 00000100:00100000:3.0:1788619924.067353:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f57480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3316:x1875503589180544:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788619924.067378:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f57480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3316:x1875503589180544:12345-0@lo:400:kworker.0 Request processed in 27us (401us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619924.067383:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 191 00000100:00100000:3.0:1788619924.067399:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589180672 00000100:00100000:3.0:1788619924.067402:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589180672 00000100:00100000:3.0:1788619924.067425:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589180928 00000100:00100000:3.0:1788619924.067428:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589180928 00000100:00100000:3.0:1788619924.067448:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 128 00000100:00100000:3.0:1788619924.067454:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3319:x1875503589180672:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788619924.067482:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3319:x1875503589180672:12345-0@lo:400:kworker.0 Request processed in 29us (682us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619924.067488:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 128 00000100:00100000:3.0:1788619924.067497:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 129 00000100:00100000:3.0:1788619924.067503:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f55c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503589180928:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788619924.067528:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f55c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503589180928:12345-0@lo:400:kworker.0 Request processed in 26us (613us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619924.067536:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 129 00000100:00080000:3.0:1788619924.067553:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1788619924.067556:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1788619924.067559:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788619924.067581:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a58700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3316:1875503589180544:0@lo:400:kworker.0 00000100:00100000:3.0:1788619924.067591:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a58a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3316:1875503589180416:0@lo:400:kworker.0 00000100:00100000:3.0:1788619924.067599:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a5bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589180928:0@lo:400:kworker.0 00000100:00100000:3.0:1788619924.067627:0:3319:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a59880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3319:1875503589180672:0@lo:400:kworker.0 00000100:00100000:3.0:1788619924.067635:0:3319:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a5b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:3319:1875503589180288:0@lo:400:kworker.0 00000100:00100000:3.0:1788619924.067643:0:3319:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a5aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3319:1875503589180800:0@lo:400:kworker.0 00000001:00080000:3.0:1788619924.068894:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967739 is committed 00000001:00080000:3.0:1788619924.068921:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967742 is committed 00000001:00080000:3.0:1788619924.069039:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967740 is committed 00000100:00100000:0.0:1788619924.069204:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63faa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571572864:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 41016us (41319us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619924.069213:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1231 00000100:00100000:2.0:1788619924.073047:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619924.073276:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571572992 00000100:00100000:3.0:1788619924.073286:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571572992 00000100:00100000:3.0:1788619924.073389:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1232 00000100:00100000:3.0:1788619924.073400:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69f8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2355:x1875503571572992:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619924.073420:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619924.073439:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368735999 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cc1b expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.073473:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687358c7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb49 expref: 9 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619924.073521:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f57800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589181056:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619924.073611:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c0c0 going to sleep for 11 seconds 00000100:00080000:2.0:1788619924.074219:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000004:00010000:1.0:1788619924.076233:0:9417:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619922 -> 1788619923 00000004:00010000:1.0:1788619924.076242:0:9417:0:(mdt_lvb.c:201:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks: 390 -> 902 00000100:00100000:1.0:1788619924.076267:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f57800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589181056:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619924.076288:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:1.0:1788619924.076306:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368735999 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc1b expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.076358:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619924.076361:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619924.076368:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368735999 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc1b expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.076383:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c800, rc 0) 00000100:00100000:1.0:1788619924.076431:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69f8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2355:x1875503571572992:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3032us (3392us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619924.076439:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1232 00000100:00100000:1.0:1788619924.079493:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619924.079734:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571573120 00000100:00100000:2.0:1788619924.079743:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571573120 00000100:00100000:2.0:1788619924.079841:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1233 00000100:00100000:2.0:1788619924.079851:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63fb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2352:x1875503571573120:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619924.079869:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619924.079890:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe53687359a0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cc22 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.079945:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687358c7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb49 expref: 9 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619924.079987:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69f8380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589181184:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619924.080083:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619924.082843:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69f8380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589181184:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619924.082870:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:2.0:1788619924.082889:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe53687359a0 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc22 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.083046:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619924.083051:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619924.083057:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe53687359a0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc22 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.083073:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6a00, rc 0) 00000100:00100000:2.0:1788619924.083168:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63fb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2352:x1875503571573120:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3318us (3692us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619924.083177:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1233 00000100:00100000:0.0:1788619924.086748:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619924.086814:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571573248 00000100:00100000:3.0:1788619924.086824:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571573248 00000100:00100000:3.0:1788619924.086883:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1004 00000100:00100000:3.0:1788619924.086894:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f26300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571573248:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619924.086922:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619924.086947:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a803caae/0x1cd7fe53687359a7 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cc29 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619924.087006:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f54700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589181312:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619924.087319:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619924.090034:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f54700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589181312:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619924.090072:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a803caae/0x1cd7fe53687359a7 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc29 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.090099:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a803caae/0x1cd7fe53687359a7 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc29 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.090113:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d000, rc 0) 00000100:00100000:3.0:1788619924.090174:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f26300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571573248:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3281us (3435us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619924.090182:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1004 00000100:00100000:1.0:1788619924.093886:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619924.093991:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571573376 00000100:00100000:0.0:1788619924.094000:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571573376 00000100:00100000:0.0:1788619924.094056:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1005 00000100:00100000:0.0:1788619924.094066:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63f8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571573376:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619924.094083:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619924.094102:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000052301317/0x1cd7fe53687359ae lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cc30 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619924.094160:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f25f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589181440:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619924.094246:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:0.0:1788619924.096502:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f25f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589181440:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619924.096548:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000052301317/0x1cd7fe53687359ae lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc30 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.096569:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000052301317/0x1cd7fe53687359ae lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc30 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.096582:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f51000, rc 0) 00000100:00100000:0.0:1788619924.096634:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63f8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571573376:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2570us (2758us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619924.096642:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1005 00000100:00100000:1.0:1788619924.100231:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619924.100317:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571573504 00000100:00100000:0.0:1788619924.100352:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571573504 00002000:00100000:0.0:1788619924.100417:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1204224->1208319) req@ffff8e77c63fa300 x1875503571573504/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/0 e 0 to 0 dl 1788619935 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619924.100428:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c63fa300 with x1875503571573504 ext(1204224->1208319) 00002000:00010000:0.0:1788619924.100435:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c63fa300 00000100:00100000:0.0:1788619924.100463:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 574 00000100:00100000:0.0:1788619924.100474:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63fa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571573504:12345-192.168.201.5@tcp:3:fsx.0 00000100:00080000:2.0:1788619924.100630:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1788619924.100931:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63fa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571573504:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 451us (703us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619924.100939:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 574 00002000:00100000:0.0:1788619924.100946:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1204224->1208319) req@ffff8e77c63fa300 x1875503571573504/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/4536 e 0 to 0 dl 1788619935 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619924.100955:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c63fa300 with x1875503571573504 ext(1204224->1208319) 00002000:00010000:0.0:1788619924.100959:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c63fa300 00000100:00100000:1.0:1788619924.103895:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619924.104110:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571573632 00000100:00100000:2.0:1788619924.104120:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571573632 00002000:00100000:2.0:1788619924.104167:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1208320->1212415) req@ffff8e77c63f9f80 x1875503571573632/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/0 e 0 to 0 dl 1788619935 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619924.104179:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c63f9f80 with x1875503571573632 ext(1208320->1212415) 00002000:00010000:2.0:1788619924.104184:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c63f9f80 00000100:00100000:2.0:1788619924.104215:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 575 00000100:00100000:2.0:1788619924.104223:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571573632:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619924.104446:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571573632:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 222us (561us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619924.104452:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 575 00002000:00100000:2.0:1788619924.104458:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1208320->1212415) req@ffff8e77c63f9f80 x1875503571573632/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/4536 e 0 to 0 dl 1788619935 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619924.104466:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c63f9f80 with x1875503571573632 ext(1208320->1212415) 00002000:00010000:2.0:1788619924.104469:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c63f9f80 00000100:00100000:2.0:1788619924.107563:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619924.107723:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571573760 00000100:00100000:0.0:1788619924.107732:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571573760 00002000:00100000:0.0:1788619924.107778:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1212416->1216511) req@ffff8e77c69f8380 x1875503571573760/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/0 e 0 to 0 dl 1788619935 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619924.107789:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c69f8380 with x1875503571573760 ext(1212416->1216511) 00002000:00010000:0.0:1788619924.107794:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c69f8380 00000100:00100000:0.0:1788619924.107825:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 576 00000100:00100000:0.0:1788619924.107835:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571573760:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619924.108070:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571573760:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 234us (515us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619924.108078:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 576 00002000:00100000:0.0:1788619924.108085:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1212416->1216511) req@ffff8e77c69f8380 x1875503571573760/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/4536 e 0 to 0 dl 1788619935 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619924.108093:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c69f8380 with x1875503571573760 ext(1212416->1216511) 00002000:00010000:0.0:1788619924.108097:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c69f8380 00000100:00100000:0.0:1788619924.111217:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619924.111316:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571573888 00000100:00100000:1.0:1788619924.111361:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571573888 00002000:00100000:1.0:1788619924.111408:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1216512->1220607) req@ffff8e77c5f25f80 x1875503571573888/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/0 e 0 to 0 dl 1788619935 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619924.111419:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5f25f80 with x1875503571573888 ext(1216512->1220607) 00002000:00010000:1.0:1788619924.111424:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5f25f80 00000100:00100000:1.0:1788619924.111444:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 577 00000100:00100000:1.0:1788619924.111452:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f25f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571573888:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619924.111650:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f25f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571573888:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 198us (441us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619924.111657:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 577 00002000:00100000:1.0:1788619924.111675:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1216512->1220607) req@ffff8e77c5f25f80 x1875503571573888/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/4536 e 0 to 0 dl 1788619935 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619924.111683:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5f25f80 with x1875503571573888 ext(1216512->1220607) 00002000:00010000:1.0:1788619924.111686:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5f25f80 00000100:00100000:1.0:1788619924.114782:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619924.114879:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571574016 00000100:00100000:2.0:1788619924.114888:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571574016 00002000:00100000:2.0:1788619924.114945:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1220608->1224703) req@ffff8e77c63f9180 x1875503571574016/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/0 e 0 to 0 dl 1788619935 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619924.114957:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c63f9180 with x1875503571574016 ext(1220608->1224703) 00002000:00010000:2.0:1788619924.114963:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c63f9180 00000100:00100000:2.0:1788619924.114994:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 578 00000100:00100000:2.0:1788619924.115003:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63f9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571574016:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619924.115223:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63f9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571574016:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 220us (450us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619924.115231:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 578 00002000:00100000:2.0:1788619924.115237:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1220608->1224703) req@ffff8e77c63f9180 x1875503571574016/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/4536 e 0 to 0 dl 1788619935 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619924.115245:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c63f9180 with x1875503571574016 ext(1220608->1224703) 00002000:00010000:2.0:1788619924.115248:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c63f9180 00000100:00100000:2.0:1788619924.117956:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619924.118219:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571574144 00000100:00100000:0.0:1788619924.118229:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571574144 00002000:00100000:0.0:1788619924.118279:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1224704->1228799) req@ffff8e77c69f9880 x1875503571574144/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/0 e 0 to 0 dl 1788619935 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619924.118290:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c69f9880 with x1875503571574144 ext(1224704->1228799) 00002000:00010000:0.0:1788619924.118295:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c69f9880 00000100:00100000:0.0:1788619924.118350:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 579 00000100:00100000:0.0:1788619924.118361:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571574144:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619924.118570:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571574144:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 210us (623us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619924.118577:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 579 00002000:00100000:0.0:1788619924.118584:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1224704->1228799) req@ffff8e77c69f9880 x1875503571574144/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/4536 e 0 to 0 dl 1788619935 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619924.118592:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c69f9880 with x1875503571574144 ext(1224704->1228799) 00002000:00010000:0.0:1788619924.118595:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c69f9880 00000100:00100000:0.0:1788619924.121988:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619924.122089:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571574272 00000100:00100000:1.0:1788619924.122098:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571574272 00002000:00100000:1.0:1788619924.122146:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1228800->1232895) req@ffff8e77c5f24380 x1875503571574272/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/0 e 0 to 0 dl 1788619935 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619924.122157:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5f24380 with x1875503571574272 ext(1228800->1232895) 00002000:00010000:1.0:1788619924.122162:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5f24380 00000100:00100000:1.0:1788619924.122194:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 580 00000100:00100000:1.0:1788619924.122204:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f24380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571574272:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619924.122430:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f24380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571574272:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 227us (451us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619924.122438:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 580 00002000:00100000:1.0:1788619924.122446:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1228800->1232895) req@ffff8e77c5f24380 x1875503571574272/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/4536 e 0 to 0 dl 1788619935 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619924.122454:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5f24380 with x1875503571574272 ext(1228800->1232895) 00002000:00010000:1.0:1788619924.122458:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5f24380 00000100:00100000:1.0:1788619924.126527:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619924.126698:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571574400 00000100:00100000:2.0:1788619924.126708:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571574400 00002000:00100000:2.0:1788619924.126755:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1232896->1236991) req@ffff8e77c63f8e00 x1875503571574400/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/0 e 0 to 0 dl 1788619935 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619924.126766:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c63f8e00 with x1875503571574400 ext(1232896->1236991) 00002000:00010000:2.0:1788619924.126772:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c63f8e00 00000100:00100000:2.0:1788619924.126808:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 581 00000100:00100000:2.0:1788619924.126817:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63f8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571574400:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619924.127026:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63f8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571574400:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 209us (507us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619924.127033:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 581 00002000:00100000:2.0:1788619924.127040:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1232896->1236991) req@ffff8e77c63f8e00 x1875503571574400/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/4536 e 0 to 0 dl 1788619935 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619924.127049:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c63f8e00 with x1875503571574400 ext(1232896->1236991) 00002000:00010000:2.0:1788619924.127053:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c63f8e00 00000100:00100000:2.0:1788619924.129957:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619924.130099:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571574528 00000100:00100000:0.0:1788619924.130108:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571574528 00002000:00100000:0.0:1788619924.130155:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1236992->1241087) req@ffff8e77c69faa00 x1875503571574528/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/0 e 0 to 0 dl 1788619935 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619924.130166:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c69faa00 with x1875503571574528 ext(1236992->1241087) 00002000:00010000:0.0:1788619924.130171:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c69faa00 00000100:00100000:0.0:1788619924.130204:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 582 00000100:00100000:0.0:1788619924.130213:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69faa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571574528:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619924.130437:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69faa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571574528:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 223us (487us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619924.130445:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 582 00002000:00100000:0.0:1788619924.130452:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1236992->1241087) req@ffff8e77c69faa00 x1875503571574528/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/4536 e 0 to 0 dl 1788619935 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619924.130462:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c69faa00 with x1875503571574528 ext(1236992->1241087) 00002000:00010000:0.0:1788619924.130465:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c69faa00 00000100:00100000:0.0:1788619924.133080:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619924.133164:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571574656 00000100:00100000:3.0:1788619924.133173:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571574656 00002000:00100000:3.0:1788619924.133218:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1241088->1245183) req@ffff8e77c5f27800 x1875503571574656/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/0 e 0 to 0 dl 1788619935 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619924.133229:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5f27800 with x1875503571574656 ext(1241088->1245183) 00002000:00010000:3.0:1788619924.133235:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5f27800 00000100:00100000:3.0:1788619924.133264:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 583 00000100:00100000:3.0:1788619924.133274:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f27800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571574656:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619924.133535:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f27800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571574656:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 262us (463us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619924.133543:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 583 00002000:00100000:3.0:1788619924.133550:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1241088->1245183) req@ffff8e77c5f27800 x1875503571574656/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/4536 e 0 to 0 dl 1788619935 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619924.133558:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5f27800 with x1875503571574656 ext(1241088->1245183) 00002000:00010000:3.0:1788619924.133561:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5f27800 00000100:00100000:1.0:1788619924.136430:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619924.136518:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571574784 00000100:00100000:2.0:1788619924.136527:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571574784 00002000:00100000:2.0:1788619924.136570:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1245184->1249279) req@ffff8e77c63fa680 x1875503571574784/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/0 e 0 to 0 dl 1788619935 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619924.136581:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c63fa680 with x1875503571574784 ext(1245184->1249279) 00002000:00010000:2.0:1788619924.136586:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c63fa680 00000100:00100000:2.0:1788619924.136614:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 584 00000100:00100000:2.0:1788619924.136622:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571574784:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619924.136809:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571574784:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 186us (385us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619924.136815:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 584 00002000:00100000:2.0:1788619924.136821:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1245184->1249279) req@ffff8e77c63fa680 x1875503571574784/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/4536 e 0 to 0 dl 1788619935 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619924.136829:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c63fa680 with x1875503571574784 ext(1245184->1249279) 00002000:00010000:2.0:1788619924.136832:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c63fa680 00000100:00100000:2.0:1788619924.139382:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619924.139465:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571574912 00000100:00100000:0.0:1788619924.139474:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571574912 00002000:00100000:0.0:1788619924.139529:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1249280->1253375) req@ffff8e77c69fbb80 x1875503571574912/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/0 e 0 to 0 dl 1788619935 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619924.139540:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c69fbb80 with x1875503571574912 ext(1249280->1253375) 00002000:00010000:0.0:1788619924.139545:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c69fbb80 00000100:00100000:0.0:1788619924.139577:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 585 00000100:00100000:0.0:1788619924.139586:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571574912:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619924.139788:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571574912:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 203us (415us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619924.139796:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 585 00002000:00100000:0.0:1788619924.139802:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1249280->1253375) req@ffff8e77c69fbb80 x1875503571574912/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/4536 e 0 to 0 dl 1788619935 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619924.139810:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c69fbb80 with x1875503571574912 ext(1249280->1253375) 00002000:00010000:0.0:1788619924.139813:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c69fbb80 00000100:00100000:0.0:1788619924.142879:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619924.143431:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571575040 00000100:00100000:3.0:1788619924.143441:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571575040 00002000:00100000:2.0:1788619924.143788:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1253376->1257471) req@ffff8e77c5f25880 x1875503571575040/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/0 e 0 to 0 dl 1788619935 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619924.143800:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5f25880 with x1875503571575040 ext(1253376->1257471) 00002000:00010000:2.0:1788619924.143806:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5f25880 00000100:00100000:2.0:1788619924.143873:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 586 00000100:00100000:2.0:1788619924.143884:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f25880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571575040:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619924.144114:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f25880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571575040:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 230us (1243us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619924.144122:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 586 00002000:00100000:2.0:1788619924.144130:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1253376->1257471) req@ffff8e77c5f25880 x1875503571575040/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/4536 e 0 to 0 dl 1788619935 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619924.144138:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5f25880 with x1875503571575040 ext(1253376->1257471) 00002000:00010000:2.0:1788619924.144142:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5f25880 00000100:00100000:1.0:1788619924.146379:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619924.147071:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571575168 00000100:00100000:0.0:1788619924.147081:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571575168 00002000:00100000:0.0:1788619924.147127:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1257472->1261567) req@ffff8e77c63f8380 x1875503571575168/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/0 e 0 to 0 dl 1788619935 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619924.147139:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c63f8380 with x1875503571575168 ext(1257472->1261567) 00002000:00010000:0.0:1788619924.147145:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c63f8380 00000100:00100000:3.0:1788619924.147175:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 587 00000100:00100000:3.0:1788619924.147185:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571575168:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619924.147410:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571575168:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 225us (1038us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619924.147417:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 587 00002000:00100000:3.0:1788619924.147424:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1257472->1261567) req@ffff8e77c63f8380 x1875503571575168/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/4536 e 0 to 0 dl 1788619935 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619924.147440:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c63f8380 with x1875503571575168 ext(1257472->1261567) 00002000:00010000:3.0:1788619924.147444:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c63f8380 00000100:00100000:3.0:1788619924.150461:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619924.150818:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571575296 00000100:00100000:1.0:1788619924.150828:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571575296 00000100:00100000:1.0:1788619924.150884:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1006 00000100:00100000:1.0:1788619924.150894:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f54700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571575296:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619924.150914:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619924.150944:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003086a8b7/0x1cd7fe53687359b5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cc37 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619924.151006:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fefd6d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589181568:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619924.151096:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7d80 going to sleep for 11 seconds 00002000:00010000:2.0:1788619924.153448:0:6698:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619923 -> 1788619924 00000100:00100000:2.0:1788619924.153477:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fefd6d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589181568:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619924.153522:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003086a8b7/0x1cd7fe53687359b5 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc37 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.153548:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003086a8b7/0x1cd7fe53687359b5 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc37 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.153562:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296400, rc 0) 00000100:00100000:2.0:1788619924.153620:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f54700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571575296:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2727us (3169us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619924.153628:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1006 00000100:00100000:2.0:1788619924.156698:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619924.156812:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571575424 00000100:00100000:3.0:1788619924.156821:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571575424 00000100:00100000:3.0:1788619924.156869:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1007 00000100:00100000:3.0:1788619924.156880:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571575424:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619924.156899:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619924.156934:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a803caae/0x1cd7fe53687359bc lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cc3e expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619924.157001:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f54000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589181696:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619924.157144:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c000 going to sleep for 11 seconds 00000100:00100000:3.0:1788619924.159796:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f54000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589181696:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619924.159834:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a803caae/0x1cd7fe53687359bc lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc3e expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.159862:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a803caae/0x1cd7fe53687359bc lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc3e expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.159875:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d000, rc 0) 00000100:00100000:3.0:1788619924.160071:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571575424:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3191us (3381us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619924.160080:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1007 00000100:00100000:1.0:1788619924.167743:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619924.167821:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571575552 00000100:00100000:3.0:1788619924.167833:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571575552 00000100:00100000:3.0:1788619924.167891:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1234 00000100:00100000:3.0:1788619924.168056:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571575552:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619924.168080:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619924.168101:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687359c3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cc45 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.168138:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687358c7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb49 expref: 9 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619924.168180:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f54000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589181824:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619924.168303:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c000 going to sleep for 11 seconds 00000004:00010000:3.0:1788619924.170688:0:5844:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619923 -> 1788619924 00000100:00100000:3.0:1788619924.170721:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f54000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589181824:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619924.170743:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:3.0:1788619924.170761:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687359c3 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc45 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.170780:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619924.170783:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619924.170789:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687359c3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc45 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.170803:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d800, rc 0) 00000100:00100000:3.0:1788619924.171128:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571575552:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3002us (3168us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619924.171142:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1234 00000100:00100000:2.0:1788619924.173877:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619924.174075:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571575680 00000100:00100000:1.0:1788619924.174084:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571575680 00000100:00100000:1.0:1788619924.174162:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1235 00000100:00100000:1.0:1788619924.174172:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69f9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2353:x1875503571575680:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619924.174191:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619924.174209:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296a00/0x1cd7fe53687359ca lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cc4c expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.174242:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687358c7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb49 expref: 9 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619924.174279:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900d5e680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589181952:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619924.174369:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7d80 going to sleep for 11 seconds 00000100:00100000:3.0:1788619924.176473:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900d5e680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589181952:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619924.176563:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:3.0:1788619924.176584:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296a00/0x1cd7fe53687359ca lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc4c expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.176604:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619924.176607:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619924.176612:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296a00/0x1cd7fe53687359ca lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc4c expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.176626:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296a00, rc 0) 00000100:00100000:3.0:1788619924.176735:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69f9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2353:x1875503571575680:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2563us (2867us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619924.176744:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1235 00000100:00100000:2.0:1788619924.180032:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619924.180193:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571575936 00000100:00100000:3.0:1788619924.180204:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571575936 00000100:00100000:3.0:1788619924.180264:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1236 00000100:00100000:3.0:1788619924.180274:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69f8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:14908:x1875503571575936:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:3.0:1788619924.180363:0:5846:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619924.180390:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297400/0x1cd7fe536873585e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cae7 expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.180409:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619924.180414:0:5846:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788619924.180951:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6e00/0x1cd7fe53687359d1 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.180968:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe536873586c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1caee expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.180978:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6e00/0x1cd7fe53687359d1 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.181013:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe536873586c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1caee expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.181027:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe536873586c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1caee expref: 9 pid: 5846 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619924.181041:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69f9500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589182080:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619924.181168:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164780 going to sleep for 11 seconds 00000100:00100000:2.0:1788619924.185947:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619924.186144:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571576064 00000100:00100000:1.0:1788619924.186153:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571576064 00010000:00010000:1.0:1788619924.186190:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe536873586c lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1caee expref: 10 pid: 5846 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619924.186248:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 414 00000100:00100000:1.0:1788619924.186257:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fb480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571576064:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619924.186270:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619924.186277:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe536873586c lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1caee expref: 10 pid: 5846 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.186289:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe536873586c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1caee expref: 10 pid: 5846 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.186310:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619924.186316:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6e00/0x1cd7fe53687359d1 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.186343:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6e00/0x1cd7fe53687359d1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.186352:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6e00/0x1cd7fe53687359d1 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.186372:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619924.186420:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fb480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571576064:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 164us (482us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619924.186427:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 414 00010000:00010000:0.0:1788619924.186564:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe536873586c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1caee expref: 9 pid: 5846 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619924.186580:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69f9500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589182080:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619924.186601:0:5846:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6e00/0x1cd7fe53687359d1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.186610:0:5846:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6e00/0x1cd7fe53687359d1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.186615:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6e00/0x1cd7fe53687359d1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788619924.186743:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967743 00000004:00010000:0.0:1788619924.186791:0:5846:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 987136 -> 987136 00000004:00010000:0.0:1788619924.186795:0:5846:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619923 -> 1788619924 00000004:00010000:0.0:1788619924.186798:0:5846:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619924 -> 1788619922 00000004:00010000:0.0:1788619924.186800:0:5846:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619923 -> 1788619924 00000004:00010000:0.0:1788619924.186802:0:5846:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 902 -> 902 00000004:00010000:0.0:1788619924.186810:0:5846:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000cd524954 reply state 0000000030680b34 transno 4294967744 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6e00/0x1cd7fe53687359d1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619924.186824:0:5846:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788619924.186890:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69f8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571575936:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 6618us (6867us total) trans 4294967744 rc 0/0 00000100:00100000:0.0:1788619924.186897:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1236 00000020:00080000:0.0:1788619924.186901:0:5846:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:1.0:1788619924.191167:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00010000:00010000:0.0:1788619924.192138:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6e00/0x1cd7fe53687359d1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.192152:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6e00/0x1cd7fe53687359d1 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.192159:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6e00/0x1cd7fe53687359d1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.192164:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78cdcc6e00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788619924.192170:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6e00/0x1cd7fe53687359d1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.192177:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6e00/0x1cd7fe53687359d1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.192193:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619924.192196:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788619924.192200:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6e00/0x1cd7fe53687359d1 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.192206:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6e00/0x1cd7fe53687359d1 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619924.192386:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571576192 00000100:00100000:3.0:1788619924.192396:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571576192 00000100:00100000:3.0:1788619924.192721:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1237 00000100:00100000:3.0:1788619924.192733:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571576192:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619924.192767:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619924.192786:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687359d8 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cc53 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.192815:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687358c7 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb49 expref: 8 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.192824:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687359d8 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc53 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.192857:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687358c7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb49 expref: 8 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.192871:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687358c7 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb49 expref: 8 pid: 5844 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619924.192885:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f54000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589182208:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619924.192991:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48ccc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619924.199555:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619924.199900:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571576320 00000100:00100000:1.0:1788619924.199928:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571576320 00010000:00010000:1.0:1788619924.199972:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687358c7 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb49 expref: 9 pid: 5844 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619924.200014:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 415 00000100:00100000:1.0:1788619924.200025:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f55880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571576320:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619924.200041:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619924.200073:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687358c7 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb49 expref: 9 pid: 5844 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.200087:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687358c7 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb49 expref: 9 pid: 5844 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.200110:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619924.200117:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687359d8 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc53 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.200128:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687359d8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc53 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.200138:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687359d8 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc53 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.200180:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687359d8 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc53 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619924.200195:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f1880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589182336:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619924.200281:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7b40 going to sleep for 11 seconds 00010000:00010000:0.0:1788619924.200822:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687358c7 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb49 expref: 9 pid: 5844 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619924.200840:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f54000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589182208:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619924.200863:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687359d8 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc53 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.200875:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619924.200878:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619924.200881:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d800, rc 0) 00000100:00100000:0.0:1788619924.200920:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571576192:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8189us (9761us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619924.200942:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1237 00000100:00100000:1.0:1788619924.203090:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f1880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589182336:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619924.203115:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619924.203172:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f55880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+8:2354:x1875503571576320:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3149us (3627us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619924.203184:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 415 00000100:00100000:3.0:1788619924.206371:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619924.206605:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571576448 00000100:00100000:2.0:1788619924.206614:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571576448 00002000:00010000:2.0:1788619924.206655:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1948097->18446744073709551615) 00002000:00100000:2.0:1788619924.206670:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5f54000 with x1875503571576448 ext(1948097->18446744073709551615) 00002000:00010000:2.0:1788619924.206682:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5f54000. 00000100:00100000:2.0:1788619924.206711:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 588 00000100:00100000:2.0:1788619924.206721:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f54000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571576448:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:2.0:1788619924.207025:0:6703:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1697912 -> 1948097 00002000:00010000:2.0:1788619924.207029:0:6703:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619923 -> 1788619924 00002000:00010000:2.0:1788619924.207031:0:6703:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619924 -> 1788619923 00002000:00010000:2.0:1788619924.207033:0:6703:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619923 -> 1788619924 00002000:00010000:2.0:1788619924.207036:0:6703:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 263 -> 263 00000100:00100000:2.0:1788619924.207095:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f54000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571576448:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 375us (774us total) trans 4294967466 rc 0/0 00000100:00100000:2.0:1788619924.207102:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 588 00002000:00010000:2.0:1788619924.207106:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1948097->18446744073709551615) 00002000:00100000:2.0:1788619924.207109:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5f54000 with x1875503571576448 ext(1948097->18446744073709551615) 00002000:00010000:2.0:1788619924.207113:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5f54000. 00000100:00100000:1.0:1788619924.213648:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619924.213840:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571576576 00000100:00100000:3.0:1788619924.213851:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571576576 00000100:00100000:3.0:1788619924.213914:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1238 00000100:00100000:3.0:1788619924.213934:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+8:14908:x1875503571576576:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619924.213964:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619924.213992:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe53687359df lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cc5a expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.214086:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe53687359e6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.214103:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe53687359e6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.214116:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe53687359e6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788619924.214142:0:9417:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1948097 00000004:00200000:3.0:1788619924.214146:0:9417:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788619924.214163:0:9417:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe53687359e6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.214187:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe53687359df lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc5a expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.214205:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe53687359e6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc5a expref: 9 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.214214:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619924.214216:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788619924.214220:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d000, rc 0) 00000100:00100000:3.0:1788619924.214290:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571576576:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 367us (652us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619924.214297:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1238 00000100:00100000:3.0:1788619924.218000:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619924.218048:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619924.218180:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571576704 00000100:00100000:0.0:1788619924.218190:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571576704 00000100:00100000:0.0:1788619924.218251:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1239 00000100:00100000:0.0:1788619924.218262:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f57800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571576704:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619924.218283:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619924.218305:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50000/0x1cd7fe53687359ed lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cc61 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.218382:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687359d8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc53 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619924.218420:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f25c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589182464:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619924.218506:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa840 going to sleep for 11 seconds 00000004:00010000:0.0:1788619924.221505:0:5846:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619922 -> 1788619924 00000100:00100000:0.0:1788619924.221535:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f25c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589182464:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619924.221560:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:0.0:1788619924.221578:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50000/0x1cd7fe53687359ed lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc61 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.221598:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619924.221601:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619924.221606:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50000/0x1cd7fe53687359ed lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc61 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.221621:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f50000, rc 0) 00000100:00100000:0.0:1788619924.221684:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f57800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571576704:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3424us (3694us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619924.221693:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1239 00000100:00100000:1.0:1788619924.228683:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571576832 00000100:00100000:1.0:1788619924.228698:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571576832 00000100:00100000:1.0:1788619924.228763:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1008 00000100:00100000:1.0:1788619924.228774:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571576832:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619924.228792:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619924.228809:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005b4a2f71/0x1cd7fe53687359f4 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cc68 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619924.228870:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be7800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589182592:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619924.228973:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7b40 going to sleep for 11 seconds 00002000:00010000:1.0:1788619924.237757:0:6696:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619923 -> 1788619924 00000100:00100000:1.0:1788619924.237790:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be7800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589182592:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619924.237826:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005b4a2f71/0x1cd7fe53687359f4 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc68 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.237851:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005b4a2f71/0x1cd7fe53687359f4 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc68 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.237864:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297c00, rc 0) 00000100:00100000:1.0:1788619924.237943:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571576832:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 9171us (19894us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619924.237954:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1008 00000100:00100000:3.0:1788619924.248585:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619924.248633:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619924.248712:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571577088 00000100:00100000:1.0:1788619924.248722:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571577088 00000100:00100000:0.0:1788619924.248738:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571576960 00000100:00100000:0.0:1788619924.248742:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571576960 00000100:00100000:1.0:1788619924.248783:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1009 00000100:00100000:0.0:1788619924.248783:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1240 00000100:00100000:1.0:1788619924.248793:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a59880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571577088:12345-192.168.201.5@tcp:101:fsx.0 00000100:00100000:0.0:1788619924.248799:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571576960:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619924.248810:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619924.248810:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619924.248826:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005b4a2f71/0x1cd7fe53687359fb lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cc76 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.248828:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50000/0x1cd7fe5368735a02 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cc6f expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.248862:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687359d8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc53 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619924.248885:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be7800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589182720:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619924.248885:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f25c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589182848:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619924.248967:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa840 going to sleep for 11 seconds 00000100:00100000:2.0:1788619924.252940:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7b40 going to sleep for 11 seconds 00010000:00010000:0.0:1788619924.264384:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788619924.264859:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be7800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589182720:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619924.264897:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005b4a2f71/0x1cd7fe53687359fb lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc76 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.264922:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005b4a2f71/0x1cd7fe53687359fb lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc76 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.264945:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297c00, rc 0) 00000100:00100000:3.0:1788619924.265004:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a59880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571577088:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 16211us (16369us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619924.265013:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1009 00000100:00100000:0.0:1788619924.266442:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f25c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589182848:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619924.266467:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:0.0:1788619924.266596:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50000/0x1cd7fe5368735a02 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc6f expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.266614:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619924.266617:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619924.266623:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50000/0x1cd7fe5368735a02 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc6f expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.266636:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f50000, rc 0) 00000100:00100000:2.0:1788619924.266744:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571576960:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 17951us (18167us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619924.266753:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1240 00000100:00100000:2.0:1788619924.272390:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619924.272491:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619924.272856:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571577216 00000100:00100000:1.0:1788619924.272866:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571577216 00000100:00100000:1.0:1788619924.272945:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1241 00000100:00100000:1.0:1788619924.272955:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69f9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571577216:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619924.272975:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619924.272996:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe5368735a09 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cc7d expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.273033:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687359d8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc53 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619924.273073:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be7800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589182976:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619924.273152:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7600 going to sleep for 11 seconds 00000100:00100000:3.0:1788619924.274053:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571577344 00000100:00100000:3.0:1788619924.274061:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571577344 00000100:00100000:3.0:1788619924.274099:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1010 00000100:00100000:3.0:1788619924.274108:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571577344:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619924.274120:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619924.274134:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a72c5200/0x1cd7fe5368735a10 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cc84 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619924.274177:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a5bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589183104:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619924.274238:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48cd80 going to sleep for 11 seconds 00000100:00100000:3.0:1788619924.277299:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a5bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589183104:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619924.277370:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a72c5200/0x1cd7fe5368735a10 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc84 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.277394:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a72c5200/0x1cd7fe5368735a10 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc84 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.277408:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c400, rc 0) 00000100:00100000:3.0:1788619924.277459:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571577344:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3352us (4977us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619924.277467:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1010 00000100:00100000:1.0:1788619924.277576:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be7800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589182976:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619924.277589:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:1.0:1788619924.277603:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe5368735a09 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc7d expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.277617:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619924.277619:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619924.277624:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe5368735a09 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc7d expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.277634:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297e00, rc 0) 00000100:00100000:1.0:1788619924.277668:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69f9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571577216:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4704us (5278us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619924.277675:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1241 00000100:00100000:2.0:1788619924.281371:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619924.281404:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619924.281465:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571577472 00000100:00100000:0.0:1788619924.281473:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571577472 00000100:00100000:0.0:1788619924.281537:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1242 00000100:00100000:0.0:1788619924.281547:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69f9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571577472:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619924.281567:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619924.281585:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50000/0x1cd7fe5368735a17 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cc8b expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.281620:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687359d8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc53 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619924.281668:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f25c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589183232:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619924.281743:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa840 going to sleep for 11 seconds 00000100:00100000:3.0:1788619924.282104:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571577600 00000100:00100000:3.0:1788619924.282110:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571577600 00000100:00100000:3.0:1788619924.282158:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1011 00000100:00100000:3.0:1788619924.282166:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571577600:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619924.282175:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619924.282187:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe5368735a1e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cc92 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619924.282220:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a5bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589183360:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619924.282260:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48cd80 going to sleep for 11 seconds 00000100:00100000:3.0:1788619924.285195:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a5bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589183360:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619924.285234:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe5368735a1e lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc92 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.285273:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe5368735a1e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc92 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.285287:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c600, rc 0) 00000100:00100000:3.0:1788619924.285388:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571577600:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3222us (3982us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619924.285397:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1011 00000100:00100000:0.0:1788619924.285670:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f25c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589183232:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619924.285687:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:0.0:1788619924.285700:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50000/0x1cd7fe5368735a17 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc8b expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.285715:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619924.285718:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619924.285723:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50000/0x1cd7fe5368735a17 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc8b expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.285734:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f50000, rc 0) 00000100:00100000:0.0:1788619924.285771:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69f9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571577472:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4225us (4445us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619924.285778:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1242 00000100:00100000:2.0:1788619924.289842:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619924.289915:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571577728 00000100:00100000:3.0:1788619924.290016:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571577728 00000100:00100000:3.0:1788619924.290079:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1243 00000100:00100000:3.0:1788619924.290089:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63fdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571577728:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619924.290112:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619924.290134:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368735a25 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cc99 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.290222:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368735a2c lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.290238:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368735a2c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.290250:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368735a2c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788619924.290275:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1948097 00000004:00200000:3.0:1788619924.290278:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788619924.290291:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368735a2c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.290310:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368735a25 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc99 expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.290438:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368735a2c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc99 expref: 15 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.290445:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619924.290447:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788619924.290450:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683dc00, rc 0) 00000100:00100000:3.0:1788619924.290535:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63fdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:14908:x1875503571577728:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 448us (704us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619924.290539:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1243 00000100:00100000:1.0:1788619924.292576:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619924.292615:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571577856 00000100:00100000:3.0:1788619924.292623:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571577856 00000100:00100000:3.0:1788619924.292685:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 293 00000100:00100000:3.0:1788619924.292695:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5ce2450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:597450a2-e90c-408c-b401-a0ce48eed204+13:2355:x1875503571577856:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:3.0:1788619924.292760:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5ce2450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:597450a2-e90c-408c-b401-a0ce48eed204+13:2355:x1875503571577856:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 66us (190us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619924.292768:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 293 00000100:00100000:1.0:1788619924.296029:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619924.296187:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571578112 00000100:00100000:0.0:1788619924.296197:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571578112 00000100:00100000:0.0:1788619924.296244:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1244 00000100:00100000:0.0:1788619924.296251:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571578112:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:0.0:1788619924.296296:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619924.296318:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe53687359e6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc5a expref: 9 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.296358:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619924.296362:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788619924.296536:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50000/0x1cd7fe5368735a33 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.296546:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368735a2c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc99 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.296552:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50000/0x1cd7fe5368735a33 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.296579:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368735a2c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc99 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.296589:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368735a2c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc99 expref: 14 pid: 5844 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619924.296601:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f25c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589183488:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619924.296674:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa840 going to sleep for 11 seconds 00000100:00100000:0.0:1788619924.298529:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f25c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589183488:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619924.298559:0:5845:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50000/0x1cd7fe5368735a33 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619924.299637:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619924.299686:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571578240 00000100:00100000:0.0:1788619924.299693:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571578240 00010000:00010000:0.0:1788619924.299733:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368735a2c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc99 expref: 15 pid: 5844 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619924.299773:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 416 00000100:00100000:0.0:1788619924.299782:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6858380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571578240:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619924.299808:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619924.299815:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368735a2c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc99 expref: 15 pid: 5844 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.299827:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368735a2c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc99 expref: 15 pid: 5844 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.299854:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368735a2c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc99 expref: 15 pid: 5844 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.299869:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619924.299873:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50000/0x1cd7fe5368735a33 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.299883:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50000/0x1cd7fe5368735a33 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.299891:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50000/0x1cd7fe5368735a33 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.299919:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788619924.299980:0:5845:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50000/0x1cd7fe5368735a33 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619924.299981:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6858380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571578240:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 201us (348us total) trans 0 rc 0/0 00010000:00010000:1.0:1788619924.299988:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50000/0x1cd7fe5368735a33 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619924.299989:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 416 00000001:00080000:1.0:1788619924.300097:0:5845:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967745 00000004:00010000:1.0:1788619924.300145:0:5845:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 987136 -> 987136 00000004:00010000:1.0:1788619924.300149:0:5845:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619924 -> 1788619924 00000004:00010000:1.0:1788619924.300152:0:5845:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619924 -> 1788619922 00000004:00010000:1.0:1788619924.300154:0:5845:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619924 -> 1788619924 00000004:00010000:1.0:1788619924.300156:0:5845:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 902 -> 902 00000004:00010000:1.0:1788619924.300162:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000001b69db7f reply state 000000006e2937d4 transno 4294967746 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50000/0x1cd7fe5368735a33 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619924.300176:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788619924.300231:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571578112:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 3980us (4209us total) trans 4294967746 rc 0/0 00000100:00100000:1.0:1788619924.300236:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1244 00000020:00080000:1.0:1788619924.300238:0:5845:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1788619924.302112:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619924.302203:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571578368 00000100:00100000:1.0:1788619924.302211:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571578368 00000100:00100000:1.0:1788619924.302259:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1245 00000100:00100000:1.0:1788619924.302269:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63fc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571578368:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619924.302290:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619924.302308:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe5368735a3a lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cca0 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.302360:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687359d8 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc53 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.302368:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe5368735a3a lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cca0 expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.302393:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687359d8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc53 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.302402:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687359d8 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc53 expref: 13 pid: 5844 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619924.302411:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f4380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589183616:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619924.302462:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7600 going to sleep for 11 seconds 00010000:00010000:0.0:1788619924.302768:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50000/0x1cd7fe5368735a33 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.302778:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50000/0x1cd7fe5368735a33 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.302784:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50000/0x1cd7fe5368735a33 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.302789:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c5f50000 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788619924.302794:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50000/0x1cd7fe5368735a33 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.302806:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50000/0x1cd7fe5368735a33 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.302820:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619924.302823:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788619924.302827:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50000/0x1cd7fe5368735a33 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.302834:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50000/0x1cd7fe5368735a33 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619924.304143:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f4380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589183616:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619924.304168:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe5368735a3a lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cca0 expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.304180:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619924.304182:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619924.304185:0:5846:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe5368735a3a lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cca0 expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.304191:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297000, rc 0) 00000100:00100000:1.0:1788619924.304227:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63fc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571578368:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 1961us (2122us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619924.304233:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1245 00000100:00100000:3.0:1788619924.305006:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619924.305081:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571578496 00000100:00100000:1.0:1788619924.305087:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571578496 00010000:00010000:1.0:1788619924.305112:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687359d8 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc53 expref: 14 pid: 5844 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619924.305141:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 417 00000100:00100000:1.0:1788619924.305146:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571578496:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619924.305156:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619924.305176:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687359d8 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc53 expref: 14 pid: 5844 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.305185:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687359d8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc53 expref: 14 pid: 5844 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.305201:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687359d8 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cc53 expref: 14 pid: 5844 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.305211:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619924.305215:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe5368735a3a lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cca0 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.305222:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe5368735a3a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cca0 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.305228:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe5368735a3a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cca0 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.305250:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe5368735a3a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cca0 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619924.305259:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f4380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589183744:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619924.305307:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7600 going to sleep for 11 seconds 00000100:00100000:1.0:1788619924.306817:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f4380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589183744:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619924.306842:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619924.306879:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571578496:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 1732us (1877us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619924.306887:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 417 00000100:00100000:3.0:1788619924.308080:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619924.308148:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571578624 00000100:00100000:1.0:1788619924.308154:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571578624 00000100:00100000:1.0:1788619924.308194:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1012 00000100:00100000:1.0:1788619924.308199:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:14908:x1875503571578624:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619924.308214:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619924.308228:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe5368735a41 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cca7 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.308250:0:6696:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000002abacdb6/0x1cd7fe5368735914 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1261568->1265663) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb96 expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.308257:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 13) 00010000:00010000:1.0:1788619924.308260:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(3 13) 00010000:00010000:1.0:1788619924.308264:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe5368735a41 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cca7 expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.308287:0:6696:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000002abacdb6/0x1cd7fe5368735914 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1261568->1265663) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb96 expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.308296:0:6696:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000002abacdb6/0x1cd7fe5368735914 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1261568->1265663) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb96 expref: 6 pid: 7654 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619924.308306:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589183872:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619924.308385:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7600 going to sleep for 11 seconds 00000100:00100000:2.0:1788619924.310700:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619924.310721:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589183872:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619924.310753:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe5368735a41 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cca7 expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.310781:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296e00, rc 0) 00000100:00100000:1.0:1788619924.310914:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:14908:x1875503571578624:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2704us (2829us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619924.310923:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1012 00000100:00100000:3.0:1788619924.311593:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571578752 00000100:00100000:3.0:1788619924.311599:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571578752 00010000:00010000:0.0:1788619924.311688:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000002abacdb6/0x1cd7fe5368735914 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1261568->1265663) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb96 expref: 7 pid: 7654 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619924.311717:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 418 00000100:00100000:0.0:1788619924.311726:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8559450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571578752:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619924.311740:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619924.311761:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000002abacdb6/0x1cd7fe5368735914 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1261568->1265663) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb96 expref: 7 pid: 7654 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.311770:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000002abacdb6/0x1cd7fe5368735914 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1261568->1265663) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb96 expref: 7 pid: 7654 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.311787:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002abacdb6/0x1cd7fe5368735914 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1261568->1265663) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cb96 expref: 7 pid: 7654 timeout: 516 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.311802:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(3 13) 00010000:00010000:0.0:1788619924.311807:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(3 13) 00010000:00010000:0.0:1788619924.311813:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe5368735a41 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cca7 expref: 5 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.311823:0:13330:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe5368735a41 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cca7 expref: 5 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.311856:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe5368735a41 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cca7 expref: 5 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619924.311867:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f25c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589184000:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619924.312016:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa840 going to sleep for 11 seconds 00000100:00100000:2.0:1788619924.313769:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619924.313782:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f25c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589184000:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619924.313804:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619924.313853:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8559450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571578752:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 2127us (3165us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619924.313864:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 418 00000100:00100000:3.0:1788619924.314544:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571578880 00000100:00100000:3.0:1788619924.314551:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571578880 00002000:00010000:3.0:1788619924.314579:0:6701:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (2028302->18446744073709551615) 00002000:00100000:3.0:1788619924.314582:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c63fdc00 with x1875503571578880 ext(2028302->18446744073709551615) 00002000:00010000:3.0:1788619924.314590:0:6701:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c63fdc00. 00000100:00100000:3.0:1788619924.314612:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 589 00000100:00100000:3.0:1788619924.314621:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571578880:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:3.0:1788619924.314897:0:6701:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1948097 -> 2028302 00002000:00010000:3.0:1788619924.314900:0:6701:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619924 -> 1788619924 00002000:00010000:3.0:1788619924.314903:0:6701:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619924 -> 1788619923 00002000:00010000:3.0:1788619924.314905:0:6701:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619924 -> 1788619924 00002000:00010000:3.0:1788619924.314908:0:6701:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 263 -> 263 00000100:00100000:3.0:1788619924.315010:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571578880:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 388us (1247us total) trans 4294967467 rc 0/0 00000100:00100000:3.0:1788619924.315019:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 589 00002000:00010000:3.0:1788619924.315022:0:6701:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (2028302->18446744073709551615) 00002000:00100000:3.0:1788619924.315024:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c63fdc00 with x1875503571578880 ext(2028302->18446744073709551615) 00002000:00010000:3.0:1788619924.315027:0:6701:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c63fdc00. 00000100:00100000:2.0:1788619924.317944:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619924.318066:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571579008 00000100:00100000:0.0:1788619924.318075:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571579008 00002000:00100000:0.0:1788619924.318130:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1699840->2031615) req@ffff8e77c63ff100 x1875503571579008/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:20/0 lens 488/0 e 0 to 0 dl 1788619935 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619924.318142:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c63ff100 with x1875503571579008 ext(1699840->2031615) 00002000:00010000:0.0:1788619924.318147:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c63ff100 00000100:00100000:0.0:1788619924.318216:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 590 00000100:00100000:0.0:1788619924.318226:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63ff100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571579008:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619924.322107:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63ff100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571579008:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 3870us (4160us total) trans 0 rc 331776/331776 00000100:00100000:1.0:1788619924.322121:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 590 00002000:00100000:1.0:1788619924.322134:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1699840->2031615) req@ffff8e77c63ff100 x1875503571579008/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:20/0 lens 488/440 e 0 to 0 dl 1788619935 ref 1 fl Complete:/600/0 rc 331776/331776 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619924.322145:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c63ff100 with x1875503571579008 ext(1699840->2031615) 00002000:00010000:1.0:1788619924.322152:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c63ff100 00000100:00100000:3.0:1788619924.325973:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619924.326133:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571579136 00000100:00100000:2.0:1788619924.326141:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571579136 00002000:00100000:2.0:1788619924.326180:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1970176->2028301) req@ffff8e77c6a58700 x1875503571579136/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:20/0 lens 488/0 e 0 to 0 dl 1788619935 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619924.326188:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a58700 with x1875503571579136 ext(1970176->2028301) 00002000:00010000:2.0:1788619924.326191:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a58700 00000100:00100000:2.0:1788619924.326218:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 591 00000100:00100000:2.0:1788619924.326227:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a58700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571579136:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:0.0:1788619924.328016:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a58700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571579136:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 1781us (2046us total) trans 4294967468 rc 0/0 00000100:00100000:0.0:1788619924.328028:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 591 00002000:00100000:0.0:1788619924.328038:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1970176->2028301) req@ffff8e77c6a58700 x1875503571579136/t4294967468(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:20/0 lens 488/448 e 0 to 0 dl 1788619935 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619924.328048:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a58700 with x1875503571579136 ext(1970176->2028301) 00002000:00010000:0.0:1788619924.328053:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a58700 00000100:00100000:3.0:1788619924.329458:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619924.329545:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571579264 00000100:00100000:0.0:1788619924.329551:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571579264 00000100:00100000:0.0:1788619924.329591:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1013 00000100:00100000:0.0:1788619924.329600:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571579264:12345-192.168.201.5@tcp:16:fsx.0 00000100:00100000:1.0:1788619924.382306:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571579264:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 52696us (52842us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619924.382320:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1013 00000001:00080000:2.0:1788619924.382424:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967468 is committed 00000001:00080000:2.0:1788619924.382440:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967466 is committed 00000100:00100000:1.0:1788619924.384966:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619924.385248:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571579392 00000100:00100000:3.0:1788619924.385258:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571579392 00000100:00100000:3.0:1788619924.385399:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1246 00000100:00100000:3.0:1788619924.385411:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571579392:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619924.385433:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619924.385454:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368735a48 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ccae expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.385553:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368735a4f lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.385570:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368735a4f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.385585:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368735a4f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788619924.385616:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 2028302 00000004:00200000:3.0:1788619924.385620:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788619924.385639:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368735a4f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.385680:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368735a48 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ccae expref: 15 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.385700:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368735a4f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ccae expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.385710:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619924.385713:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788619924.385717:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c600, rc 0) 00000100:00100000:3.0:1788619924.385779:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571579392:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 371us (823us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619924.385786:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1246 00000100:00100000:3.0:1788619924.392717:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619924.393801:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571579520 00000100:00100000:1.0:1788619924.393813:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571579520 00000100:00100000:1.0:1788619924.393883:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1247 00000100:00100000:1.0:1788619924.393894:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69ff480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571579520:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619924.393915:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619924.394123:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296600/0x1cd7fe5368735a56 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ccb5 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.394163:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe5368735a3a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cca0 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619924.394204:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f5500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589184128:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619924.394292:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7540 going to sleep for 11 seconds 00000100:00100000:1.0:1788619924.396591:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619924.396692:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571579648 00000100:00100000:3.0:1788619924.396700:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571579648 00000100:00100000:3.0:1788619924.396770:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1014 00000100:00100000:3.0:1788619924.396780:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571579648:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619924.396796:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619924.396812:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a72c5200/0x1cd7fe5368735a5d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ccbc expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619924.396868:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69fd180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589184256:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619924.396969:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c300 going to sleep for 11 seconds 00000004:00010000:1.0:1788619924.398379:0:5845:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619922 -> 1788619924 00000100:00100000:1.0:1788619924.398405:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f5500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589184128:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619924.398427:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:1.0:1788619924.398443:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296600/0x1cd7fe5368735a56 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ccb5 expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.398461:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619924.398464:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619924.398470:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296600/0x1cd7fe5368735a56 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ccb5 expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.398485:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296600, rc 0) 00000100:00100000:1.0:1788619924.398537:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69ff480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571579520:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4645us (5829us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619924.398545:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1247 00002000:00010000:3.0:1788619924.400740:0:6698:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619923 -> 1788619924 00002000:00010000:3.0:1788619924.400748:0:6698:0:(ofd_lvb.c:284:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks: 263 -> 391 00000100:00100000:3.0:1788619924.400770:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69fd180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589184256:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619924.400800:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a72c5200/0x1cd7fe5368735a5d lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ccbc expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.400822:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a72c5200/0x1cd7fe5368735a5d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ccbc expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.400834:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c400, rc 0) 00000100:00100000:3.0:1788619924.400955:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571579648:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4176us (4372us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619924.400963:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1014 00000100:00100000:3.0:1788619924.404685:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619924.404828:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619924.405133:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571579776 00000100:00100000:1.0:1788619924.405142:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571579776 00000100:00100000:1.0:1788619924.405202:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1248 00000100:00100000:1.0:1788619924.405213:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69ff800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571579776:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619924.405232:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619924.405251:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296600/0x1cd7fe5368735a64 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ccc3 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.405293:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe5368735a3a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cca0 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619924.405366:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f5500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589184384:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619924.405442:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7540 going to sleep for 11 seconds 00000100:00100000:2.0:1788619924.407215:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571579904 00000100:00100000:2.0:1788619924.407227:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571579904 00000100:00100000:2.0:1788619924.407347:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1015 00000100:00100000:2.0:1788619924.407359:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571579904:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619924.407375:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619924.407390:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d2409dc/0x1cd7fe5368735a6b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ccca expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619924.407446:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5026d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589184512:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619924.407529:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164240 going to sleep for 11 seconds 00000100:00100000:2.0:1788619924.409566:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5026d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589184512:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619924.409606:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000d2409dc/0x1cd7fe5368735a6b lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ccca expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.409631:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d2409dc/0x1cd7fe5368735a6b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ccca expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.409644:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6000, rc 0) 00000100:00100000:2.0:1788619924.409748:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571579904:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2392us (5073us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619924.409757:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1015 00000100:00100000:1.0:1788619924.409965:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f5500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589184384:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619924.409980:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:1.0:1788619924.409994:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296600/0x1cd7fe5368735a64 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ccc3 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.410008:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619924.410011:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619924.410016:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296600/0x1cd7fe5368735a64 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ccc3 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.410027:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296600, rc 0) 00000100:00100000:1.0:1788619924.410062:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69ff800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571579776:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4851us (5234us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619924.410069:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1248 00000100:00100000:0.0:1788619924.418934:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619924.419196:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571580032 00000100:00100000:3.0:1788619924.419205:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571580032 00000100:00100000:3.0:1788619924.419258:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1249 00000100:00100000:3.0:1788619924.419265:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f25c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571580032:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619924.419284:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619924.419301:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368735a72 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ccd1 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.419402:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368735a79 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.419414:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368735a79 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.419423:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368735a79 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788619924.419444:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 2028302 00000004:00200000:3.0:1788619924.419448:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788619924.419462:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368735a79 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.419482:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368735a72 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ccd1 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.419499:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368735a79 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ccd1 expref: 12 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.419507:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619924.419510:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788619924.419514:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683de00, rc 0) 00000100:00100000:3.0:1788619924.419570:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f25c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:14908:x1875503571580032:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 305us (649us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619924.419575:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1249 00000100:00100000:0.0:1788619924.425073:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619924.425298:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619924.425838:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571580160 00000100:00100000:1.0:1788619924.425849:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571580160 00000100:00100000:1.0:1788619924.425958:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1250 00000100:00100000:1.0:1788619924.425968:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df96a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2353:x1875503571580160:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619924.425988:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619924.426008:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296c00/0x1cd7fe5368735a80 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ccd8 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.426053:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe5368735a3a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cca0 expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619924.426095:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f5500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589184640:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619924.426176:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7540 going to sleep for 11 seconds 00000100:00100000:3.0:1788619924.426684:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571580288 00000100:00100000:3.0:1788619924.426690:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571580288 00000100:00100000:3.0:1788619924.426735:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1016 00000100:00100000:3.0:1788619924.426744:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69f8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571580288:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619924.426753:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619924.426763:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a72c5200/0x1cd7fe5368735a87 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ccdf expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619924.426798:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69fca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589184768:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619924.426849:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c300 going to sleep for 11 seconds 00000100:00100000:3.0:1788619924.428826:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69fca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589184768:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619924.428860:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a72c5200/0x1cd7fe5368735a87 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ccdf expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.428883:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a72c5200/0x1cd7fe5368735a87 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ccdf expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.428896:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c400, rc 0) 00000100:00100000:3.0:1788619924.428961:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69f8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571580288:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2217us (3662us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619924.428969:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1016 00000100:00100000:1.0:1788619924.429527:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f5500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589184640:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619924.429542:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:1.0:1788619924.429553:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296c00/0x1cd7fe5368735a80 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ccd8 expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.429568:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619924.429571:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619924.429575:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296c00/0x1cd7fe5368735a80 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ccd8 expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.429586:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296c00, rc 0) 00000100:00100000:1.0:1788619924.429623:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df96a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2353:x1875503571580160:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3657us (4562us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619924.429629:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1250 00000100:00100000:0.0:1788619924.434254:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619924.434468:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571580416 00000100:00100000:1.0:1788619924.434477:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571580416 00000100:00100000:1.0:1788619924.434565:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1251 00000100:00100000:1.0:1788619924.434575:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2352:x1875503571580416:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619924.434595:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619924.434613:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe5368735a8e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cce6 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.434649:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe5368735a3a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cca0 expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619924.434701:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f5500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589184896:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619924.434794:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7540 going to sleep for 11 seconds 00000100:00100000:3.0:1788619924.437419:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619924.437540:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571580544 00000100:00100000:1.0:1788619924.437548:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571580544 00000100:00100000:1.0:1788619924.437594:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1017 00000100:00100000:1.0:1788619924.437603:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571580544:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619924.437618:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619924.437634:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098665f62/0x1cd7fe5368735a95 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cced expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619924.437699:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df8b2300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589185024:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619924.437772:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7600 going to sleep for 11 seconds 00000100:00100000:1.0:1788619924.441010:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f5500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589184896:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619924.441049:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:1.0:1788619924.441070:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe5368735a8e lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cce6 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.441090:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619924.441093:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619924.441099:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe5368735a8e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cce6 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.441120:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297600, rc 0) 00000100:00100000:1.0:1788619924.441185:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2352:x1875503571580416:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6611us (6942us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619924.441195:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1251 00000100:00100000:1.0:1788619924.442186:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df8b2300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589185024:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619924.442212:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000098665f62/0x1cd7fe5368735a95 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cced expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.442232:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098665f62/0x1cd7fe5368735a95 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cced expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.442249:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297800, rc 0) 00000100:00100000:1.0:1788619924.442284:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571580544:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4681us (4874us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619924.442290:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1017 00000100:00100000:3.0:1788619924.447525:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619924.447789:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571580672 00000100:00100000:0.0:1788619924.447799:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571580672 00000100:00100000:0.0:1788619924.447860:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1018 00000100:00100000:0.0:1788619924.447872:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:14908:x1875503571580672:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619924.447891:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619924.447909:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003fff4610/0x1cd7fe5368735a9c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ccf4 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.447939:0:6698:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe5368735a41 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cca7 expref: 5 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.447953:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=1s v=(3 13) 00010000:00010000:0.0:1788619924.447957:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(4 13) 00010000:00010000:0.0:1788619924.447962:0:6698:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003fff4610/0x1cd7fe5368735a9c lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [2015232->2031615] (req 2015232->2031615) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ccf4 expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.447996:0:6698:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe5368735a41 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cca7 expref: 5 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.448011:0:6698:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe5368735a41 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cca7 expref: 5 pid: 6696 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619924.448025:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f55880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589185152:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619924.448222:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a240 going to sleep for 11 seconds 00000100:00100000:0.0:1788619924.452914:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619924.453005:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571580800 00000100:00100000:2.0:1788619924.453014:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571580800 00010000:00010000:2.0:1788619924.453063:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe5368735a41 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cca7 expref: 6 pid: 6696 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619924.453110:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 419 00000100:00100000:2.0:1788619924.453121:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6858380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571580800:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619924.453136:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1788619924.453149:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f55880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589185152:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619924.453156:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe5368735a41 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cca7 expref: 6 pid: 6696 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.453169:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe5368735a41 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cca7 expref: 6 pid: 6696 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.453769:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003fff4610/0x1cd7fe5368735a9c lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [2015232->2031615] (req 2015232->2031615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ccf4 expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.453803:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f51200, rc 0) 00000100:00100000:3.0:1788619924.454132:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:14908:x1875503571580672:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6261us (6615us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619924.454141:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1018 00010000:00010000:2.0:1788619924.455513:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe5368735a41 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cca7 expref: 6 pid: 6696 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.455535:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(4 13) 00010000:00010000:2.0:1788619924.455539:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(4 13) 00010000:00010000:2.0:1788619924.455547:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000003fff4610/0x1cd7fe5368735a9c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2015232->2031615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ccf4 expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.455554:0:5829:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003fff4610/0x1cd7fe5368735a9c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2015232->2031615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ccf4 expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.455588:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000003fff4610/0x1cd7fe5368735a9c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2015232->2031615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ccf4 expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619924.455599:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5026d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1875503589185280:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619924.455672:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164240 going to sleep for 11 seconds 00000100:00100000:0.0:1788619924.460576:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619924.460705:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571580928 00000100:00100000:3.0:1788619924.460712:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571580928 00000100:00100000:2.0:1788619924.460717:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5026d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1875503589185280:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619924.460740:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619924.460788:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6858380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571580800:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 7668us (7884us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619924.460799:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 419 00000100:00100000:3.0:1788619924.460880:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 592 00000100:00100000:3.0:1788619924.460891:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5d5ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571580928:12345-192.168.201.5@tcp:22:fsx.0 00002000:00010000:3.0:1788619924.461715:0:6702:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 2028302 -> 2028302 00002000:00010000:3.0:1788619924.461720:0:6702:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619924 -> 1788619924 00002000:00010000:3.0:1788619924.461723:0:6702:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619924 -> 1788619924 00002000:00010000:3.0:1788619924.461725:0:6702:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619924 -> 1788619924 00002000:00010000:3.0:1788619924.461727:0:6702:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 391 -> 391 00000100:00100000:3.0:1788619924.461761:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5d5ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571580928:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 872us (1196us total) trans 4294967469 rc 0/0 00000100:00100000:3.0:1788619924.461769:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 592 00000100:00100000:0.0:1788619924.465380:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619924.465474:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571581056 00000100:00100000:2.0:1788619924.465549:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571581056 00000100:00100000:2.0:1788619924.465610:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1252 00000100:00100000:2.0:1788619924.465620:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df965f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2353:x1875503571581056:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619924.465639:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619924.465668:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe5368735aa3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ccfb expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.465706:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe5368735a3a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cca0 expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619924.465745:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5026d80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589185408:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619924.466095:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164240 going to sleep for 11 seconds 00000100:00100000:2.0:1788619924.468573:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5026d80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589185408:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619924.468600:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:2.0:1788619924.468617:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe5368735aa3 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ccfb expref: 15 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.468639:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619924.468642:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619924.468649:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe5368735aa3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ccfb expref: 15 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.468679:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6000, rc 0) 00000100:00100000:2.0:1788619924.468771:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df965f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2353:x1875503571581056:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3151us (3400us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619924.468779:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1252 00000100:00100000:0.0:1788619924.472592:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619924.472702:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571581184 00000100:00100000:1.0:1788619924.472712:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571581184 00000100:00100000:1.0:1788619924.472808:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1253 00000100:00100000:1.0:1788619924.472819:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64ab800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2353:x1875503571581184:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619924.472838:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619924.472857:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297800/0x1cd7fe5368735aaa lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cd02 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.472894:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe5368735a3a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cca0 expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619924.473150:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df8b2300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589185536:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619924.473229:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7600 going to sleep for 11 seconds 00000100:00100000:1.0:1788619924.475558:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df8b2300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589185536:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619924.475585:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:1.0:1788619924.475603:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297800/0x1cd7fe5368735aaa lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd02 expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.475625:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619924.475628:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619924.475633:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297800/0x1cd7fe5368735aaa lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd02 expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.475647:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297800, rc 0) 00000100:00100000:1.0:1788619924.475703:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64ab800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2353:x1875503571581184:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2886us (3120us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619924.475711:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1253 00000100:00100000:0.0:1788619924.479151:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619924.479286:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571581312 00000100:00100000:1.0:1788619924.479295:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571581312 00000100:00100000:1.0:1788619924.479356:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1254 00000100:00100000:1.0:1788619924.479365:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64a8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571581312:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619924.479382:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619924.479400:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe5368735ab1 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cd09 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.479430:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe5368735a3a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cca0 expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.479442:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe5368735ab1 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd09 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.479472:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe5368735a3a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cca0 expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.479573:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe5368735a3a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cca0 expref: 11 pid: 5846 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619924.479588:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df8b2300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589185664:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619924.480666:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7600 going to sleep for 11 seconds 00000100:00100000:0.0:1788619924.485567:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619924.489589:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df8b2300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589185664:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619924.489625:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe5368735ab1 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd09 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.489643:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619924.489647:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619924.489664:0:5846:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe5368735ab1 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd09 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.489677:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297600, rc 0) 00000100:00100000:1.0:1788619924.490667:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64a8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:14908:x1875503571581312:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 11287us (11508us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619924.490678:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1254 00000100:00100000:2.0:1788619924.492597:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571581440 00000100:00100000:2.0:1788619924.492611:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571581440 00010000:00010000:2.0:1788619924.492668:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe5368735a3a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cca0 expref: 12 pid: 5846 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619924.492702:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 420 00000100:00100000:2.0:1788619924.493076:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64aad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571581440:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619924.493090:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619924.493117:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe5368735a3a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cca0 expref: 12 pid: 5846 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.493131:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe5368735a3a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cca0 expref: 12 pid: 5846 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.493154:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe5368735a3a lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cca0 expref: 12 pid: 5846 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.493172:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619924.493177:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe5368735ab1 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd09 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.493195:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe5368735ab1 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd09 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.493205:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe5368735ab1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd09 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.493244:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe5368735ab1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd09 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619924.493258:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5026d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589185792:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619924.493375:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164240 going to sleep for 11 seconds 00000100:00100000:0.0:1788619924.497144:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619924.497435:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5026d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589185792:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619924.497460:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619924.497517:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64aad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571581440:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 4447us (11958us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619924.497528:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 420 00000100:00100000:2.0:1788619924.498857:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571581568 00000100:00100000:2.0:1788619924.498868:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571581568 00000100:00100000:2.0:1788619924.499144:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 762 00000100:00100000:2.0:1788619924.499156:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64aa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571581568:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:2.0:1788619924.499488:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 987136 -> 987136 00000004:00010000:2.0:1788619924.499493:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619924 -> 1788619924 00000004:00010000:2.0:1788619924.499496:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619924 -> 1788619924 00000004:00010000:2.0:1788619924.499498:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619924 -> 1788619924 00000004:00010000:2.0:1788619924.499500:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 902 -> 902 00000100:00100000:2.0:1788619924.499566:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64aa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571581568:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 412us (2429us total) trans 4294967747 rc 0/0 00000100:00100000:2.0:1788619924.505550:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 762 00000100:00100000:0.0:1788619924.506963:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619924.506999:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619924.507126:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571581696 00000100:00100000:2.0:1788619924.507136:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571581696 00000100:00100000:2.0:1788619924.507186:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1255 00000100:00100000:2.0:1788619924.507196:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64a9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+11:2352:x1875503571581696:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619924.507215:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619924.507232:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe5368735ab8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cd10 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.507268:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe5368735ab1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd09 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619924.507304:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5026d80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589185920:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619924.507395:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164d80 going to sleep for 11 seconds 00000100:00100000:3.0:1788619924.509937:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571581824 00000100:00100000:3.0:1788619924.509949:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571581824 00000100:00100000:3.0:1788619924.510019:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1019 00000100:00100000:3.0:1788619924.510029:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64ab100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571581824:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619924.510048:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619924.510066:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a72c5200/0x1cd7fe5368735abf lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cd17 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619924.510128:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69fe300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589186048:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619924.510211:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c300 going to sleep for 11 seconds 00000100:00100000:3.0:1788619924.513468:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69fe300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589186048:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619924.513515:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a72c5200/0x1cd7fe5368735abf lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd17 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.513539:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a72c5200/0x1cd7fe5368735abf lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd17 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.513555:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c400, rc 0) 00000100:00100000:3.0:1788619924.513615:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64ab100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571581824:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3588us (6614us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619924.513625:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1019 00000100:00100000:2.0:1788619924.515734:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5026d80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589185920:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619924.515760:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:2.0:1788619924.515777:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe5368735ab8 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd10 expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.515931:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619924.515935:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619924.516038:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe5368735ab8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd10 expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.516055:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6000, rc 0) 00000100:00100000:2.0:1788619924.516124:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64a9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+11:2352:x1875503571581696:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8928us (9166us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619924.516134:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1255 00000100:00100000:0.0:1788619924.518317:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619924.518699:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619924.518876:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571582080 00000100:00100000:3.0:1788619924.518885:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571582080 00000100:00100000:3.0:1788619924.518949:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1020 00000100:00100000:3.0:1788619924.518960:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64a8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571582080:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619924.518978:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619924.518995:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad29727f/0x1cd7fe5368735ac6 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cd25 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619924.519052:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69fe300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589186176:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619924.519140:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c300 going to sleep for 11 seconds 00000100:00100000:2.0:1788619924.525323:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69fe300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589186176:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619924.525379:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ad29727f/0x1cd7fe5368735ac6 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd25 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.525406:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad29727f/0x1cd7fe5368735ac6 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd25 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.525420:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c000, rc 0) 00000100:00100000:2.0:1788619924.525779:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64a8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571582080:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6532us (6791us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619924.525856:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1020 00000100:00100000:1.0:1788619924.530601:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571581952 00000100:00100000:1.0:1788619924.530617:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571581952 00000100:00100000:1.0:1788619924.530692:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1256 00000100:00100000:1.0:1788619924.531050:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64a8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571581952:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619924.531074:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619924.531100:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296a00/0x1cd7fe5368735acd lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cd1e expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.531139:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe5368735ab1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd09 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619924.531184:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df8b2300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589186304:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619924.531306:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7600 going to sleep for 11 seconds 00000100:00100000:1.0:1788619924.533661:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df8b2300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589186304:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619924.533686:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:1.0:1788619924.533703:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296a00/0x1cd7fe5368735acd lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd1e expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.533907:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619924.533910:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619924.533917:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296a00/0x1cd7fe5368735acd lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd1e expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.534066:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296a00, rc 0) 00000100:00100000:1.0:1788619924.534437:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64a8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571581952:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3734us (16126us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619924.534447:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1256 00000100:00100000:2.0:1788619924.544521:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619924.544693:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571582336 00000100:00100000:3.0:1788619924.544703:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571582336 00000100:00100000:3.0:1788619924.544774:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1257 00000100:00100000:3.0:1788619924.544786:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5026d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:14908:x1875503571582336:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:3.0:1788619924.544842:0:5846:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619924.544870:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368735a4f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ccae expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.544889:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619924.544895:0:5846:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788619924.545117:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368735ad4 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.545132:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368735a79 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ccd1 expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.545142:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368735ad4 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.545174:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368735a79 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ccd1 expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.545188:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368735a79 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ccd1 expref: 10 pid: 5844 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619924.545201:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69fdf80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589186432:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619924.545279:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48ce40 going to sleep for 11 seconds 00000100:00100000:3.0:1788619924.547302:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69fdf80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589186432:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619924.547365:0:5846:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368735ad4 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619924.548242:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619924.548772:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571582464 00000100:00100000:3.0:1788619924.548779:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571582464 00010000:00010000:3.0:1788619924.548815:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368735a79 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ccd1 expref: 11 pid: 5844 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619924.548856:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 421 00000100:00100000:3.0:1788619924.548865:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64ac380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571582464:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619924.548878:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619924.548884:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368735a79 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ccd1 expref: 11 pid: 5844 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.548898:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368735a79 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ccd1 expref: 11 pid: 5844 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.548927:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368735a79 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ccd1 expref: 11 pid: 5844 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.549164:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619924.549170:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368735ad4 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.549182:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368735ad4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.549190:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368735ad4 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.549225:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619924.549279:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64ac380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571582464:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 414us (1039us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619924.549287:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 421 00010000:00010000:0.0:1788619924.549359:0:5846:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368735ad4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.549368:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368735ad4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788619924.549486:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967748 00000004:00010000:0.0:1788619924.549540:0:5846:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 987136 -> 987136 00000004:00010000:0.0:1788619924.549544:0:5846:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619924 -> 1788619924 00000004:00010000:0.0:1788619924.549546:0:5846:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619924 -> 1788619924 00000004:00010000:0.0:1788619924.549549:0:5846:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619924 -> 1788619924 00000004:00010000:0.0:1788619924.549551:0:5846:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 902 -> 902 00000004:00010000:0.0:1788619924.549558:0:5846:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000c7e6eda4 reply state 00000000d7f90fc5 transno 4294967749 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368735ad4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619924.549566:0:5846:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788619924.549631:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5026d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+18:14908:x1875503571582336:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 4848us (5121us total) trans 4294967749 rc 0/0 00000100:00100000:0.0:1788619924.549639:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1257 00000020:00080000:0.0:1788619924.549643:0:5846:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788619924.551148:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368735ad4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.551163:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368735ad4 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.551171:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368735ad4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.551177:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c683c400 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788619924.551182:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368735ad4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.551193:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368735ad4 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.551209:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619924.551211:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788619924.551216:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368735ad4 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.551223:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368735ad4 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619924.552858:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619924.552935:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571582592 00000100:00100000:0.0:1788619924.552952:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571582592 00002000:00010000:0.0:1788619924.552994:0:6701:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1610010->18446744073709551615) 00002000:00100000:0.0:1788619924.552999:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64ad880 with x1875503571582592 ext(1610010->18446744073709551615) 00002000:00010000:0.0:1788619924.553013:0:6701:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64ad880. 00000100:00100000:0.0:1788619924.553047:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 593 00000100:00100000:0.0:1788619924.553057:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64ad880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571582592:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:0.0:1788619924.553405:0:6701:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 2028302 -> 1610010 00002000:00010000:0.0:1788619924.553409:0:6701:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619924 -> 1788619924 00002000:00010000:0.0:1788619924.553412:0:6701:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619924 -> 1788619924 00002000:00010000:0.0:1788619924.553414:0:6701:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619924 -> 1788619924 00002000:00010000:0.0:1788619924.553416:0:6701:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 391 -> 391 00000100:00100000:0.0:1788619924.553484:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64ad880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571582592:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 418us (625us total) trans 4294967470 rc 0/0 00000100:00100000:0.0:1788619924.553810:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 593 00002000:00010000:0.0:1788619924.553815:0:6701:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1610010->18446744073709551615) 00002000:00100000:0.0:1788619924.553819:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64ad880 with x1875503571582592 ext(1610010->18446744073709551615) 00002000:00010000:0.0:1788619924.553824:0:6701:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64ad880. 00000100:00100000:2.0:1788619924.556057:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619924.556135:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571582720 00000100:00100000:3.0:1788619924.556143:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571582720 00000100:00100000:3.0:1788619924.556215:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1258 00000100:00100000:3.0:1788619924.556224:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64aea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+17:14908:x1875503571582720:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619924.556247:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619924.556264:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368735adb lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cd2c expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.556371:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368735ae2 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.556386:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368735ae2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.556397:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368735ae2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788619924.556425:0:11397:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1610010 00000004:00200000:3.0:1788619924.556430:0:11397:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788619924.556446:0:11397:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368735ae2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.556471:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368735adb lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd2c expref: 19 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.556508:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368735ae2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd2c expref: 18 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.556517:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619924.556521:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788619924.556524:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d600, rc 0) 00000100:00100000:3.0:1788619924.556579:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64aea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+18:14908:x1875503571582720:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 357us (532us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619924.556586:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1258 00000100:00100000:2.0:1788619924.563142:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619924.563221:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571582848 00000100:00100000:1.0:1788619924.563231:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571582848 00000100:00100000:1.0:1788619924.563289:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1259 00000100:00100000:1.0:1788619924.563301:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64afb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:14908:x1875503571582848:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619924.563349:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619924.563371:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297c00/0x1cd7fe5368735ae9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cd33 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.563460:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297400/0x1cd7fe5368735af0 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.563486:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297400/0x1cd7fe5368735af0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.563506:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297400/0x1cd7fe5368735af0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788619924.563536:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1610010 00000004:00200000:1.0:1788619924.563541:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788619924.563560:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297400/0x1cd7fe5368735af0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.563587:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297c00/0x1cd7fe5368735ae9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd33 expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.563612:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297400/0x1cd7fe5368735af0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd33 expref: 11 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.563622:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619924.563625:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788619924.563629:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297400, rc 0) 00000100:00100000:1.0:1788619924.563697:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64afb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571582848:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 399us (564us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619924.563704:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1259 00000100:00100000:1.0:1788619924.567048:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619924.567098:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619924.567170:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571583104 00000100:00100000:2.0:1788619924.567179:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571583104 00000100:00100000:2.0:1788619924.567236:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1021 00000100:00100000:2.0:1788619924.567246:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571583104:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619924.567266:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619924.567287:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1c87370/0x1cd7fe5368735af7 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cd41 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619924.567374:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64ae680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589186560:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619924.567449:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164d80 going to sleep for 11 seconds 00000100:00100000:3.0:1788619924.567841:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571582976 00000100:00100000:3.0:1788619924.567847:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571582976 00000100:00100000:3.0:1788619924.567878:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1260 00000100:00100000:3.0:1788619924.567886:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571582976:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619924.567897:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619924.567910:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368735afe lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cd3a expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.568004:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe5368735ab1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd09 expref: 17 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619924.568035:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69fdf80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589186688:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619924.568161:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48ce40 going to sleep for 11 seconds 00000100:00100000:1.0:1788619924.570222:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64ae680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589186560:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619924.570261:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b1c87370/0x1cd7fe5368735af7 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd41 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.570291:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1c87370/0x1cd7fe5368735af7 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd41 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.570304:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6200, rc 0) 00000100:00100000:1.0:1788619924.570393:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571583104:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3148us (3293us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619924.570402:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1021 00000100:00100000:3.0:1788619924.571768:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69fdf80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589186688:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619924.571790:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:3.0:1788619924.571806:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368735afe lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd3a expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.571821:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619924.571824:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619924.571829:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368735afe lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd3a expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.571840:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d400, rc 0) 00000100:00100000:3.0:1788619924.572095:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571582976:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4208us (5055us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619924.572112:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1260 00000100:00100000:3.0:1788619924.574923:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619924.575179:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619924.575263:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571583360 00000100:00100000:0.0:1788619924.575272:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571583360 00000100:00100000:0.0:1788619924.575400:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1022 00000100:00100000:0.0:1788619924.575410:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69ff100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571583360:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619924.575428:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619924.575445:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000de364e00/0x1cd7fe5368735b05 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cd4f expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619924.575515:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64a8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589186816:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619924.575590:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a240 going to sleep for 11 seconds 00000100:00100000:2.0:1788619924.576348:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571583232 00000100:00100000:2.0:1788619924.576356:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571583232 00000100:00100000:2.0:1788619924.576390:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1261 00000100:00100000:2.0:1788619924.576397:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+11:2352:x1875503571583232:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619924.576411:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619924.576424:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe5368735b0c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cd48 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.576452:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe5368735ab1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd09 expref: 17 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619924.576647:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64adc00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589186944:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619924.576703:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164d80 going to sleep for 11 seconds 00000100:00100000:3.0:1788619924.578723:0:3316:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1788619924.578741:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69fc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589187072:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1788619924.578798:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619924.578839:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589187072 00000100:00100000:3.0:1788619924.578846:0:6699:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589187072 00000100:00100000:3.0:1788619924.578888:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 90 00000100:00100000:3.0:1788619924.578897:0:6699:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503589187072:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1788619924.579125:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3856662528 avail 3854565376 00000100:00100000:3.0:1788619924.579174:0:6699:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503589187072:12345-0@lo:13:osp-pre-1-0.0 Request processed in 278us (375us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619924.579181:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 90 00000100:00100000:3.0:1788619924.579230:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69fc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589187072:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1788619924.583184:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64a8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589186816:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619924.583220:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000de364e00/0x1cd7fe5368735b05 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd4f expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.583244:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000de364e00/0x1cd7fe5368735b05 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd4f expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.583256:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f51800, rc 0) 00000100:00100000:0.0:1788619924.583309:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69ff100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571583360:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7901us (8129us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619924.583318:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1022 00000100:00100000:3.0:1788619924.584201:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64adc00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589186944:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619924.584221:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:3.0:1788619924.584235:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe5368735b0c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd48 expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.584250:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619924.584253:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619924.584258:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6a00/0x1cd7fe5368735b0c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd48 expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.584271:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6a00, rc 0) 00000100:00100000:3.0:1788619924.584315:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+11:2352:x1875503571583232:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7918us (9409us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619924.584322:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1261 00000100:00100000:2.0:1788619924.588441:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619924.588715:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571583616 00000100:00100000:0.0:1788619924.588725:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571583616 00000100:00100000:0.0:1788619924.588787:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1262 00000100:00100000:0.0:1788619924.588798:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64ace00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571583616:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:0.0:1788619924.588854:0:5844:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619924.588881:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297400/0x1cd7fe5368735af0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd33 expref: 11 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.588900:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619924.588923:0:5844:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788619924.589288:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51600/0x1cd7fe5368735b13 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.589303:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368735ae2 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd2c expref: 17 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.589311:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51600/0x1cd7fe5368735b13 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.589373:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368735ae2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd2c expref: 17 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.589387:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368735ae2 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd2c expref: 17 pid: 11397 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619924.589400:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64a8380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589187200:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619924.589713:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c133a240 going to sleep for 11 seconds 00000100:00100000:1.0:1788619924.596178:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619924.596506:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571583744 00000100:00100000:0.0:1788619924.596516:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571583744 00010000:00010000:0.0:1788619924.596556:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368735ae2 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd2c expref: 18 pid: 11397 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619924.596603:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 422 00000100:00100000:0.0:1788619924.596613:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64d3480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2354:x1875503571583744:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619924.596627:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619924.596634:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368735ae2 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd2c expref: 18 pid: 11397 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.596648:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368735ae2 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd2c expref: 18 pid: 11397 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.596683:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619924.596689:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51600/0x1cd7fe5368735b13 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.596700:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51600/0x1cd7fe5368735b13 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.596716:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51600/0x1cd7fe5368735b13 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.596736:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619924.596790:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64d3480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2354:x1875503571583744:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 178us (621us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619924.596797:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 422 00010000:00010000:2.0:1788619924.598596:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368735ae2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd2c expref: 17 pid: 11397 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619924.598618:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64a8380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589187200:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619924.598641:0:5844:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51600/0x1cd7fe5368735b13 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.598661:0:5844:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51600/0x1cd7fe5368735b13 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.598667:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51600/0x1cd7fe5368735b13 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788619924.598826:0:5844:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967750 00000004:00010000:2.0:1788619924.598870:0:5844:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 987136 -> 987136 00000004:00010000:2.0:1788619924.598873:0:5844:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619924 -> 1788619924 00000004:00010000:2.0:1788619924.598876:0:5844:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619924 -> 1788619924 00000004:00010000:2.0:1788619924.598879:0:5844:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619924 -> 1788619924 00000004:00010000:2.0:1788619924.598881:0:5844:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 902 -> 902 00000004:00010000:2.0:1788619924.598887:0:5844:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000059d57522 reply state 0000000069686a0e transno 4294967751 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51600/0x1cd7fe5368735b13 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619924.598896:0:5844:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788619924.599217:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64ace00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+13:14908:x1875503571583616:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 10420us (10783us total) trans 4294967751 rc 0/0 00000100:00100000:2.0:1788619924.599225:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1262 00000020:00080000:2.0:1788619924.599230:0:5844:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788619924.603648:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51600/0x1cd7fe5368735b13 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.603675:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51600/0x1cd7fe5368735b13 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.603683:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51600/0x1cd7fe5368735b13 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.603690:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c5f51600 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788619924.603695:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51600/0x1cd7fe5368735b13 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.603703:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51600/0x1cd7fe5368735b13 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.603730:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619924.603734:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788619924.603739:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51600/0x1cd7fe5368735b13 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.603746:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51600/0x1cd7fe5368735b13 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619924.603944:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619924.604069:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571583872 00000100:00100000:0.0:1788619924.604078:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571583872 00000100:00100000:0.0:1788619924.604131:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1263 00000100:00100000:0.0:1788619924.604141:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64d1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:14908:x1875503571583872:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619924.604163:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619924.604179:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51400/0x1cd7fe5368735b1a lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cd56 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.604204:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe5368735ab1 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd09 expref: 16 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.604219:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51400/0x1cd7fe5368735b1a lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd56 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.604253:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe5368735ab1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd09 expref: 16 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.604267:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe5368735ab1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd09 expref: 16 pid: 5846 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619924.604281:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64a9880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589187328:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619924.604370:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78e03f5600 going to sleep for 11 seconds 00000100:00100000:0.0:1788619924.608535:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64a9880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589187328:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619924.608573:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51400/0x1cd7fe5368735b1a lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd56 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.608593:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619924.608596:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619924.608601:0:5846:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51400/0x1cd7fe5368735b1a lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd56 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.608611:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f51400, rc 0) 00000100:00100000:0.0:1788619924.608669:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64d1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+13:14908:x1875503571583872:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4530us (4736us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619924.608678:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1263 00000100:00100000:1.0:1788619924.609099:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619924.609147:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571584000 00000100:00100000:3.0:1788619924.609154:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571584000 00010000:00010000:3.0:1788619924.609185:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe5368735ab1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd09 expref: 17 pid: 5846 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619924.609220:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 423 00000100:00100000:3.0:1788619924.609229:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64d3800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2354:x1875503571584000:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619924.609242:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619924.609304:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe5368735ab1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd09 expref: 17 pid: 5846 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.609317:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe5368735ab1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd09 expref: 17 pid: 5846 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.609355:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe5368735ab1 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd09 expref: 17 pid: 5846 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.609372:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619924.609377:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51400/0x1cd7fe5368735b1a lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd56 expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.609387:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51400/0x1cd7fe5368735b1a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd56 expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.609396:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51400/0x1cd7fe5368735b1a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd56 expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.609463:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51400/0x1cd7fe5368735b1a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd56 expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619924.609478:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64a9880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589187456:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619924.609604:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78e03f5600 going to sleep for 11 seconds 00000100:00100000:0.0:1788619924.611431:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64a9880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589187456:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619924.611454:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619924.611613:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64d3800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571584000:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 2382us (2677us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619924.611623:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 423 00000100:00100000:1.0:1788619924.613959:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619924.614034:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571584128 00000100:00100000:3.0:1788619924.614043:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571584128 00000100:00100000:3.0:1788619924.614099:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1023 00000100:00100000:3.0:1788619924.614109:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64d0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:14908:x1875503571584128:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619924.614128:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619924.614145:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007736e663/0x1cd7fe5368735b21 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cd5d expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.614222:0:7654:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000003fff4610/0x1cd7fe5368735a9c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2015232->2031615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ccf4 expref: 7 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.614232:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(4 13) 00010000:00010000:3.0:1788619924.614236:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(5 13) 00010000:00010000:3.0:1788619924.614242:0:7654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007736e663/0x1cd7fe5368735b21 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd5d expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.614283:0:7654:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000003fff4610/0x1cd7fe5368735a9c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2015232->2031615) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ccf4 expref: 7 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.614298:0:7654:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000003fff4610/0x1cd7fe5368735a9c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2015232->2031615) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ccf4 expref: 7 pid: 6698 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619924.614312:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69fc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589187584:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619924.614421:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48ce40 going to sleep for 11 seconds 00000100:00100000:2.0:1788619924.617185:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619924.617271:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571584256 00000100:00100000:3.0:1788619924.617277:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69fc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589187584:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619924.617278:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571584256 00010000:00010000:3.0:1788619924.617310:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007736e663/0x1cd7fe5368735b21 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd5d expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.617312:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000003fff4610/0x1cd7fe5368735a9c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2015232->2031615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ccf4 expref: 8 pid: 6698 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.617367:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d400, rc 0) 00000100:00100000:0.0:1788619924.617370:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 424 00000100:00100000:0.0:1788619924.617378:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64ad500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571584256:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619924.617402:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1788619924.617415:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64d0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:14908:x1875503571584128:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3309us (3464us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619924.617421:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1023 00010000:00010000:0.0:1788619924.617424:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000003fff4610/0x1cd7fe5368735a9c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2015232->2031615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ccf4 expref: 8 pid: 6698 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.617436:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000003fff4610/0x1cd7fe5368735a9c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2015232->2031615) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ccf4 expref: 8 pid: 6698 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.617457:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003fff4610/0x1cd7fe5368735a9c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2015232->2031615) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ccf4 expref: 8 pid: 6698 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.617475:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(5 13) 00010000:00010000:0.0:1788619924.617479:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(5 13) 00010000:00010000:0.0:1788619924.617495:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000007736e663/0x1cd7fe5368735b21 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd5d expref: 5 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.617506:0:13330:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007736e663/0x1cd7fe5368735b21 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd5d expref: 5 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.617539:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000007736e663/0x1cd7fe5368735b21 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd5d expref: 5 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619924.617552:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64a9880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589187712:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619924.617617:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78e03f5600 going to sleep for 11 seconds 00000100:00100000:0.0:1788619924.620105:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64a9880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589187712:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619924.620130:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619924.620184:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64ad500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571584256:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 2805us (3007us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619924.620194:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 424 00000100:00100000:3.0:1788619924.621892:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619924.622169:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571584384 00000100:00100000:1.0:1788619924.622176:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571584384 00002000:00010000:1.0:1788619924.622207:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (2092012->18446744073709551615) 00002000:00100000:1.0:1788619924.622211:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c69fc700 with x1875503571584384 ext(2092012->18446744073709551615) 00002000:00010000:1.0:1788619924.622219:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c69fc700. 00000100:00100000:1.0:1788619924.622246:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 594 00000100:00100000:1.0:1788619924.622252:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571584384:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:1.0:1788619924.622517:0:6702:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1610010 -> 2092012 00002000:00010000:1.0:1788619924.622521:0:6702:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619924 -> 1788619924 00002000:00010000:1.0:1788619924.622524:0:6702:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619924 -> 1788619924 00002000:00010000:1.0:1788619924.622526:0:6702:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619924 -> 1788619924 00002000:00010000:1.0:1788619924.622528:0:6702:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 391 -> 391 00000100:00100000:1.0:1788619924.622570:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2353:x1875503571584384:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 317us (683us total) trans 4294967471 rc 0/0 00000100:00100000:1.0:1788619924.622577:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 594 00002000:00010000:1.0:1788619924.622580:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (2092012->18446744073709551615) 00002000:00100000:1.0:1788619924.622583:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c69fc700 with x1875503571584384 ext(2092012->18446744073709551615) 00002000:00010000:1.0:1788619924.622587:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c69fc700. 00000100:00100000:1.0:1788619924.624889:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619924.624994:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571584512 00000100:00100000:3.0:1788619924.625002:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571584512 00000100:00100000:3.0:1788619924.625094:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1264 00000100:00100000:3.0:1788619924.625104:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64d2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+16:14908:x1875503571584512:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619924.625123:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619924.625141:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368735b28 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cd64 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.625224:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368735b2f lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.625239:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368735b2f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.625252:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368735b2f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788619924.625277:0:11397:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 2092012 00000004:00200000:3.0:1788619924.625281:0:11397:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788619924.625299:0:11397:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368735b2f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.625359:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368735b28 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd64 expref: 18 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.625379:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368735b2f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd64 expref: 17 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.625388:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619924.625390:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788619924.625394:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683ce00, rc 0) 00000100:00100000:3.0:1788619924.625451:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64d2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+17:14908:x1875503571584512:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 348us (569us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619924.625458:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1264 00000100:00100000:2.0:1788619924.627854:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619924.627894:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619924.627963:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571584768 00000100:00100000:0.0:1788619924.627967:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571584640 00000100:00100000:3.0:1788619924.627970:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571584768 00000100:00100000:0.0:1788619924.627971:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571584640 00000100:00100000:3.0:1788619924.628023:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1024 00000100:00100000:0.0:1788619924.628023:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1265 00000100:00100000:3.0:1788619924.628033:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571584768:12345-192.168.201.5@tcp:101:fsx.0 00000100:00100000:0.0:1788619924.628033:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64ad180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2355:x1875503571584640:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619924.628049:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619924.628049:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619924.628063:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe5368735b3d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cd72 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.628064:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51200/0x1cd7fe5368735b36 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cd6b expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.628095:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51400/0x1cd7fe5368735b1a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd56 expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619924.628119:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69fc380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589187840:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619924.628119:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64a9880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589187968:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619924.628312:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78e03f5600 going to sleep for 11 seconds 00000100:00100000:3.0:1788619924.628441:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48ce40 going to sleep for 11 seconds 00000100:00100000:3.0:1788619924.630957:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69fc380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589187840:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619924.630973:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64a9880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589187968:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619924.630984:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:3.0:1788619924.630991:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe5368735b3d lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd72 expref: 8 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.630995:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51200/0x1cd7fe5368735b36 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd6b expref: 18 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.631012:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619924.631015:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe5368735b3d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd72 expref: 8 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.631015:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619924.631019:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51200/0x1cd7fe5368735b36 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd6b expref: 18 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.631027:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c600, rc 0) 00010000:00010000:0.0:1788619924.631029:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f51200, rc 0) 00000100:00100000:3.0:1788619924.631076:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571584768:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3044us (3230us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619924.631076:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64ad180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2355:x1875503571584640:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3044us (3180us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619924.631085:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1024 00000100:00100000:0.0:1788619924.631085:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1265 00000100:00100000:1.0:1788619924.633537:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619924.633594:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571584896 00000100:00100000:0.0:1788619924.633603:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571584896 00000100:00100000:0.0:1788619924.633668:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1266 00000100:00100000:0.0:1788619924.633678:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64d2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2354:x1875503571584896:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619924.633698:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619924.633728:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51600/0x1cd7fe5368735b44 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cd79 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.633764:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51400/0x1cd7fe5368735b1a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd56 expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619924.633803:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64a9880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589188096:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619924.633877:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78e03f5600 going to sleep for 11 seconds 00000100:00100000:2.0:1788619924.634382:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619924.634454:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571585024 00000100:00100000:1.0:1788619924.634459:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571585024 00000100:00100000:1.0:1788619924.634499:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1025 00000100:00100000:1.0:1788619924.634507:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64af800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571585024:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619924.634519:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619924.634531:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe5368735b4b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cd80 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619924.634567:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64d1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589188224:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619924.634616:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7600 going to sleep for 11 seconds 00000100:00100000:0.0:1788619924.636639:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64a9880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589188096:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619924.636672:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:0.0:1788619924.636690:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51600/0x1cd7fe5368735b44 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd79 expref: 18 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619924.636823:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64d1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589188224:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619924.636840:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe5368735b4b lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd80 expref: 8 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.636854:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe5368735b4b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd80 expref: 8 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.636863:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296e00, rc 0) 00000100:00100000:2.0:1788619924.636903:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64af800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571585024:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2397us (2522us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619924.636911:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1025 00010000:00010000:0.0:1788619924.637025:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619924.637029:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619924.637035:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51600/0x1cd7fe5368735b44 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd79 expref: 18 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.637046:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f51600, rc 0) 00000100:00100000:0.0:1788619924.637069:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64d2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2354:x1875503571584896:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3392us (3541us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619924.637076:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1266 00000100:00100000:3.0:1788619924.640676:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619924.640792:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571585152 00000100:00100000:2.0:1788619924.640801:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571585152 00000100:00100000:2.0:1788619924.640859:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1267 00000100:00100000:2.0:1788619924.640870:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2354:x1875503571585152:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619924.640889:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619924.640909:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe5368735b52 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cd87 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.641014:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51400/0x1cd7fe5368735b1a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd56 expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619924.641055:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64aca80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589188352:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619924.641183:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164540 going to sleep for 11 seconds 00000100:00100000:3.0:1788619924.645085:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619924.645267:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571585280 00000100:00100000:2.0:1788619924.645277:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571585280 00000100:00100000:2.0:1788619924.645353:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1026 00000100:00100000:2.0:1788619924.645364:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571585280:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619924.645382:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619924.645399:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fb640cb5/0x1cd7fe5368735b59 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cd8e expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619924.645463:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64aed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589188480:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619924.645660:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64aca80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589188352:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619924.645673:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164000 going to sleep for 11 seconds 00002000:00010000:3.0:1788619924.645678:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:3.0:1788619924.645693:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe5368735b52 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd87 expref: 18 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.645718:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619924.645721:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619924.645727:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe5368735b52 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd87 expref: 18 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.645740:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7000, rc 0) 00000100:00100000:3.0:1788619924.645766:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2354:x1875503571585152:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4899us (5100us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619924.645773:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1267 00000100:00100000:2.0:1788619924.650593:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64aed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589188480:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619924.650632:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000fb640cb5/0x1cd7fe5368735b59 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd8e expref: 8 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.650670:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fb640cb5/0x1cd7fe5368735b59 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd8e expref: 8 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.650687:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6e00, rc 0) 00000100:00100000:2.0:1788619924.650870:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571585280:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5506us (5795us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619924.650880:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1026 00000100:00100000:0.0:1788619924.661567:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619924.661610:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571585408 00000100:00100000:2.0:1788619924.661620:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571585408 00000100:00100000:2.0:1788619924.661690:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1268 00000100:00100000:2.0:1788619924.661700:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64a9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+17:14908:x1875503571585408:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619924.661728:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619924.661862:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe5368735b60 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cd95 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.661897:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51400/0x1cd7fe5368735b1a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd56 expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.661909:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe5368735b60 lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd95 expref: 18 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.662014:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51400/0x1cd7fe5368735b1a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd56 expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.662029:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51400/0x1cd7fe5368735b1a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd56 expref: 12 pid: 5846 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619924.662044:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64aed80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589188608:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619924.662123:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164000 going to sleep for 11 seconds 00000100:00100000:2.0:1788619924.664787:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64aed80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589188608:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619924.664821:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe5368735b60 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd95 expref: 18 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.664840:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619924.664843:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619924.664847:0:5844:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe5368735b60 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd95 expref: 18 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.664858:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7000, rc 0) 00000100:00100000:2.0:1788619924.664927:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64a9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+18:14908:x1875503571585408:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3228us (3370us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619924.665136:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1268 00000100:00100000:3.0:1788619924.665840:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619924.666080:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571585536 00000100:00100000:1.0:1788619924.666088:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571585536 00010000:00010000:1.0:1788619924.666123:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51400/0x1cd7fe5368735b1a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd56 expref: 13 pid: 5846 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619924.666159:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 425 00000100:00100000:1.0:1788619924.666169:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c855b050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571585536:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619924.666182:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619924.666204:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51400/0x1cd7fe5368735b1a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd56 expref: 13 pid: 5846 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.666217:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51400/0x1cd7fe5368735b1a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd56 expref: 13 pid: 5846 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.666238:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51400/0x1cd7fe5368735b1a lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd56 expref: 13 pid: 5846 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.666253:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619924.666258:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe5368735b60 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd95 expref: 17 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.666274:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe5368735b60 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd95 expref: 17 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.666285:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe5368735b60 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd95 expref: 17 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.666321:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe5368735b60 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd95 expref: 17 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619924.666371:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64d3100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589188736:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619924.666442:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7540 going to sleep for 11 seconds 00000100:00100000:1.0:1788619924.668449:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64d3100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589188736:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619924.668481:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619924.668600:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c855b050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571585536:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 2430us (2760us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619924.668610:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 425 00000100:00100000:3.0:1788619924.671461:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619924.671585:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571585664 00000100:00100000:1.0:1788619924.671594:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571585664 00000100:00100000:1.0:1788619924.671721:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1027 00000100:00100000:1.0:1788619924.671731:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571585664:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619924.672061:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619924.672079:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002abacdb6/0x1cd7fe5368735b67 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cd9c expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619924.672135:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64d3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589188864:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619924.672217:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7b40 going to sleep for 11 seconds 00000100:00100000:1.0:1788619924.678193:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64d3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589188864:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619924.678235:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002abacdb6/0x1cd7fe5368735b67 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd9c expref: 8 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.678260:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002abacdb6/0x1cd7fe5368735b67 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd9c expref: 8 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.678273:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296200, rc 0) 00000100:00100000:1.0:1788619924.678501:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571585664:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6770us (7046us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619924.678510:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1027 00000100:00100000:0.0:1788619924.690095:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619924.690670:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571585792 00000100:00100000:1.0:1788619924.690680:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571585792 00002000:00100000:1.0:1788619924.690861:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c64aaa00 x1875503571585792/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 504/0 e 0 to 0 dl 1788619935 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619924.690874:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64aaa00 with x1875503571585792 00000100:00100000:1.0:1788619924.691057:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 763 00000100:00100000:1.0:1788619924.691068:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64aaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2354:x1875503571585792:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619924.702479:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64aaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2354:x1875503571585792:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 11404us (12386us total) trans 0 rc 987136/987136 00000100:00100000:1.0:1788619924.702502:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 763 00002000:00100000:1.0:1788619924.702515:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c64aaa00 x1875503571585792/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 504/440 e 0 to 0 dl 1788619935 ref 1 fl Complete:/600/0 rc 987136/987136 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619924.702531:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64aaa00 with x1875503571585792 00000100:00100000:2.0:1788619924.705258:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619924.705353:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571585920 00000100:00100000:3.0:1788619924.705363:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571585920 00000100:00100000:3.0:1788619924.705457:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1028 00000100:00100000:3.0:1788619924.705468:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64aed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571585920:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619924.705495:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619924.705514:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001fc792ba/0x1cd7fe5368735b6e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cda3 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619924.705569:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69ffb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589188992:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619924.705658:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48ce40 going to sleep for 11 seconds 00000100:00100000:2.0:1788619924.707388:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69ffb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589188992:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619924.707423:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001fc792ba/0x1cd7fe5368735b6e lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cda3 expref: 8 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.707446:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001fc792ba/0x1cd7fe5368735b6e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cda3 expref: 8 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.707470:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d800, rc 0) 00000100:00100000:2.0:1788619924.707588:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64aed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571585920:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2121us (2337us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619924.707596:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1028 00000100:00100000:0.0:1788619924.709848:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619924.709929:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571586048 00000100:00100000:2.0:1788619924.709939:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571586048 00002000:00100000:2.0:1788619924.709991:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c64abb80 x1875503571586048/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/0 e 0 to 0 dl 1788619935 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619924.710002:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64abb80 with x1875503571586048 00000100:00100000:2.0:1788619924.710037:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 764 00000100:00100000:2.0:1788619924.710047:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64abb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2355:x1875503571586048:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619924.710272:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64abb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2355:x1875503571586048:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 226us (432us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619924.710280:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 764 00002000:00100000:2.0:1788619924.710287:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c64abb80 x1875503571586048/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/440 e 0 to 0 dl 1788619935 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619924.710295:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64abb80 with x1875503571586048 00000100:00100000:0.0:1788619924.713464:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619924.713592:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571586176 00000100:00100000:2.0:1788619924.713603:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571586176 00000100:00100000:2.0:1788619924.713721:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1029 00000100:00100000:2.0:1788619924.713732:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe13b050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571586176:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619924.713750:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619924.713767:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3b293bd/0x1cd7fe5368735b75 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cdaa expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619924.713826:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64ae300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589189120:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619924.713923:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164000 going to sleep for 11 seconds 00000100:00100000:2.0:1788619924.716216:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64ae300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589189120:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619924.716250:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f3b293bd/0x1cd7fe5368735b75 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cdaa expref: 8 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.716274:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3b293bd/0x1cd7fe5368735b75 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cdaa expref: 8 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.716286:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7e00, rc 0) 00000100:00100000:2.0:1788619924.716374:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe13b050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571586176:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2644us (2920us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619924.716382:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1029 00000100:00100000:0.0:1788619924.719249:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619924.719419:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571586304 00000100:00100000:1.0:1788619924.719427:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571586304 00002000:00100000:1.0:1788619924.719470:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c64aa680 x1875503571586304/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/0 e 0 to 0 dl 1788619935 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619924.719538:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64aa680 with x1875503571586304 00000100:00100000:1.0:1788619924.719592:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 765 00000100:00100000:1.0:1788619924.719601:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64aa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2355:x1875503571586304:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619924.719835:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64aa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2355:x1875503571586304:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 233us (592us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619924.719842:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 765 00002000:00100000:1.0:1788619924.719849:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c64aa680 x1875503571586304/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/440 e 0 to 0 dl 1788619935 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619924.719857:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64aa680 with x1875503571586304 00000100:00100000:0.0:1788619924.722238:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619924.722346:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571586432 00000100:00100000:2.0:1788619924.722354:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571586432 00000100:00100000:2.0:1788619924.722408:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1030 00000100:00100000:2.0:1788619924.722419:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64a9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571586432:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619924.722436:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619924.722454:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003a2d9d44/0x1cd7fe5368735b7c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cdb1 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619924.722522:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64af480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589189248:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619924.722593:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164000 going to sleep for 11 seconds 00000100:00100000:2.0:1788619924.726419:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64af480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589189248:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619924.726459:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003a2d9d44/0x1cd7fe5368735b7c lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cdb1 expref: 8 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.726496:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a2d9d44/0x1cd7fe5368735b7c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cdb1 expref: 8 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.726511:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7600, rc 0) 00000100:00100000:2.0:1788619924.726566:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64a9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571586432:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4150us (4337us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619924.726575:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1030 00000100:00100000:0.0:1788619924.731057:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619924.731139:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571586560 00000100:00100000:3.0:1788619924.731148:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571586560 00002000:00100000:3.0:1788619924.731197:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c64a8e00 x1875503571586560/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/0 e 0 to 0 dl 1788619935 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619924.731215:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64a8e00 with x1875503571586560 00000100:00100000:3.0:1788619924.731255:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 766 00000100:00100000:3.0:1788619924.731265:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64a8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2354:x1875503571586560:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619924.731534:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64a8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2354:x1875503571586560:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 270us (488us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619924.731542:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 766 00002000:00100000:3.0:1788619924.731549:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c64a8e00 x1875503571586560/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/440 e 0 to 0 dl 1788619935 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619924.731557:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64a8e00 with x1875503571586560 00000100:00100000:0.0:1788619924.734132:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619924.734226:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571586688 00000100:00100000:1.0:1788619924.734235:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571586688 00000100:00100000:1.0:1788619924.734284:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1031 00000100:00100000:1.0:1788619924.734294:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64a9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571586688:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619924.734310:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619924.734357:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd485a32/0x1cd7fe5368735b83 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cdb8 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619924.734416:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64d3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589189376:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619924.734517:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7b40 going to sleep for 11 seconds 00000100:00100000:3.0:1788619924.737148:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64d3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589189376:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619924.737186:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000cd485a32/0x1cd7fe5368735b83 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cdb8 expref: 8 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.737212:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd485a32/0x1cd7fe5368735b83 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cdb8 expref: 8 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.737227:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296600, rc 0) 00000100:00100000:3.0:1788619924.737288:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64a9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571586688:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2994us (3166us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619924.737297:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1031 00000100:00100000:0.0:1788619924.739711:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619924.739807:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571586816 00000100:00100000:1.0:1788619924.739816:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571586816 00002000:00100000:1.0:1788619924.739861:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c64ab480 x1875503571586816/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/0 e 0 to 0 dl 1788619935 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619924.739873:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64ab480 with x1875503571586816 00000100:00100000:1.0:1788619924.739915:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 767 00000100:00100000:1.0:1788619924.739931:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64ab480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2355:x1875503571586816:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619924.740247:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64ab480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2355:x1875503571586816:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 316us (544us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619924.740255:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 767 00002000:00100000:1.0:1788619924.740262:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c64ab480 x1875503571586816/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/440 e 0 to 0 dl 1788619935 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619924.740271:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64ab480 with x1875503571586816 00000100:00100000:0.0:1788619924.744181:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619924.744269:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571586944 00000100:00100000:3.0:1788619924.744280:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571586944 00000100:00100000:3.0:1788619924.744362:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1032 00000100:00100000:3.0:1788619924.744372:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64a9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2353:x1875503571586944:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619924.744389:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619924.744408:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f980cda4/0x1cd7fe5368735b8a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cdbf expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619924.744468:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69fe680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589189504:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619924.744554:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619924.750546:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69fe680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589189504:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619924.750586:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f980cda4/0x1cd7fe5368735b8a lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cdbf expref: 8 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.750613:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f980cda4/0x1cd7fe5368735b8a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cdbf expref: 8 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.750627:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d200, rc 0) 00000100:00100000:3.0:1788619924.750689:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64a9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2353:x1875503571586944:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6318us (6520us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619924.750699:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1032 00000100:00100000:0.0:1788619924.753299:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619924.753548:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571587072 00000100:00100000:2.0:1788619924.753558:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571587072 00002000:00100000:2.0:1788619924.753610:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c64ab100 x1875503571587072/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/0 e 0 to 0 dl 1788619935 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619924.753622:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64ab100 with x1875503571587072 00000100:00100000:2.0:1788619924.753695:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 768 00000100:00100000:2.0:1788619924.753705:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64ab100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2353:x1875503571587072:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619924.754045:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64ab100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2353:x1875503571587072:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 340us (753us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619924.754053:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 768 00002000:00100000:2.0:1788619924.754060:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c64ab100 x1875503571587072/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/440 e 0 to 0 dl 1788619935 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619924.754068:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64ab100 with x1875503571587072 00000100:00100000:0.0:1788619924.757233:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619924.757378:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571587200 00000100:00100000:2.0:1788619924.757387:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571587200 00000100:00100000:2.0:1788619924.757466:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1033 00000100:00100000:2.0:1788619924.757543:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64a8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2352:x1875503571587200:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619924.757561:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619924.757578:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3b293bd/0x1cd7fe5368735b91 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cdc6 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619924.757631:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64af480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589189632:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619924.757754:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164000 going to sleep for 11 seconds 00000100:00100000:2.0:1788619924.759711:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64af480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589189632:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619924.759746:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f3b293bd/0x1cd7fe5368735b91 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cdc6 expref: 8 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.759771:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3b293bd/0x1cd7fe5368735b91 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cdc6 expref: 8 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.759784:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7e00, rc 0) 00000100:00100000:2.0:1788619924.759874:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64a8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2352:x1875503571587200:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2398us (2647us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619924.759882:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1033 00000100:00100000:0.0:1788619924.762814:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619924.763015:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571587328 00000100:00100000:1.0:1788619924.763024:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571587328 00002000:00100000:1.0:1788619924.763071:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c64aad80 x1875503571587328/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/0 e 0 to 0 dl 1788619935 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619924.763084:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64aad80 with x1875503571587328 00000100:00100000:1.0:1788619924.763152:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 769 00000100:00100000:1.0:1788619924.763163:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64aad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2353:x1875503571587328:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619924.763507:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64aad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2353:x1875503571587328:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 345us (700us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619924.763514:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 769 00002000:00100000:1.0:1788619924.763521:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c64aad80 x1875503571587328/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/440 e 0 to 0 dl 1788619935 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619924.763537:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64aad80 with x1875503571587328 00000100:00100000:0.0:1788619924.766316:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619924.766470:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571587456 00000100:00100000:2.0:1788619924.766537:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571587456 00000100:00100000:2.0:1788619924.766621:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1034 00000100:00100000:2.0:1788619924.766631:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64a8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2352:x1875503571587456:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619924.766674:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619924.766691:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000073be6ea/0x1cd7fe5368735b98 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cdcd expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619924.766750:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64af480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589189760:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619924.766828:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164000 going to sleep for 11 seconds 00000100:00100000:0.0:1788619924.768814:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64af480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589189760:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619924.768851:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000073be6ea/0x1cd7fe5368735b98 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cdcd expref: 8 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.768874:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000073be6ea/0x1cd7fe5368735b98 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cdcd expref: 8 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.768887:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7c00, rc 0) 00000100:00100000:0.0:1788619924.769066:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64a8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2352:x1875503571587456:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2435us (2756us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619924.769075:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1034 00000100:00100000:0.0:1788619924.771668:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619924.771750:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571587584 00000100:00100000:2.0:1788619924.771759:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571587584 00002000:00100000:2.0:1788619924.771806:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c7431c00 x1875503571587584/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/0 e 0 to 0 dl 1788619935 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619924.771818:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c7431c00 with x1875503571587584 00000100:00100000:2.0:1788619924.771882:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 770 00000100:00100000:2.0:1788619924.771893:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7431c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2352:x1875503571587584:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619924.772159:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7431c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2352:x1875503571587584:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 267us (498us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619924.772166:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 770 00002000:00100000:2.0:1788619924.772173:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c7431c00 x1875503571587584/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/440 e 0 to 0 dl 1788619935 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619924.772181:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c7431c00 with x1875503571587584 00000100:00100000:2.0:1788619924.775644:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619924.777569:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571587712 00000100:00100000:3.0:1788619924.777580:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571587712 00000100:00100000:2.0:1788619924.777773:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1035 00000100:00100000:2.0:1788619924.777785:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64af800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2353:x1875503571587712:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619924.777804:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619924.777822:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f3b293bd/0x1cd7fe5368735b9f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cdd4 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619924.777881:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64aed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589189888:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619924.778069:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81640c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619924.780441:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64aed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589189888:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619924.780489:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f3b293bd/0x1cd7fe5368735b9f lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cdd4 expref: 8 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.780514:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f3b293bd/0x1cd7fe5368735b9f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cdd4 expref: 8 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.780527:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7e00, rc 0) 00000100:00100000:2.0:1788619924.780583:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64af800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2353:x1875503571587712:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2801us (4948us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619924.780592:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1035 00000100:00100000:0.0:1788619924.783661:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619924.783893:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571587840 00000100:00100000:1.0:1788619924.783918:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571587840 00002000:00100000:1.0:1788619924.783969:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e6479c00 x1875503571587840/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/0 e 0 to 0 dl 1788619935 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619924.783980:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e6479c00 with x1875503571587840 00000100:00100000:1.0:1788619924.784019:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 771 00000100:00100000:1.0:1788619924.784029:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e6479c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2352:x1875503571587840:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619924.784237:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e6479c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2352:x1875503571587840:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 208us (600us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619924.784245:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 771 00002000:00100000:1.0:1788619924.784611:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e6479c00 x1875503571587840/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/440 e 0 to 0 dl 1788619935 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619924.784621:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e6479c00 with x1875503571587840 00000100:00100000:0.0:1788619924.788795:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619924.788877:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571587968 00000100:00100000:3.0:1788619924.788886:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571587968 00000100:00100000:3.0:1788619924.788947:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1036 00000100:00100000:3.0:1788619924.788956:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd93b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571587968:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619924.788974:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619924.788992:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f980cda4/0x1cd7fe5368735ba6 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cddb expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619924.789053:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69fe680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589190016:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619924.789146:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619924.791318:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69fe680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589190016:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619924.791391:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f980cda4/0x1cd7fe5368735ba6 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cddb expref: 8 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.791415:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f980cda4/0x1cd7fe5368735ba6 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cddb expref: 8 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.791430:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d200, rc 0) 00000100:00100000:3.0:1788619924.791480:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd93b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571587968:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2516us (2685us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619924.791489:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1036 00000100:00100000:2.0:1788619924.794220:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619924.794307:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571588096 00000100:00100000:0.0:1788619924.794316:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571588096 00002000:00100000:0.0:1788619924.794396:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c64aed80 x1875503571588096/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/0 e 0 to 0 dl 1788619935 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619924.794415:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64aed80 with x1875503571588096 00000100:00100000:0.0:1788619924.794443:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 772 00000100:00100000:0.0:1788619924.794453:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64aed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2353:x1875503571588096:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619924.794691:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64aed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2353:x1875503571588096:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 239us (480us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619924.794699:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 772 00002000:00100000:0.0:1788619924.794707:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c64aed80 x1875503571588096/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/440 e 0 to 0 dl 1788619935 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619924.794715:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64aed80 with x1875503571588096 00000100:00100000:0.0:1788619924.797750:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619924.797832:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571588224 00000100:00100000:1.0:1788619924.797840:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571588224 00000100:00100000:1.0:1788619924.797890:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1037 00000100:00100000:1.0:1788619924.797899:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571588224:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619924.797928:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619924.797946:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe5368735bad lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cde2 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619924.798004:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64d3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589190144:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619924.798089:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7d80 going to sleep for 11 seconds 00000100:00100000:1.0:1788619924.800415:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64d3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589190144:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619924.800453:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe5368735bad lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cde2 expref: 8 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.800489:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe5368735bad lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cde2 expref: 8 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.800502:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296e00, rc 0) 00000100:00100000:1.0:1788619924.800546:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571588224:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2648us (2806us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619924.800555:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1037 00000100:00100000:0.0:1788619924.807107:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619924.807190:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571588352 00000100:00100000:1.0:1788619924.807199:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571588352 00002000:00100000:1.0:1788619924.807246:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5cd9f80 x1875503571588352/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/0 e 0 to 0 dl 1788619935 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619924.807258:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5cd9f80 with x1875503571588352 00000100:00100000:1.0:1788619924.807292:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 773 00000100:00100000:1.0:1788619924.807302:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2353:x1875503571588352:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619924.807551:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2353:x1875503571588352:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 250us (457us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619924.807560:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 773 00002000:00100000:1.0:1788619924.807567:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5cd9f80 x1875503571588352/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/440 e 0 to 0 dl 1788619935 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619924.807575:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5cd9f80 with x1875503571588352 00000100:00100000:0.0:1788619924.810377:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619924.810462:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571588480 00000100:00100000:2.0:1788619924.810477:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571588480 00000100:00100000:2.0:1788619924.810528:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1038 00000100:00100000:2.0:1788619924.810538:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74a9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2353:x1875503571588480:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619924.810553:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619924.810570:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d2409dc/0x1cd7fe5368735bb4 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cde9 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619924.810624:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64ae300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589190272:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619924.810703:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81640c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619924.812937:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64ae300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589190272:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619924.812974:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000d2409dc/0x1cd7fe5368735bb4 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cde9 expref: 8 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.813001:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d2409dc/0x1cd7fe5368735bb4 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cde9 expref: 8 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.813015:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6000, rc 0) 00000100:00100000:0.0:1788619924.813070:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74a9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2353:x1875503571588480:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2533us (2700us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619924.813079:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1038 00000100:00100000:0.0:1788619924.815693:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619924.815767:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571588608 00000100:00100000:3.0:1788619924.815776:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571588608 00002000:00100000:3.0:1788619924.815822:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c74a8000 x1875503571588608/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/0 e 0 to 0 dl 1788619935 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619924.815834:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c74a8000 with x1875503571588608 00000100:00100000:3.0:1788619924.815871:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 774 00000100:00100000:3.0:1788619924.815880:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74a8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2352:x1875503571588608:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619924.816133:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74a8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2352:x1875503571588608:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 253us (447us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619924.816140:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 774 00002000:00100000:3.0:1788619924.816148:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c74a8000 x1875503571588608/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/440 e 0 to 0 dl 1788619935 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619924.816156:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c74a8000 with x1875503571588608 00000100:00100000:2.0:1788619924.818997:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619924.819090:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571588736 00000100:00100000:1.0:1788619924.819099:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571588736 00000100:00100000:1.0:1788619924.819153:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1039 00000100:00100000:1.0:1788619924.819162:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64adf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+7:14908:x1875503571588736:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619924.819179:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619924.819195:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe5368735bbb lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cdf0 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.819226:0:7654:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000007736e663/0x1cd7fe5368735b21 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd5d expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.819234:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(5 13) 00010000:00010000:1.0:1788619924.819238:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(6 13) 00010000:00010000:1.0:1788619924.819243:0:7654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe5368735bbb lrc: 4/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->1052671] (req 1048576->1052671) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cdf0 expref: 8 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.819272:0:7654:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000007736e663/0x1cd7fe5368735b21 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd5d expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.819286:0:7654:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000007736e663/0x1cd7fe5368735b21 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd5d expref: 6 pid: 7654 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619924.819299:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64d3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589190400:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619924.819396:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7d80 going to sleep for 11 seconds 00000100:00100000:2.0:1788619924.822097:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619924.822218:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64d3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589190400:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619924.822252:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe5368735bbb lrc: 3/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->1052671] (req 1048576->1052671) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cdf0 expref: 8 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.822277:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296e00, rc 0) 00000100:00100000:1.0:1788619924.822364:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64adf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+8:14908:x1875503571588736:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3203us (3378us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619924.822372:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1039 00000100:00100000:3.0:1788619924.822477:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571588864 00000100:00100000:3.0:1788619924.822492:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571588864 00010000:00010000:3.0:1788619924.822523:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000007736e663/0x1cd7fe5368735b21 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd5d expref: 7 pid: 7654 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619924.822557:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 426 00000100:00100000:3.0:1788619924.822566:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64af100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571588864:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619924.822578:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619924.822598:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000007736e663/0x1cd7fe5368735b21 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd5d expref: 7 pid: 7654 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.822609:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000007736e663/0x1cd7fe5368735b21 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd5d expref: 7 pid: 7654 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.822629:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007736e663/0x1cd7fe5368735b21 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd5d expref: 7 pid: 7654 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.822641:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(6 13) 00010000:00010000:3.0:1788619924.822661:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(6 13) 00010000:00010000:3.0:1788619924.822670:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe5368735bbb lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cdf0 expref: 7 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.822680:0:5831:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe5368735bbb lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cdf0 expref: 7 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.822711:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe5368735bbb lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cdf0 expref: 7 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619924.822723:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69fe680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5831:1875503589190528:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619924.822769:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619924.825120:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69fe680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5831:1875503589190528:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619924.825142:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619924.825194:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64af100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571588864:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 2628us (3106us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619924.825204:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 426 00000100:00100000:2.0:1788619924.835214:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619924.835256:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619924.835372:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571588992 00000100:00100000:1.0:1788619924.835382:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571588992 00002000:00100000:1.0:1788619924.835434:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1048576->2093055) req@ffff8e77c64ac000 x1875503571588992/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/0 e 0 to 0 dl 1788619935 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619924.835445:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64ac000 with x1875503571588992 ext(1048576->2093055) 00002000:00010000:1.0:1788619924.835451:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64ac000 00000100:00100000:1.0:1788619924.835485:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 595 00000100:00100000:1.0:1788619924.835496:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64ac000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571588992:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619924.837257:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571589120 00000100:00100000:0.0:1788619924.837270:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571589120 00002000:00100000:0.0:1788619924.837310:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c64ac700 x1875503571589120/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/0 e 0 to 0 dl 1788619935 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619924.837322:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64ac700 with x1875503571589120 00000100:00100000:0.0:1788619924.837381:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 775 00000100:00100000:0.0:1788619924.837389:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64ac700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2355:x1875503571589120:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619924.837587:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64ac700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2355:x1875503571589120:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 197us (2381us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619924.837594:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 775 00002000:00100000:0.0:1788619924.837603:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c64ac700 x1875503571589120/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/440 e 0 to 0 dl 1788619935 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619924.837610:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64ac700 with x1875503571589120 00000100:00100000:1.0:1788619924.850435:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64ac000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571588992:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 14929us (15166us total) trans 0 rc 1044480/1044480 00000100:00100000:1.0:1788619924.850450:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 595 00002000:00100000:1.0:1788619924.850463:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1048576->2093055) req@ffff8e77c64ac000 x1875503571588992/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:20/0 lens 488/440 e 0 to 0 dl 1788619935 ref 1 fl Complete:/600/0 rc 1044480/1044480 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619924.850485:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64ac000 with x1875503571588992 ext(1048576->2093055) 00002000:00010000:1.0:1788619924.850491:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64ac000 00000100:00100000:2.0:1788619924.854450:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619924.854593:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571589248 00000100:00100000:0.0:1788619924.854602:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571589248 00000100:00100000:0.0:1788619924.854673:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1269 00000100:00100000:0.0:1788619924.854683:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64ae300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:14908:x1875503571589248:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619924.854706:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619924.854727:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50a00/0x1cd7fe5368735bc2 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cdf7 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.854816:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50600/0x1cd7fe5368735bc9 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.854832:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50600/0x1cd7fe5368735bc9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.854848:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50600/0x1cd7fe5368735bc9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619924.854876:0:5846:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 2092012 00000004:00200000:0.0:1788619924.854881:0:5846:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619924.854901:0:5846:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50600/0x1cd7fe5368735bc9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.854939:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50a00/0x1cd7fe5368735bc2 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cdf7 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.854958:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50600/0x1cd7fe5368735bc9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cdf7 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619924.854966:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619924.854969:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619924.854973:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f50600, rc 0) 00000100:00100000:0.0:1788619924.855034:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64ae300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+13:14908:x1875503571589248:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 352us (595us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619924.855041:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1269 00000100:00100000:2.0:1788619924.857842:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619924.858142:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571589376 00000100:00100000:3.0:1788619924.858152:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571589376 00000100:00100000:3.0:1788619924.858209:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1270 00000100:00100000:3.0:1788619924.858218:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64af480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571589376:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619924.858237:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619924.858258:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368735bd0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cdfe expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.858293:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368735bd0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cdfe expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788619924.858305:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:3.0:1788619924.858313:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368735bd0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cdfe expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.858360:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619924.858363:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619924.858367:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d400, rc 0) 00000100:00100000:3.0:1788619924.858422:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64af480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+14:2354:x1875503571589376:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 205us (590us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619924.858429:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1270 00000100:00100000:2.0:1788619924.860769:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619924.860848:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571589504 00000100:00100000:3.0:1788619924.860856:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571589504 00000100:00100000:3.0:1788619924.860905:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1040 00000100:00100000:3.0:1788619924.860914:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64ac380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571589504:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619924.860939:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619924.860956:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f980cda4/0x1cd7fe5368735bd7 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ce05 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.860981:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(6 13) 00010000:00010000:3.0:1788619924.860985:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(6 13) 00010000:00010000:3.0:1788619924.860996:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f980cda4/0x1cd7fe5368735bd7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce05 expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.861013:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f980cda4/0x1cd7fe5368735bd7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce05 expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619924.861025:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d200, rc 0) 00000100:00100000:3.0:1788619924.861079:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64ac380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571589504:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 166us (317us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619924.861089:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1040 00000100:00100000:0.0:1788619924.875880:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619924.875939:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619924.876023:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571589760 00000100:00100000:3.0:1788619924.876034:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571589760 00002000:00100000:3.0:1788619924.876086:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c63fed80 x1875503571589760/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:20/0 lens 488/0 e 0 to 0 dl 1788619935 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619924.876100:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c63fed80 with x1875503571589760 00000100:00100000:3.0:1788619924.876137:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 776 00000100:00100000:3.0:1788619924.876146:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63fed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+14:2354:x1875503571589760:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619924.877061:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571589632 00000100:00100000:1.0:1788619924.877068:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571589632 00002000:00100000:1.0:1788619924.877099:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1048576->2093055) req@ffff8e77c5019f80 x1875503571589632/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:20/0 lens 488/0 e 0 to 0 dl 1788619935 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619924.877116:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5019f80 with x1875503571589632 ext(1048576->2093055) 00002000:00010000:1.0:1788619924.877120:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5019f80 00000100:00100000:1.0:1788619924.877143:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 596 00000100:00100000:1.0:1788619924.877151:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5019f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571589632:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619924.891660:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63fed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+14:2354:x1875503571589760:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 15495us (15700us total) trans 0 rc 987136/987136 00000100:00100000:3.0:1788619924.891674:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 776 00002000:00100000:3.0:1788619924.891687:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c63fed80 x1875503571589760/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:20/0 lens 488/440 e 0 to 0 dl 1788619935 ref 1 fl Complete:/600/0 rc 987136/987136 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619924.891697:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c63fed80 with x1875503571589760 00000100:00100000:1.0:1788619924.897939:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5019f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571589632:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 20777us (22058us total) trans 0 rc 1044480/1044480 00000100:00100000:1.0:1788619924.897953:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 596 00002000:00100000:1.0:1788619924.897965:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1048576->2093055) req@ffff8e77c5019f80 x1875503571589632/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:20/0 lens 488/440 e 0 to 0 dl 1788619935 ref 1 fl Complete:/600/0 rc 1044480/1044480 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619924.897976:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5019f80 with x1875503571589632 ext(1048576->2093055) 00002000:00010000:1.0:1788619924.897984:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5019f80 00000100:00100000:0.0:1788619924.903604:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619924.903767:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571589888 00000100:00100000:2.0:1788619924.903778:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571589888 00000100:00100000:2.0:1788619924.903844:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1041 00000100:00100000:2.0:1788619924.903856:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63f8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:14908:x1875503571589888:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619924.903877:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619924.903898:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011e2507b/0x1cd7fe5368735bde lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ce0c expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.903944:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe5368735bbb lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cdf0 expref: 7 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.903953:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000f980cda4/0x1cd7fe5368735bd7 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce05 expref: 8 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.903961:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(6 13) 00010000:00010000:2.0:1788619924.903965:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(6 13) 00010000:00010000:2.0:1788619924.903972:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011e2507b/0x1cd7fe5368735bde lrc: 5/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [1536000->1540095] (req 1536000->1540095) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce0c expref: 8 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.904009:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000f980cda4/0x1cd7fe5368735bd7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce05 expref: 8 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.904029:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000f980cda4/0x1cd7fe5368735bd7 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce05 expref: 8 pid: 6696 timeout: 517 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619924.904046:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64ad500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589190656:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619924.904142:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe5368735bbb lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cdf0 expref: 7 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.904152:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe5368735bbb lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cdf0 expref: 7 pid: 7654 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619924.904162:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64aca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589190784:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619924.904185:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81649c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619924.906913:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64ad500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589190656:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619924.906938:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64aca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589190784:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619924.906972:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000011e2507b/0x1cd7fe5368735bde lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [1536000->1540095] (req 1536000->1540095) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce0c expref: 8 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619924.906994:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f73800, rc 0) 00000100:00100000:1.0:1788619924.907049:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63f8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+8:14908:x1875503571589888:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3196us (3455us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619924.907057:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1041 00000100:00100000:3.0:1788619924.911844:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619924.911885:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619924.912378:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571590016 00000100:00100000:2.0:1788619924.912388:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571590016 00010000:00010000:2.0:1788619924.912430:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000f980cda4/0x1cd7fe5368735bd7 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce05 expref: 8 pid: 6696 timeout: 517 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619924.912462:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571590144 00000100:00100000:2.0:1788619924.912465:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571590144 00010000:00010000:2.0:1788619924.912479:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe5368735bbb lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cdf0 expref: 8 pid: 7654 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619924.912505:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 427 00000100:00100000:2.0:1788619924.912514:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fe680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+8:2354:x1875503571590016:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619924.912529:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619924.912550:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000f980cda4/0x1cd7fe5368735bd7 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce05 expref: 8 pid: 6696 timeout: 517 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.912564:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000f980cda4/0x1cd7fe5368735bd7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce05 expref: 8 pid: 6696 timeout: 517 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.912587:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f980cda4/0x1cd7fe5368735bd7 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce05 expref: 8 pid: 6696 timeout: 517 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.912607:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619924.912671:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fe680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571590016:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 158us (838us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619924.912679:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 427 00000100:00100000:2.0:1788619924.912696:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 428 00000100:00100000:2.0:1788619924.912703:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571590144:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619924.912711:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619924.912721:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe5368735bbb lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cdf0 expref: 8 pid: 7654 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.912731:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe5368735bbb lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cdf0 expref: 8 pid: 7654 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.912745:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe5368735bbb lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1052671) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cdf0 expref: 8 pid: 7654 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.912761:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(6 13) 00010000:00010000:2.0:1788619924.912766:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(6 13) 00010000:00010000:2.0:1788619924.912773:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000011e2507b/0x1cd7fe5368735bde lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1536000->1540095) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce0c expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.912783:0:5829:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011e2507b/0x1cd7fe5368735bde lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1536000->1540095) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce0c expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619924.912820:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000011e2507b/0x1cd7fe5368735bde lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1536000->1540095) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce0c expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619924.912835:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64adc00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1875503589190912:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619924.912886:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164480 going to sleep for 11 seconds 00000100:00100000:2.0:1788619924.918051:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64adc00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1875503589190912:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619924.918078:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619924.918133:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571590144:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 5429us (6245us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619924.918145:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 428 00000100:00100000:2.0:1788619924.931423:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619924.931729:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571590272 00000100:00100000:1.0:1788619924.931739:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571590272 00002000:00100000:1.0:1788619924.931794:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1536000->1540095) req@ffff8e77c64adc00 x1875503571590272/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:20/0 lens 488/0 e 0 to 0 dl 1788619935 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619924.931807:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64adc00 with x1875503571590272 ext(1536000->1540095) 00002000:00010000:1.0:1788619924.931814:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64adc00 00000100:00100000:1.0:1788619924.931887:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 597 00000100:00100000:1.0:1788619924.931897:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64adc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571590272:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619924.932160:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64adc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571590272:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 263us (841us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619924.932168:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 597 00002000:00100000:1.0:1788619924.932175:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1536000->1540095) req@ffff8e77c64adc00 x1875503571590272/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:20/0 lens 488/4536 e 0 to 0 dl 1788619935 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619924.932182:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64adc00 with x1875503571590272 ext(1536000->1540095) 00002000:00010000:1.0:1788619924.932186:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64adc00 00000100:00100000:3.0:1788619924.935186:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619924.935363:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571590400 00000100:00100000:1.0:1788619924.935373:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571590400 00002000:00100000:1.0:1788619924.935421:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1540096->1544191) req@ffff8e77c69fce00 x1875503571590400/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:20/0 lens 488/0 e 0 to 0 dl 1788619935 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619924.935433:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c69fce00 with x1875503571590400 ext(1540096->1544191) 00002000:00010000:1.0:1788619924.935438:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c69fce00 00000100:00100000:1.0:1788619924.935466:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 598 00000100:00100000:1.0:1788619924.935480:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571590400:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619924.935734:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571590400:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 253us (556us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619924.935741:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 598 00002000:00100000:1.0:1788619924.935749:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1540096->1544191) req@ffff8e77c69fce00 x1875503571590400/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:20/0 lens 488/4536 e 0 to 0 dl 1788619935 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619924.935757:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c69fce00 with x1875503571590400 ext(1540096->1544191) 00002000:00010000:1.0:1788619924.935760:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c69fce00 00000100:00100000:0.0:1788619924.941043:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619924.941352:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571590528 00000100:00100000:1.0:1788619924.941363:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571590528 00002000:00100000:1.0:1788619924.941413:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1544192->1548287) req@ffff8e77c5f26d80 x1875503571590528/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:20/0 lens 488/0 e 0 to 0 dl 1788619935 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619924.941425:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5f26d80 with x1875503571590528 ext(1544192->1548287) 00002000:00010000:1.0:1788619924.941431:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5f26d80 00000100:00100000:1.0:1788619924.941464:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 599 00000100:00100000:1.0:1788619924.941483:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f26d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2352:x1875503571590528:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619924.941754:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f26d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2352:x1875503571590528:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 280us (720us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619924.941763:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 599 00002000:00100000:1.0:1788619924.941769:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1544192->1548287) req@ffff8e77c5f26d80 x1875503571590528/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:20/0 lens 488/4536 e 0 to 0 dl 1788619935 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619924.941777:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5f26d80 with x1875503571590528 ext(1544192->1548287) 00002000:00010000:1.0:1788619924.941780:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5f26d80 00000100:00100000:2.0:1788619924.944809:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619924.944902:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571590656 00000100:00100000:3.0:1788619924.944923:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571590656 00002000:00100000:3.0:1788619924.944976:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1548288->1552383) req@ffff8e77c64ae680 x1875503571590656/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:20/0 lens 488/0 e 0 to 0 dl 1788619935 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619924.944988:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64ae680 with x1875503571590656 ext(1548288->1552383) 00002000:00010000:3.0:1788619924.944994:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64ae680 00000100:00100000:3.0:1788619924.945029:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 600 00000100:00100000:3.0:1788619924.945038:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64ae680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2352:x1875503571590656:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619924.945312:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64ae680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2352:x1875503571590656:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 273us (513us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619924.945320:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 600 00002000:00100000:3.0:1788619924.945364:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1548288->1552383) req@ffff8e77c64ae680 x1875503571590656/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:20/0 lens 488/4536 e 0 to 0 dl 1788619935 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619924.945372:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64ae680 with x1875503571590656 ext(1548288->1552383) 00002000:00010000:3.0:1788619924.945376:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64ae680 00000100:00100000:3.0:1788619924.949186:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619924.949290:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571590784 00000100:00100000:0.0:1788619924.949299:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571590784 00002000:00100000:0.0:1788619924.949371:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1552384->1556479) req@ffff8e77c69fd500 x1875503571590784/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:20/0 lens 488/0 e 0 to 0 dl 1788619935 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619924.949388:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c69fd500 with x1875503571590784 ext(1552384->1556479) 00002000:00010000:0.0:1788619924.949393:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c69fd500 00000100:00100000:0.0:1788619924.949425:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 601 00000100:00100000:0.0:1788619924.949434:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2353:x1875503571590784:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619924.949790:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2353:x1875503571590784:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 356us (612us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619924.949799:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 601 00002000:00100000:0.0:1788619924.949806:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1552384->1556479) req@ffff8e77c69fd500 x1875503571590784/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:20/0 lens 488/4536 e 0 to 0 dl 1788619935 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619924.949814:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c69fd500 with x1875503571590784 ext(1552384->1556479) 00002000:00010000:0.0:1788619924.949818:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c69fd500 00000100:00100000:0.0:1788619924.953718:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619924.953835:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571590912 00000100:00100000:1.0:1788619924.953846:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571590912 00002000:00100000:1.0:1788619924.953898:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1556480->1560575) req@ffff8e77c5f26300 x1875503571590912/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:20/0 lens 488/0 e 0 to 0 dl 1788619935 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619924.953908:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5f26300 with x1875503571590912 ext(1556480->1560575) 00002000:00010000:1.0:1788619924.953973:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5f26300 00000100:00100000:1.0:1788619924.954041:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 602 00000100:00100000:1.0:1788619924.954052:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f26300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2353:x1875503571590912:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619924.954322:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f26300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2353:x1875503571590912:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 271us (614us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619924.954353:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 602 00002000:00100000:1.0:1788619924.954361:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1556480->1560575) req@ffff8e77c5f26300 x1875503571590912/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:20/0 lens 488/4536 e 0 to 0 dl 1788619935 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619924.954370:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5f26300 with x1875503571590912 ext(1556480->1560575) 00002000:00010000:1.0:1788619924.954373:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5f26300 00000100:00100000:2.0:1788619924.957963:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619924.958050:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571591040 00000100:00100000:0.0:1788619924.958060:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571591040 00002000:00100000:0.0:1788619924.958111:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1560576->1564671) req@ffff8e77c64aca80 x1875503571591040/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:20/0 lens 488/0 e 0 to 0 dl 1788619935 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619924.958122:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64aca80 with x1875503571591040 ext(1560576->1564671) 00002000:00010000:0.0:1788619924.958128:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64aca80 00000100:00100000:0.0:1788619924.958164:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 603 00000100:00100000:0.0:1788619924.958173:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64aca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2353:x1875503571591040:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619924.958451:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64aca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2353:x1875503571591040:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 278us (496us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619924.958459:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 603 00002000:00100000:0.0:1788619924.958477:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1560576->1564671) req@ffff8e77c64aca80 x1875503571591040/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:20/0 lens 488/4536 e 0 to 0 dl 1788619935 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619924.958487:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64aca80 with x1875503571591040 ext(1560576->1564671) 00002000:00010000:0.0:1788619924.958490:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64aca80 00000100:00100000:2.0:1788619924.962065:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619924.962166:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571591168 00000100:00100000:0.0:1788619924.962175:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571591168 00002000:00100000:0.0:1788619924.962224:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1564672->1568767) req@ffff8e77c64ad500 x1875503571591168/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:20/0 lens 488/0 e 0 to 0 dl 1788619935 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619924.962235:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64ad500 with x1875503571591168 ext(1564672->1568767) 00002000:00010000:0.0:1788619924.962240:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64ad500 00000100:00100000:0.0:1788619924.962272:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 604 00000100:00100000:0.0:1788619924.962281:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64ad500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571591168:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619924.962562:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64ad500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571591168:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 281us (505us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619924.962570:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 604 00002000:00100000:0.0:1788619924.962576:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1564672->1568767) req@ffff8e77c64ad500 x1875503571591168/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:20/0 lens 488/4536 e 0 to 0 dl 1788619935 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619924.962583:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64ad500 with x1875503571591168 ext(1564672->1568767) 00002000:00010000:0.0:1788619924.962587:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64ad500 00000100:00100000:3.0:1788619924.966074:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619924.966271:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571591296 00000100:00100000:1.0:1788619924.966280:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571591296 00002000:00100000:1.0:1788619924.966359:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1568768->1572863) req@ffff8e77c69fd180 x1875503571591296/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:20/0 lens 488/0 e 0 to 0 dl 1788619935 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619924.966371:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c69fd180 with x1875503571591296 ext(1568768->1572863) 00002000:00010000:1.0:1788619924.966376:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c69fd180 00000100:00100000:1.0:1788619924.966399:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 605 00000100:00100000:1.0:1788619924.966408:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571591296:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619924.966662:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571591296:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 254us (595us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619924.966670:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 605 00002000:00100000:1.0:1788619924.966677:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1568768->1572863) req@ffff8e77c69fd180 x1875503571591296/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:20/0 lens 488/4536 e 0 to 0 dl 1788619935 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619924.966685:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c69fd180 with x1875503571591296 ext(1568768->1572863) 00002000:00010000:1.0:1788619924.966690:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c69fd180 00000100:00100000:2.0:1788619924.969843:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619924.969943:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571591424 00000100:00100000:3.0:1788619924.969952:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571591424 00002000:00100000:3.0:1788619924.969999:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1572864->1576959) req@ffff8e77c6a5e680 x1875503571591424/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:20/0 lens 488/0 e 0 to 0 dl 1788619935 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619924.970011:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a5e680 with x1875503571591424 ext(1572864->1576959) 00002000:00010000:3.0:1788619924.970017:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a5e680 00000100:00100000:3.0:1788619924.970052:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 606 00000100:00100000:3.0:1788619924.970061:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2352:x1875503571591424:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619924.970315:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2352:x1875503571591424:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 254us (482us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619924.970323:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 606 00002000:00100000:3.0:1788619924.970368:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1572864->1576959) req@ffff8e77c6a5e680 x1875503571591424/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:20/0 lens 488/4536 e 0 to 0 dl 1788619935 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619924.970375:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a5e680 with x1875503571591424 ext(1572864->1576959) 00002000:00010000:3.0:1788619924.970379:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a5e680 00000100:00100000:2.0:1788619924.973603:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619924.973713:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571591552 00000100:00100000:0.0:1788619924.973723:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571591552 00002000:00100000:0.0:1788619924.973770:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1576960->1581055) req@ffff8e77c6a5dc00 x1875503571591552/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:20/0 lens 488/0 e 0 to 0 dl 1788619935 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619924.973782:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a5dc00 with x1875503571591552 ext(1576960->1581055) 00002000:00010000:0.0:1788619924.973787:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a5dc00 00000100:00100000:0.0:1788619924.973818:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 607 00000100:00100000:0.0:1788619924.973828:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2353:x1875503571591552:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619924.974105:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2353:x1875503571591552:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 277us (510us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619924.974114:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 607 00002000:00100000:0.0:1788619924.974121:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1576960->1581055) req@ffff8e77c6a5dc00 x1875503571591552/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:20/0 lens 488/4536 e 0 to 0 dl 1788619935 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619924.974129:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a5dc00 with x1875503571591552 ext(1576960->1581055) 00002000:00010000:0.0:1788619924.974133:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a5dc00 00000100:00100000:0.0:1788619924.977252:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619924.977355:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571591680 00000100:00100000:1.0:1788619924.977365:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571591680 00002000:00100000:1.0:1788619924.977413:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1581056->1585151) req@ffff8e78c8596300 x1875503571591680/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:20/0 lens 488/0 e 0 to 0 dl 1788619935 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619924.977425:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c8596300 with x1875503571591680 ext(1581056->1585151) 00002000:00010000:1.0:1788619924.977430:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c8596300 00000100:00100000:1.0:1788619924.977462:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 608 00000100:00100000:1.0:1788619924.977473:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8596300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2353:x1875503571591680:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619924.977832:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8596300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2353:x1875503571591680:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 358us (589us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619924.977840:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 608 00002000:00100000:1.0:1788619924.977847:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1581056->1585151) req@ffff8e78c8596300 x1875503571591680/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:20/0 lens 488/4536 e 0 to 0 dl 1788619935 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619924.977855:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c8596300 with x1875503571591680 ext(1581056->1585151) 00002000:00010000:1.0:1788619924.977859:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c8596300 00000100:00100000:2.0:1788619924.981904:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619924.982034:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571591808 00000100:00100000:3.0:1788619924.982044:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571591808 00002000:00100000:3.0:1788619924.982094:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1536000->1585151) req@ffff8e77c6a5ce00 x1875503571591808/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:20/0 lens 488/0 e 0 to 0 dl 1788619935 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619924.982106:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a5ce00 with x1875503571591808 ext(1536000->1585151) 00002000:00010000:3.0:1788619924.982111:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a5ce00 00000100:00100000:3.0:1788619924.982149:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 609 00000100:00100000:3.0:1788619924.982159:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2353:x1875503571591808:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:3.0:1788619924.985129:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571591808:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 2960us (3224us total) trans 4294967472 rc 0/0 00000100:00100000:3.0:1788619924.985143:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 609 00002000:00100000:3.0:1788619924.985155:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1536000->1585151) req@ffff8e77c6a5ce00 x1875503571591808/t4294967472(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:20/0 lens 488/448 e 0 to 0 dl 1788619935 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619924.985166:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a5ce00 with x1875503571591808 ext(1536000->1585151) 00002000:00010000:3.0:1788619924.985172:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a5ce00 00000100:00100000:2.0:1788619924.993346:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619924.993435:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571591936 00000100:00100000:3.0:1788619924.993445:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571591936 00000100:00100000:3.0:1788619924.993616:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1042 00000100:00100000:3.0:1788619924.993627:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571591936:12345-192.168.201.5@tcp:16:fsx.0 00000001:00080000:1.0:1788619925.025239:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967470 is committed 00000001:00080000:1.0:1788619925.025258:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967471 is committed 00000001:00080000:1.0:1788619925.025264:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967472 is committed 00000100:00100000:2.0:1788619925.025667:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571591936:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 32036us (32340us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619925.025680:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1042 00000100:00100000:1.0:1788619925.029987:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619925.030083:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571592064 00000100:00100000:0.0:1788619925.030094:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571592064 00002000:00100000:0.0:1788619925.030147:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1916928->1921023) req@ffff8e77c64d3800 x1875503571592064/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 488/0 e 0 to 0 dl 1788619936 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619925.030160:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64d3800 with x1875503571592064 ext(1916928->1921023) 00002000:00010000:0.0:1788619925.030166:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64d3800 00000100:00100000:0.0:1788619925.030233:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 610 00000100:00100000:0.0:1788619925.030245:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64d3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571592064:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619925.030629:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64d3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571592064:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 385us (651us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619925.030638:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 610 00002000:00100000:0.0:1788619925.030655:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1916928->1921023) req@ffff8e77c64d3800 x1875503571592064/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 488/4536 e 0 to 0 dl 1788619936 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619925.030671:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64d3800 with x1875503571592064 ext(1916928->1921023) 00002000:00010000:0.0:1788619925.030676:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64d3800 00000100:00100000:0.0:1788619925.033662:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619925.033786:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571592192 00000100:00100000:3.0:1788619925.033797:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571592192 00002000:00100000:3.0:1788619925.033843:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1933312->1937407) req@ffff8e78c8595180 x1875503571592192/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 488/0 e 0 to 0 dl 1788619936 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619925.033854:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c8595180 with x1875503571592192 ext(1933312->1937407) 00002000:00010000:3.0:1788619925.033861:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c8595180 00000100:00100000:3.0:1788619925.033895:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 611 00000100:00100000:3.0:1788619925.033903:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8595180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571592192:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619925.034288:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8595180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571592192:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 384us (645us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619925.034296:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 611 00002000:00100000:3.0:1788619925.034303:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1933312->1937407) req@ffff8e78c8595180 x1875503571592192/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 488/4536 e 0 to 0 dl 1788619936 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619925.034312:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c8595180 with x1875503571592192 ext(1933312->1937407) 00002000:00010000:3.0:1788619925.034316:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c8595180 00000100:00100000:2.0:1788619925.038123:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619925.038225:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571592320 00000100:00100000:1.0:1788619925.038236:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571592320 00000100:00100000:1.0:1788619925.038356:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1043 00000100:00100000:1.0:1788619925.038367:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571592320:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619925.038386:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619925.038404:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c5e22f67/0x1cd7fe5368735be5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ce13 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619925.038560:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64d3100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589191040:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619925.038725:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7b40 going to sleep for 11 seconds 00002000:00010000:1.0:1788619925.040995:0:7654:0:(ofd_lvb.c:284:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks: 391 -> 519 00000100:00100000:1.0:1788619925.041028:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64d3100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589191040:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619925.041066:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c5e22f67/0x1cd7fe5368735be5 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce13 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.041091:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c5e22f67/0x1cd7fe5368735be5 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce13 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.041105:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297000, rc 0) 00000100:00100000:1.0:1788619925.041195:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571592320:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2829us (3081us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619925.041204:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1043 00000100:00100000:0.0:1788619925.044025:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619925.044135:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571592448 00000100:00100000:2.0:1788619925.044147:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571592448 00000100:00100000:2.0:1788619925.044298:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1044 00000100:00100000:2.0:1788619925.044310:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571592448:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619925.044355:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619925.044374:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002dd9059c/0x1cd7fe5368735bec lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ce1a expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619925.044435:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a5ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589191168:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619925.044661:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619925.047133:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a5ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589191168:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619925.047171:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002dd9059c/0x1cd7fe5368735bec lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce1a expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.047194:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002dd9059c/0x1cd7fe5368735bec lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce1a expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.047206:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f73c00, rc 0) 00000100:00100000:0.0:1788619925.047293:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571592448:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2986us (3277us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619925.047303:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1044 00000100:00100000:1.0:1788619925.051650:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619925.051740:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571592576 00000100:00100000:2.0:1788619925.051749:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571592576 00000100:00100000:2.0:1788619925.051834:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1271 00000100:00100000:2.0:1788619925.051844:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64d3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+14:14908:x1875503571592576:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619925.051865:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619925.051868:0:5844:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1788619925.051928:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368735bd0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cdfe expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.051947:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619925.051953:0:5844:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788619925.051968:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735bf3 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ce21 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.051996:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe5368735b60 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd95 expref: 17 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.052006:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735bf3 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce21 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.052039:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe5368735b60 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd95 expref: 17 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.052052:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe5368735b60 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd95 expref: 17 pid: 5844 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619925.052065:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a5ca80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589191296:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619925.052186:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164600 going to sleep for 11 seconds 00000100:00100000:3.0:1788619925.057892:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619925.058141:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a5ca80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589191296:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619925.058173:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735bf3 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce21 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.058197:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619925.058200:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619925.058205:0:5844:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735bf3 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce21 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.058216:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f73400, rc 0) 00000100:00100000:2.0:1788619925.058280:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64d3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+14:14908:x1875503571592576:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6437us (6649us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619925.058289:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1271 00000100:00100000:1.0:1788619925.058397:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571592704 00000100:00100000:1.0:1788619925.058403:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571592704 00010000:00010000:1.0:1788619925.058434:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe5368735b60 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd95 expref: 18 pid: 5844 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619925.058464:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 429 00000100:00100000:1.0:1788619925.058658:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2355:x1875503571592704:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619925.058671:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619925.058699:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe5368735b60 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd95 expref: 18 pid: 5844 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.058712:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe5368735b60 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd95 expref: 18 pid: 5844 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.058734:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7000/0x1cd7fe5368735b60 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd95 expref: 18 pid: 5844 timeout: 517 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.058748:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619925.058753:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735bf3 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce21 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.058762:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735bf3 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce21 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.058771:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735bf3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce21 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.058804:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735bf3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce21 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619925.058820:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64d0700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589191424:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619925.058873:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7b40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619925.062364:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619925.062439:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571592832 00000100:00100000:3.0:1788619925.062448:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571592832 00000100:00100000:1.0:1788619925.062587:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64d0700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589191424:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619925.062609:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619925.062666:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2355:x1875503571592704:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 4020us (4784us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619925.062677:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 429 00000100:00100000:3.0:1788619925.062759:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 777 00000100:00100000:3.0:1788619925.062770:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+14:2353:x1875503571592832:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:0.0:1788619925.064246:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 987136 -> 987136 00000004:00010000:0.0:1788619925.064253:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619924 -> 1788619924 00000004:00010000:0.0:1788619925.064256:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619924 -> 1788619924 00000004:00010000:0.0:1788619925.064259:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619924 -> 1788619924 00000004:00010000:0.0:1788619925.064261:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 902 -> 902 00000100:00100000:0.0:1788619925.064393:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+15:2353:x1875503571592832:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 1622us (2069us total) trans 4294967752 rc 0/0 00000100:00100000:0.0:1788619925.064402:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 777 00000100:00100000:0.0:1788619925.067279:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619925.067321:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619925.067423:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571592960 00000100:00100000:1.0:1788619925.067433:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571592960 00000100:00100000:1.0:1788619925.067488:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1272 00000100:00100000:1.0:1788619925.067499:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a59880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2353:x1875503571592960:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619925.067518:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619925.067538:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe5368735bfa lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ce28 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.067576:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735bf3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce21 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619925.067616:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64d0700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589191552:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619925.067708:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7b40 going to sleep for 11 seconds 00000100:00100000:2.0:1788619925.068228:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571593088 00000100:00100000:2.0:1788619925.068234:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571593088 00000100:00100000:2.0:1788619925.068272:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1045 00000100:00100000:2.0:1788619925.068282:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571593088:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619925.068292:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619925.068303:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b040813e/0x1cd7fe5368735c01 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ce2f expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619925.068375:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a5ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589191680:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619925.068554:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164600 going to sleep for 11 seconds 00000100:00100000:1.0:1788619925.070811:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64d0700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589191552:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619925.070840:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:1.0:1788619925.070862:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe5368735bfa lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce28 expref: 18 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.070883:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619925.070886:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619925.070892:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe5368735bfa lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce28 expref: 18 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.071001:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297000, rc 0) 00000100:00100000:1.0:1788619925.071064:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a59880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2353:x1875503571592960:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3567us (3793us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619925.071073:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1272 00000100:00100000:2.0:1788619925.073651:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a5ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589191680:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619925.073692:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b040813e/0x1cd7fe5368735c01 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce2f expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.073717:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b040813e/0x1cd7fe5368735c01 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce2f expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.073729:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f72600, rc 0) 00000100:00100000:2.0:1788619925.073788:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571593088:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5508us (6466us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619925.073797:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1045 00000100:00100000:0.0:1788619925.076873:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619925.076927:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619925.076995:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571593344 00000100:00100000:1.0:1788619925.077004:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571593344 00000100:00100000:1.0:1788619925.077056:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1046 00000100:00100000:1.0:1788619925.077064:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571593344:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619925.077081:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619925.077096:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003086a8b7/0x1cd7fe5368735c08 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ce3d expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619925.077152:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64d0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589191808:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619925.077232:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7b40 going to sleep for 11 seconds 00000100:00100000:2.0:1788619925.077275:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571593216 00000100:00100000:2.0:1788619925.077279:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571593216 00000100:00100000:2.0:1788619925.077317:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1273 00000100:00100000:2.0:1788619925.077346:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2353:x1875503571593216:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619925.077357:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619925.077370:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72a00/0x1cd7fe5368735c0f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ce36 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.077397:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735bf3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce21 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619925.077420:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a5ca80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589191936:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619925.077445:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164600 going to sleep for 11 seconds 00000100:00100000:1.0:1788619925.080216:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64d0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589191808:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619925.080252:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003086a8b7/0x1cd7fe5368735c08 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce3d expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.080274:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003086a8b7/0x1cd7fe5368735c08 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce3d expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.080287:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296400, rc 0) 00000100:00100000:1.0:1788619925.080373:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571593344:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3309us (3444us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619925.080382:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1046 00000100:00100000:2.0:1788619925.080654:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a5ca80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589191936:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619925.080670:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:2.0:1788619925.080681:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72a00/0x1cd7fe5368735c0f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce36 expref: 18 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.080694:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619925.080698:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619925.080702:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72a00/0x1cd7fe5368735c0f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce36 expref: 18 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.080712:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f72a00, rc 0) 00000100:00100000:2.0:1788619925.080737:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2353:x1875503571593216:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3414us (3875us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619925.080744:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1273 00000100:00100000:0.0:1788619925.087730:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619925.087822:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571593472 00000100:00100000:1.0:1788619925.087832:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571593472 00002000:00100000:1.0:1788619925.087881:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1908736->1912831) req@ffff8e77c69f8700 x1875503571593472/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 488/0 e 0 to 0 dl 1788619936 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619925.087893:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c69f8700 with x1875503571593472 ext(1908736->1912831) 00002000:00010000:1.0:1788619925.087898:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c69f8700 00000100:00100000:1.0:1788619925.087937:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 612 00000100:00100000:1.0:1788619925.087946:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571593472:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619925.088189:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571593472:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 242us (465us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619925.088196:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 612 00002000:00100000:1.0:1788619925.088204:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1908736->1912831) req@ffff8e77c69f8700 x1875503571593472/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 488/4536 e 0 to 0 dl 1788619936 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619925.088211:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c69f8700 with x1875503571593472 ext(1908736->1912831) 00002000:00010000:1.0:1788619925.088214:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c69f8700 00000100:00100000:3.0:1788619925.091675:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619925.091878:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571593600 00000100:00100000:1.0:1788619925.091888:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571593600 00002000:00100000:1.0:1788619925.091947:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1912832->1916927) req@ffff8e77c69fed80 x1875503571593600/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 488/0 e 0 to 0 dl 1788619936 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619925.091958:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c69fed80 with x1875503571593600 ext(1912832->1916927) 00002000:00010000:1.0:1788619925.091965:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c69fed80 00000100:00100000:1.0:1788619925.092002:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 613 00000100:00100000:1.0:1788619925.092012:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571593600:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619925.092589:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571593600:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 576us (923us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619925.092598:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 613 00002000:00100000:1.0:1788619925.092606:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1912832->1916927) req@ffff8e77c69fed80 x1875503571593600/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 488/4536 e 0 to 0 dl 1788619936 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619925.092614:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c69fed80 with x1875503571593600 ext(1912832->1916927) 00002000:00010000:1.0:1788619925.092618:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c69fed80 00000100:00100000:2.0:1788619925.097529:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619925.097658:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571593728 00000100:00100000:0.0:1788619925.097668:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571593728 00002000:00100000:0.0:1788619925.097715:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1908736->1937407) req@ffff8e77c6a5ca80 x1875503571593728/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 488/0 e 0 to 0 dl 1788619936 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619925.097727:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a5ca80 with x1875503571593728 ext(1908736->1937407) 00002000:00010000:0.0:1788619925.097732:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a5ca80 00000100:00100000:0.0:1788619925.097816:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 614 00000100:00100000:0.0:1788619925.097826:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571593728:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:0.0:1788619925.100749:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+8:2353:x1875503571593728:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 2914us (3283us total) trans 4294967473 rc 0/0 00000100:00100000:0.0:1788619925.100762:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 614 00002000:00100000:0.0:1788619925.100774:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1908736->1937407) req@ffff8e77c6a5ca80 x1875503571593728/t4294967473(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 488/448 e 0 to 0 dl 1788619936 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619925.100784:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a5ca80 with x1875503571593728 ext(1908736->1937407) 00002000:00010000:0.0:1788619925.100790:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a5ca80 00000100:00100000:2.0:1788619925.103288:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619925.103408:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571593856 00000100:00100000:3.0:1788619925.103418:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571593856 00000100:00100000:3.0:1788619925.103484:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1047 00000100:00100000:3.0:1788619925.103496:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+8:2354:x1875503571593856:12345-192.168.201.5@tcp:16:fsx.0 00000100:00100000:0.0:1788619925.135252:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+8:2354:x1875503571593856:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 31748us (31976us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619925.135267:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1047 00000001:00080000:1.0:1788619925.135926:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967473 is committed 00000100:00100000:3.0:1788619925.138753:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619925.138798:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619925.138900:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571594112 00000100:00100000:0.0:1788619925.138911:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571594112 00000100:00100000:0.0:1788619925.138967:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1048 00000100:00100000:0.0:1788619925.138978:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69ff100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571594112:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619925.138997:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619925.139015:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000052301317/0x1cd7fe5368735c16 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ce4b expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619925.139074:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69fa680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589192064:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619925.139160:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590840 going to sleep for 11 seconds 00000100:00100000:2.0:1788619925.139440:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571593984 00000100:00100000:2.0:1788619925.139445:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571593984 00000100:00100000:2.0:1788619925.139495:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1274 00000100:00100000:2.0:1788619925.139504:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2355:x1875503571593984:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619925.139515:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619925.139529:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72c00/0x1cd7fe5368735c1d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ce44 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.139562:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735bf3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce21 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619925.139589:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f5500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589192192:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619925.139625:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164600 going to sleep for 11 seconds 00002000:00010000:0.0:1788619925.141917:0:6698:0:(ofd_lvb.c:284:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks: 519 -> 647 00000100:00100000:0.0:1788619925.141952:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69fa680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589192064:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619925.141988:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000052301317/0x1cd7fe5368735c16 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce4b expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.142013:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000052301317/0x1cd7fe5368735c16 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce4b expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.142028:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f51000, rc 0) 00000100:00100000:0.0:1788619925.142086:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69ff100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571594112:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3109us (3343us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619925.142095:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1048 00000100:00100000:2.0:1788619925.144745:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f5500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589192192:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619925.144772:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:2.0:1788619925.144792:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72c00/0x1cd7fe5368735c1d lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce44 expref: 18 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.144813:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619925.144816:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619925.144822:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72c00/0x1cd7fe5368735c1d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce44 expref: 18 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.144837:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f72c00, rc 0) 00000100:00100000:2.0:1788619925.144896:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2355:x1875503571593984:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5393us (6096us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619925.144919:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1274 00000100:00100000:3.0:1788619925.147899:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619925.147952:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619925.148038:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571594240 00000100:00100000:1.0:1788619925.148047:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571594240 00000100:00100000:1.0:1788619925.148109:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1275 00000100:00100000:1.0:1788619925.148121:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2353:x1875503571594240:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619925.148142:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619925.148163:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296400/0x1cd7fe5368735c24 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ce52 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.148202:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735bf3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce21 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619925.148243:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64d0700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589192320:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619925.148353:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7780 going to sleep for 11 seconds 00000100:00100000:2.0:1788619925.148551:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571594368 00000100:00100000:2.0:1788619925.148556:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571594368 00000100:00100000:2.0:1788619925.148592:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1049 00000100:00100000:2.0:1788619925.148600:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69ffb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571594368:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619925.148608:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619925.148620:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002cc077c3/0x1cd7fe5368735c2b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ce59 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619925.148667:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f5500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589192448:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619925.148700:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164600 going to sleep for 11 seconds 00000100:00100000:2.0:1788619925.151247:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f5500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589192448:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619925.151286:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002cc077c3/0x1cd7fe5368735c2b lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce59 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.151312:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002cc077c3/0x1cd7fe5368735c2b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce59 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.151355:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f72e00, rc 0) 00000100:00100000:2.0:1788619925.151413:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69ffb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571594368:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2813us (3459us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619925.151421:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1049 00000100:00100000:1.0:1788619925.152807:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64d0700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589192320:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619925.152832:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:1.0:1788619925.152850:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296400/0x1cd7fe5368735c24 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce52 expref: 18 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.152868:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619925.152871:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619925.152877:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296400/0x1cd7fe5368735c24 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce52 expref: 18 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.152898:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296400, rc 0) 00000100:00100000:1.0:1788619925.153156:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2353:x1875503571594240:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5037us (5265us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619925.153165:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1275 00000100:00100000:3.0:1788619925.156149:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619925.156276:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571594496 00000100:00100000:1.0:1788619925.156286:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571594496 00002000:00100000:1.0:1788619925.156370:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1527808->1531903) req@ffff8e77c69fe300 x1875503571594496/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 488/0 e 0 to 0 dl 1788619936 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619925.156381:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c69fe300 with x1875503571594496 ext(1527808->1531903) 00002000:00010000:1.0:1788619925.156387:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c69fe300 00000100:00100000:1.0:1788619925.156417:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 615 00000100:00100000:1.0:1788619925.156428:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571594496:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619925.156662:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571594496:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 236us (526us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619925.156670:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 615 00002000:00100000:1.0:1788619925.156677:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1527808->1531903) req@ffff8e77c69fe300 x1875503571594496/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 488/4536 e 0 to 0 dl 1788619936 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619925.156685:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c69fe300 with x1875503571594496 ext(1527808->1531903) 00002000:00010000:1.0:1788619925.156689:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c69fe300 00000100:00100000:3.0:1788619925.162506:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619925.162964:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571594752 00000100:00100000:1.0:1788619925.162979:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571594752 00000100:00100000:1.0:1788619925.163045:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1050 00000100:00100000:1.0:1788619925.163055:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f55c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571594752:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619925.163079:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619925.163098:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003086a8b7/0x1cd7fe5368735c32 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ce67 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619925.163163:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64d0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589192576:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619925.163470:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7780 going to sleep for 11 seconds 00000100:00100000:2.0:1788619925.164379:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619925.164527:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571594624 00000100:00100000:0.0:1788619925.164534:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571594624 00000100:00100000:0.0:1788619925.164585:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1276 00000100:00100000:0.0:1788619925.164594:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2355:x1875503571594624:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619925.164610:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619925.164631:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51000/0x1cd7fe5368735c39 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ce60 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.164678:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735bf3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce21 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619925.164729:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69fa680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589192704:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619925.164795:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590840 going to sleep for 11 seconds 00002000:00010000:1.0:1788619925.170186:0:6696:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime: 1788619924 -> 1788619925 00002000:00010000:1.0:1788619925.170196:0:6696:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime: 1788619924 -> 1788619925 00000100:00100000:1.0:1788619925.170225:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64d0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589192576:192.168.201.5@tcp:106: 00000004:00010000:0.0:1788619925.170367:0:9417:0:(mdt_lvb.c:183:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime: 1788619924 -> 1788619925 00000004:00010000:0.0:1788619925.170372:0:9417:0:(mdt_lvb.c:195:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime: 1788619924 -> 1788619925 00000100:00100000:0.0:1788619925.170390:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69fa680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589192704:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619925.170406:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:0.0:1788619925.170421:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51000/0x1cd7fe5368735c39 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce60 expref: 18 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.170441:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619925.170444:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619925.170450:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51000/0x1cd7fe5368735c39 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce60 expref: 18 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.170464:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f51000, rc 0) 00010000:00010000:1.0:1788619925.170482:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003086a8b7/0x1cd7fe5368735c32 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce67 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.170499:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003086a8b7/0x1cd7fe5368735c32 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce67 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.170508:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296400, rc 0) 00000100:00100000:0.0:1788619925.170537:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2355:x1875503571594624:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5944us (6160us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619925.170545:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1276 00000100:00100000:1.0:1788619925.170575:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f55c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571594752:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7522us (8079us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619925.170582:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1050 00000100:00100000:3.0:1788619925.173656:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619925.173707:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619925.173871:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571594880 00000100:00100000:2.0:1788619925.173881:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571594880 00000100:00100000:2.0:1788619925.173955:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1277 00000100:00100000:2.0:1788619925.173966:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f54700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2353:x1875503571594880:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619925.173987:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619925.174008:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72c00/0x1cd7fe5368735c40 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ce6e expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.174044:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735bf3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce21 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619925.174109:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f74700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589192832:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619925.174210:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164600 going to sleep for 11 seconds 00000100:00100000:1.0:1788619925.175041:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571595008 00000100:00100000:1.0:1788619925.175047:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571595008 00000100:00100000:1.0:1788619925.175092:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1051 00000100:00100000:1.0:1788619925.175100:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f55880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571595008:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619925.175111:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619925.175124:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c5e22f67/0x1cd7fe5368735c47 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ce75 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619925.175547:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64d0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589192960:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619925.175607:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7780 going to sleep for 11 seconds 00000100:00100000:2.0:1788619925.179630:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f74700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589192832:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619925.179670:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:2.0:1788619925.179691:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72c00/0x1cd7fe5368735c40 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce6e expref: 18 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.179713:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619925.179716:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619925.179722:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72c00/0x1cd7fe5368735c40 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce6e expref: 18 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.179744:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f72c00, rc 0) 00000100:00100000:2.0:1788619925.179808:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f54700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2353:x1875503571594880:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5844us (6169us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619925.179817:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1277 00000100:00100000:1.0:1788619925.182404:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64d0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589192960:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619925.182440:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c5e22f67/0x1cd7fe5368735c47 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce75 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.182464:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c5e22f67/0x1cd7fe5368735c47 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce75 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.182475:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297000, rc 0) 00000100:00100000:1.0:1788619925.182524:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f55880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571595008:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7425us (8817us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619925.182530:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1051 00000100:00100000:3.0:1788619925.187520:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619925.188449:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571595136 00000100:00100000:1.0:1788619925.188539:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571595136 00002000:00100000:1.0:1788619925.188590:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69d3b80 x1875503571595136/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 488/0 e 0 to 0 dl 1788619936 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619925.188600:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69d3b80 with x1875503571595136 00000100:00100000:1.0:1788619925.188685:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 778 00000100:00100000:1.0:1788619925.188696:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69d3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+15:2355:x1875503571595136:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619925.189111:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69d3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+15:2355:x1875503571595136:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 415us (1600us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619925.189120:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 778 00002000:00100000:1.0:1788619925.189127:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69d3b80 x1875503571595136/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 488/4536 e 0 to 0 dl 1788619936 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619925.189136:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69d3b80 with x1875503571595136 00000100:00100000:0.0:1788619925.206720:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619925.215960:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571595264 00000100:00100000:1.0:1788619925.215975:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571595264 00002000:00100000:1.0:1788619925.216039:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69fa680 x1875503571595264/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 488/0 e 0 to 0 dl 1788619936 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619925.216051:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69fa680 with x1875503571595264 00000100:00100000:1.0:1788619925.216092:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 779 00000100:00100000:1.0:1788619925.216102:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+15:2353:x1875503571595264:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619925.216392:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+15:2353:x1875503571595264:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 290us (9680us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619925.216399:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 779 00002000:00100000:1.0:1788619925.216405:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69fa680 x1875503571595264/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 488/4536 e 0 to 0 dl 1788619936 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619925.216413:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69fa680 with x1875503571595264 00000100:00100000:2.0:1788619925.225685:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619925.226264:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571595392 00000100:00100000:1.0:1788619925.226274:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571595392 00002000:00100000:1.0:1788619925.226588:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5f74700 x1875503571595392/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 488/0 e 0 to 0 dl 1788619936 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619925.226600:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5f74700 with x1875503571595392 00000100:00100000:1.0:1788619925.226683:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 780 00000100:00100000:1.0:1788619925.226695:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f74700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+15:2352:x1875503571595392:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619925.227839:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f74700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+15:2352:x1875503571595392:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 1142us (2161us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619925.227853:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 780 00002000:00100000:1.0:1788619925.227861:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5f74700 x1875503571595392/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 488/4536 e 0 to 0 dl 1788619936 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619925.227870:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5f74700 with x1875503571595392 00000100:00100000:3.0:1788619925.235269:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619925.241364:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571595520 00000100:00100000:0.0:1788619925.241379:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571595520 00002000:00100000:0.0:1788619925.241450:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69d2680 x1875503571595520/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 488/0 e 0 to 0 dl 1788619936 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619925.241473:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69d2680 with x1875503571595520 00000100:00100000:0.0:1788619925.241674:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 781 00000100:00100000:0.0:1788619925.241685:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69d2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+15:2353:x1875503571595520:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619925.242074:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69d2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+15:2353:x1875503571595520:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 390us (6815us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619925.242083:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 781 00002000:00100000:0.0:1788619925.242090:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69d2680 x1875503571595520/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 488/4536 e 0 to 0 dl 1788619936 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619925.242099:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69d2680 with x1875503571595520 00000100:00100000:3.0:1788619925.247270:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619925.247424:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571595648 00000100:00100000:0.0:1788619925.247436:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571595648 00002000:00100000:0.0:1788619925.247601:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69d1c00 x1875503571595648/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 488/0 e 0 to 0 dl 1788619936 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619925.247612:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69d1c00 with x1875503571595648 00000100:00100000:0.0:1788619925.247661:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 782 00000100:00100000:0.0:1788619925.247671:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69d1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+15:2353:x1875503571595648:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619925.248074:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69d1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+15:2353:x1875503571595648:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 403us (813us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619925.248082:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 782 00002000:00100000:0.0:1788619925.248090:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69d1c00 x1875503571595648/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 488/4536 e 0 to 0 dl 1788619936 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619925.248098:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69d1c00 with x1875503571595648 00000100:00100000:3.0:1788619925.253134:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619925.256300:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571595776 00000100:00100000:0.0:1788619925.256317:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571595776 00002000:00100000:0.0:1788619925.256385:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69d0e00 x1875503571595776/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 488/0 e 0 to 0 dl 1788619936 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619925.256398:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69d0e00 with x1875503571595776 00000100:00100000:0.0:1788619925.256437:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 783 00000100:00100000:0.0:1788619925.256448:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69d0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+15:2353:x1875503571595776:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619925.257137:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69d0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+15:2353:x1875503571595776:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 689us (4011us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619925.257146:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 783 00002000:00100000:0.0:1788619925.257167:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69d0e00 x1875503571595776/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 488/4536 e 0 to 0 dl 1788619936 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619925.257176:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69d0e00 with x1875503571595776 00000100:00100000:3.0:1788619925.262302:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619925.262658:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571595904 00000100:00100000:1.0:1788619925.262670:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571595904 00002000:00100000:1.0:1788619925.262722:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78cefaf450 x1875503571595904/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 488/0 e 0 to 0 dl 1788619936 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619925.262734:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78cefaf450 with x1875503571595904 00000100:00100000:1.0:1788619925.262774:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 784 00000100:00100000:1.0:1788619925.262784:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cefaf450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+15:2353:x1875503571595904:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619925.263061:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cefaf450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+15:2353:x1875503571595904:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 278us (768us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619925.263069:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 784 00002000:00100000:1.0:1788619925.263076:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78cefaf450 x1875503571595904/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 488/4536 e 0 to 0 dl 1788619936 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619925.263084:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78cefaf450 with x1875503571595904 00000100:00100000:3.0:1788619925.277128:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619925.277439:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571596032 00000100:00100000:0.0:1788619925.277449:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571596032 00002000:00100000:0.0:1788619925.277510:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69d1500 x1875503571596032/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 488/0 e 0 to 0 dl 1788619936 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619925.277521:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69d1500 with x1875503571596032 00000100:00100000:0.0:1788619925.277555:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 785 00000100:00100000:0.0:1788619925.277564:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69d1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+15:2354:x1875503571596032:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619925.277819:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69d1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+15:2354:x1875503571596032:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 254us (700us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619925.277827:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 785 00002000:00100000:0.0:1788619925.277834:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69d1500 x1875503571596032/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 488/4536 e 0 to 0 dl 1788619936 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619925.277842:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69d1500 with x1875503571596032 00000100:00100000:3.0:1788619925.281935:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619925.282083:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571596160 00000100:00100000:2.0:1788619925.282093:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571596160 00002000:00100000:2.0:1788619925.282141:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69d3480 x1875503571596160/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 488/0 e 0 to 0 dl 1788619936 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619925.282151:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69d3480 with x1875503571596160 00000100:00100000:2.0:1788619925.282198:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 786 00000100:00100000:2.0:1788619925.282208:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69d3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+15:2355:x1875503571596160:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619925.282588:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69d3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+15:2355:x1875503571596160:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 381us (663us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619925.282596:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 786 00002000:00100000:2.0:1788619925.282602:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69d3480 x1875503571596160/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 488/4536 e 0 to 0 dl 1788619936 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619925.282610:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69d3480 with x1875503571596160 00010000:00010000:3.0:1788619925.282869:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788619925.287918:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619925.288063:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571596288 00000100:00100000:2.0:1788619925.288073:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571596288 00002000:00100000:2.0:1788619925.288119:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69d1180 x1875503571596288/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 488/0 e 0 to 0 dl 1788619936 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619925.288130:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69d1180 with x1875503571596288 00000100:00100000:2.0:1788619925.288166:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 787 00000100:00100000:2.0:1788619925.288177:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69d1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+15:2354:x1875503571596288:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619925.288490:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69d1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+15:2354:x1875503571596288:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 313us (589us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619925.288498:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 787 00002000:00100000:2.0:1788619925.288506:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69d1180 x1875503571596288/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 488/4536 e 0 to 0 dl 1788619936 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619925.288520:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69d1180 with x1875503571596288 00000100:00100000:3.0:1788619925.294692:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619925.295194:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571596416 00000100:00100000:0.0:1788619925.295203:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571596416 00002000:00100000:0.0:1788619925.295252:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69d3800 x1875503571596416/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 488/0 e 0 to 0 dl 1788619936 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619925.295264:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69d3800 with x1875503571596416 00000100:00100000:0.0:1788619925.295305:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 788 00000100:00100000:0.0:1788619925.295315:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69d3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+15:2353:x1875503571596416:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619925.295612:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69d3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+15:2353:x1875503571596416:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 296us (928us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619925.295620:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 788 00002000:00100000:0.0:1788619925.295627:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69d3800 x1875503571596416/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 488/4536 e 0 to 0 dl 1788619936 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619925.295636:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69d3800 with x1875503571596416 00000100:00100000:0.0:1788619925.300263:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619925.300374:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571596544 00000100:00100000:3.0:1788619925.300385:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571596544 00002000:00100000:3.0:1788619925.300435:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69fb800 x1875503571596544/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 488/0 e 0 to 0 dl 1788619936 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619925.300446:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69fb800 with x1875503571596544 00000100:00100000:3.0:1788619925.300488:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 789 00000100:00100000:3.0:1788619925.300498:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+15:2352:x1875503571596544:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619925.300789:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+15:2352:x1875503571596544:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 291us (535us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619925.300797:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 789 00002000:00100000:3.0:1788619925.300805:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69fb800 x1875503571596544/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 488/4536 e 0 to 0 dl 1788619936 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619925.300813:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69fb800 with x1875503571596544 00000100:00100000:1.0:1788619925.304198:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619925.304295:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571596672 00000100:00100000:2.0:1788619925.304305:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571596672 00002000:00100000:2.0:1788619925.304392:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c64d0700 x1875503571596672/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 488/0 e 0 to 0 dl 1788619936 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619925.304403:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64d0700 with x1875503571596672 00000100:00100000:2.0:1788619925.304434:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 790 00000100:00100000:2.0:1788619925.304444:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64d0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+15:2353:x1875503571596672:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619925.304828:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64d0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+15:2353:x1875503571596672:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 384us (639us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619925.304836:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 790 00002000:00100000:2.0:1788619925.304844:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c64d0700 x1875503571596672/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 488/4536 e 0 to 0 dl 1788619936 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619925.304852:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64d0700 with x1875503571596672 00000100:00100000:1.0:1788619925.309418:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619925.309528:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571596800 00000100:00100000:0.0:1788619925.309539:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571596800 00002000:00100000:0.0:1788619925.309592:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c64d0000 x1875503571596800/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 488/0 e 0 to 0 dl 1788619936 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619925.309603:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64d0000 with x1875503571596800 00000100:00100000:0.0:1788619925.309647:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 791 00000100:00100000:0.0:1788619925.309658:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64d0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+15:2352:x1875503571596800:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619925.309955:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64d0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+15:2352:x1875503571596800:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 297us (546us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619925.309962:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 791 00002000:00100000:0.0:1788619925.309970:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c64d0000 x1875503571596800/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 488/4536 e 0 to 0 dl 1788619936 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619925.309979:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64d0000 with x1875503571596800 00000100:00100000:0.0:1788619925.313775:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619925.313925:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571596928 00000100:00100000:3.0:1788619925.313936:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571596928 00002000:00100000:3.0:1788619925.313985:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69f8a80 x1875503571596928/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 488/0 e 0 to 0 dl 1788619936 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619925.313997:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69f8a80 with x1875503571596928 00000100:00100000:3.0:1788619925.314032:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 792 00000100:00100000:3.0:1788619925.314042:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69f8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+15:2353:x1875503571596928:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:3.0:1788619925.318877:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 987136 -> 987136 00000004:00010000:3.0:1788619925.318887:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619925 -> 1788619925 00000004:00010000:3.0:1788619925.318890:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619924 -> 1788619924 00000004:00010000:3.0:1788619925.318892:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619925 -> 1788619925 00000004:00010000:3.0:1788619925.318894:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 902 -> 902 00000100:00100000:3.0:1788619925.319034:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69f8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+17:2353:x1875503571596928:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 4988us (5265us total) trans 4294967753 rc 0/0 00000100:00100000:3.0:1788619925.319047:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 792 00002000:00100000:3.0:1788619925.319059:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69f8a80 x1875503571596928/t4294967753(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 488/448 e 0 to 0 dl 1788619936 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619925.319070:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69f8a80 with x1875503571596928 00000100:00100000:1.0:1788619925.322125:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619925.322227:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571597056 00000100:00100000:0.0:1788619925.322236:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571597056 00000100:00100000:0.0:1788619925.322292:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1278 00000100:00100000:0.0:1788619925.322303:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64d0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+17:2355:x1875503571597056:12345-192.168.201.5@tcp:16:fsx.0 00000001:00080000:2.0:1788619925.360889:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967750 is committed 00000001:00080000:2.0:1788619925.361222:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967749 is committed 00000001:00080000:3.0:1788619925.361356:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967751 is committed 00000001:00080000:3.0:1788619925.361366:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967753 is committed 00000100:00100000:0.0:1788619925.361651:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64d0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571597056:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 39336us (39520us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619925.361663:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1278 00000100:00100000:1.0:1788619925.367176:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619925.368341:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571597184 00000100:00100000:0.0:1788619925.368354:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571597184 00002000:00100000:0.0:1788619925.368408:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1527808->1585151) req@ffff8e77c64d1880 x1875503571597184/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 488/0 e 0 to 0 dl 1788619936 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619925.368420:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64d1880 with x1875503571597184 ext(1527808->1585151) 00002000:00010000:0.0:1788619925.368426:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64d1880 00000100:00100000:0.0:1788619925.368615:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 616 00000100:00100000:0.0:1788619925.368624:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571597184:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:0.0:1788619925.372176:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+8:2353:x1875503571597184:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 3542us (4997us total) trans 4294967474 rc 0/0 00000100:00100000:0.0:1788619925.372190:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 616 00002000:00100000:0.0:1788619925.372203:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1527808->1585151) req@ffff8e77c64d1880 x1875503571597184/t4294967474(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 488/448 e 0 to 0 dl 1788619936 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619925.372213:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64d1880 with x1875503571597184 ext(1527808->1585151) 00002000:00010000:0.0:1788619925.372220:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64d1880 00000100:00100000:0.0:1788619925.376819:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619925.376900:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571597312 00000100:00100000:2.0:1788619925.376974:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571597312 00000100:00100000:2.0:1788619925.377092:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 617 00000100:00100000:2.0:1788619925.377104:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+8:2353:x1875503571597312:12345-192.168.201.5@tcp:22:fsx.0 00002000:00010000:2.0:1788619925.377353:0:6701:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 2092012 -> 2092012 00002000:00010000:2.0:1788619925.377357:0:6701:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619925 -> 1788619925 00002000:00010000:2.0:1788619925.377360:0:6701:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619924 -> 1788619924 00002000:00010000:2.0:1788619925.377362:0:6701:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619925 -> 1788619925 00002000:00010000:2.0:1788619925.377364:0:6701:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 647 -> 647 00000100:00100000:2.0:1788619925.377429:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571597312:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 327us (619us total) trans 4294967475 rc 0/0 00000100:00100000:2.0:1788619925.377436:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 617 00000100:00100000:2.0:1788619925.381810:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619925.381922:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571597440 00000100:00100000:0.0:1788619925.381932:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571597440 00000100:00100000:0.0:1788619925.382005:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 618 00000100:00100000:0.0:1788619925.382015:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f74000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571597440:12345-192.168.201.5@tcp:22:fsx.0 00002000:00010000:0.0:1788619925.382260:0:6703:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 2092012 -> 2092012 00002000:00010000:0.0:1788619925.382264:0:6703:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619925 -> 1788619925 00002000:00010000:0.0:1788619925.382267:0:6703:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619924 -> 1788619924 00002000:00010000:0.0:1788619925.382269:0:6703:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619925 -> 1788619925 00002000:00010000:0.0:1788619925.382272:0:6703:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 647 -> 647 00000100:00100000:0.0:1788619925.382787:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f74000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571597440:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 771us (983us total) trans 4294967476 rc 0/0 00000100:00100000:0.0:1788619925.382796:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 618 00000100:00100000:3.0:1788619925.385914:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619925.385946:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619925.386110:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571597696 00000100:00100000:2.0:1788619925.386120:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571597696 00000100:00100000:2.0:1788619925.386179:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1052 00000100:00100000:2.0:1788619925.386188:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69d2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571597696:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619925.386206:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619925.386223:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4727b52/0x1cd7fe5368735c4e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ce83 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619925.386278:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f74380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589193088:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619925.386379:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164c00 going to sleep for 11 seconds 00000100:00100000:0.0:1788619925.387321:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571597568 00000100:00100000:0.0:1788619925.387359:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571597568 00000100:00100000:0.0:1788619925.387399:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1279 00000100:00100000:0.0:1788619925.387408:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69d0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2353:x1875503571597568:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619925.387422:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619925.387437:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50c00/0x1cd7fe5368735c55 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ce7c expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.387578:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735bf3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce21 expref: 9 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619925.387613:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78cdcc2300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589193216:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619925.387688:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590a80 going to sleep for 11 seconds 00002000:00010000:3.0:1788619925.391708:0:6698:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619924 -> 1788619925 00000100:00100000:3.0:1788619925.391740:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f74380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589193088:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619925.391776:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a4727b52/0x1cd7fe5368735c4e lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce83 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619925.391807:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4727b52/0x1cd7fe5368735c4e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce83 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619925.391820:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f73a00, rc 0) 00000100:00100000:3.0:1788619925.391879:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69d2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571597696:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5692us (5931us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619925.391888:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1052 00000004:00010000:0.0:1788619925.392511:0:9417:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619924 -> 1788619925 00000004:00010000:0.0:1788619925.392515:0:9417:0:(mdt_lvb.c:201:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks: 902 -> 1158 00000100:00100000:0.0:1788619925.392532:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78cdcc2300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589193216:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619925.392550:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:0.0:1788619925.392562:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50c00/0x1cd7fe5368735c55 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce7c expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.392576:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619925.392579:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619925.392583:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50c00/0x1cd7fe5368735c55 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce7c expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.392594:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f50c00, rc 0) 00000100:00100000:0.0:1788619925.392643:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69d0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2353:x1875503571597568:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5228us (6740us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619925.392650:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1279 00000100:00100000:2.0:1788619925.397129:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619925.397213:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571597824 00000100:00100000:0.0:1788619925.397222:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571597824 00000100:00100000:0.0:1788619925.397278:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1280 00000100:00100000:0.0:1788619925.397288:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f75880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2353:x1875503571597824:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619925.397308:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619925.397363:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50e00/0x1cd7fe5368735c5c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ce8a expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.397403:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735bf3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce21 expref: 9 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619925.397452:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78cdcc2300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589193344:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619925.397543:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590a80 going to sleep for 11 seconds 00000100:00100000:3.0:1788619925.400129:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619925.400267:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571597952 00000100:00100000:2.0:1788619925.400275:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571597952 00000100:00100000:2.0:1788619925.400353:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1053 00000100:00100000:2.0:1788619925.400364:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69d2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571597952:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619925.400380:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619925.400397:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b040813e/0x1cd7fe5368735c63 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ce91 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619925.400455:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f76a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589193472:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619925.400548:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164a80 going to sleep for 11 seconds 00000100:00100000:0.0:1788619925.403028:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78cdcc2300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589193344:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619925.403056:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:0.0:1788619925.403074:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50e00/0x1cd7fe5368735c5c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce8a expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.403095:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619925.403098:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619925.403104:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50e00/0x1cd7fe5368735c5c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce8a expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.403119:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f50e00, rc 0) 00000100:00100000:0.0:1788619925.403180:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f75880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2353:x1875503571597824:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5894us (6061us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619925.403189:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1280 00000100:00100000:1.0:1788619925.406319:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f76a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589193472:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619925.406397:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b040813e/0x1cd7fe5368735c63 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce91 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.406429:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b040813e/0x1cd7fe5368735c63 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce91 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.406441:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f72600, rc 0) 00000100:00100000:1.0:1788619925.406514:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69d2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571597952:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6152us (6393us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619925.406523:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1053 00000100:00100000:3.0:1788619925.410058:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619925.410100:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619925.410158:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571598208 00000100:00100000:1.0:1788619925.410166:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571598208 00000100:00100000:1.0:1788619925.410215:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1054 00000100:00100000:1.0:1788619925.410223:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69d3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571598208:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619925.410238:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619925.410254:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000945af99a/0x1cd7fe5368735c6a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ce9f expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619925.410306:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64d2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589193600:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619925.410402:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7780 going to sleep for 11 seconds 00000100:00100000:0.0:1788619925.410814:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571598080 00000100:00100000:0.0:1788619925.410818:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571598080 00000100:00100000:0.0:1788619925.410852:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1281 00000100:00100000:0.0:1788619925.410859:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69d1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2353:x1875503571598080:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619925.410869:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619925.410881:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50e00/0x1cd7fe5368735c71 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ce98 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.410919:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735bf3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce21 expref: 9 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619925.410946:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78cdcc2300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589193728:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619925.410984:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590a80 going to sleep for 11 seconds 00000100:00100000:1.0:1788619925.414849:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64d2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589193600:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619925.414892:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000945af99a/0x1cd7fe5368735c6a lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce9f expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.414929:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000945af99a/0x1cd7fe5368735c6a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce9f expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.414948:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297400, rc 0) 00000100:00100000:1.0:1788619925.415008:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69d3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571598208:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4785us (4907us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619925.415017:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1054 00000100:00100000:0.0:1788619925.415595:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78cdcc2300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589193728:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619925.415615:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:0.0:1788619925.415629:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50e00/0x1cd7fe5368735c71 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce98 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.415653:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619925.415656:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619925.415661:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50e00/0x1cd7fe5368735c71 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce98 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.415672:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f50e00, rc 0) 00000100:00100000:0.0:1788619925.415783:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69d1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2353:x1875503571598080:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4923us (5734us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619925.415790:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1281 00000100:00100000:3.0:1788619925.420027:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619925.420114:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571598336 00000100:00100000:0.0:1788619925.420123:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571598336 00000100:00100000:0.0:1788619925.420185:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1282 00000100:00100000:0.0:1788619925.420195:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69d0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2353:x1875503571598336:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619925.420216:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619925.420236:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50c00/0x1cd7fe5368735c78 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cea6 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.420275:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735bf3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce21 expref: 9 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619925.420317:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78cdcc2300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589193856:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619925.420425:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590a80 going to sleep for 11 seconds 00000100:00100000:2.0:1788619925.422217:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619925.422302:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571598464 00000100:00100000:1.0:1788619925.422310:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571598464 00000100:00100000:1.0:1788619925.422391:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1055 00000100:00100000:1.0:1788619925.422401:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f77b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571598464:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619925.422415:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619925.422431:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000945af99a/0x1cd7fe5368735c7f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cead expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619925.422497:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64d2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589193984:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619925.422567:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7780 going to sleep for 11 seconds 00000100:00100000:0.0:1788619925.425645:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78cdcc2300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589193856:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619925.425672:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:0.0:1788619925.425701:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50c00/0x1cd7fe5368735c78 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cea6 expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.425722:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619925.425725:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619925.425731:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50c00/0x1cd7fe5368735c78 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cea6 expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.425745:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f50c00, rc 0) 00000100:00100000:0.0:1788619925.425805:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69d0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2353:x1875503571598336:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5611us (5788us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619925.425815:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1282 00000100:00100000:1.0:1788619925.426819:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64d2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589193984:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619925.426844:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000945af99a/0x1cd7fe5368735c7f lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cead expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.426861:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000945af99a/0x1cd7fe5368735c7f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cead expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.426872:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297400, rc 0) 00000100:00100000:1.0:1788619925.426923:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f77b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571598464:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4523us (4712us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619925.426931:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1055 00000100:00100000:3.0:1788619925.437625:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619925.437848:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571598592 00000100:00100000:2.0:1788619925.437858:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571598592 00002000:00100000:2.0:1788619925.437917:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69d0000 x1875503571598592/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 488/0 e 0 to 0 dl 1788619936 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619925.437929:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69d0000 with x1875503571598592 00000100:00100000:2.0:1788619925.437967:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 793 00000100:00100000:2.0:1788619925.437976:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69d0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571598592:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619925.438241:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69d0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571598592:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 264us (625us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619925.438249:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 793 00002000:00100000:2.0:1788619925.438257:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69d0000 x1875503571598592/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 488/4536 e 0 to 0 dl 1788619936 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619925.438265:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69d0000 with x1875503571598592 00000100:00100000:2.0:1788619925.442384:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619925.442513:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571598720 00000100:00100000:3.0:1788619925.442522:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571598720 00002000:00100000:3.0:1788619925.442569:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5f76680 x1875503571598720/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 488/0 e 0 to 0 dl 1788619936 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619925.442582:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5f76680 with x1875503571598720 00000100:00100000:3.0:1788619925.442617:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 794 00000100:00100000:3.0:1788619925.442627:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f76680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571598720:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619925.443163:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f76680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571598720:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 535us (787us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619925.443171:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 794 00002000:00100000:3.0:1788619925.443178:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5f76680 x1875503571598720/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 488/4536 e 0 to 0 dl 1788619936 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619925.443193:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5f76680 with x1875503571598720 00000100:00100000:3.0:1788619925.447845:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619925.447898:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619925.448181:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571598976 00000100:00100000:2.0:1788619925.448191:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571598976 00000100:00100000:2.0:1788619925.448256:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1056 00000100:00100000:2.0:1788619925.448267:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69d0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571598976:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619925.448288:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619925.448308:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a802d33a/0x1cd7fe5368735c86 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cebb expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619925.448411:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f75c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589194112:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619925.448504:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81640c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619925.448805:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571598848 00000100:00100000:0.0:1788619925.448810:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571598848 00000100:00100000:0.0:1788619925.448851:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1283 00000100:00100000:0.0:1788619925.448859:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69d1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571598848:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619925.448870:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619925.448883:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50e00/0x1cd7fe5368735c8d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ceb4 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.448924:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735bf3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce21 expref: 9 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619925.448953:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78cdcc2300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589194240:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619925.448981:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590a80 going to sleep for 11 seconds 00000100:00100000:2.0:1788619925.452277:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78cdcc2300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589194240:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619925.452306:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:2.0:1788619925.452360:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50e00/0x1cd7fe5368735c8d lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ceb4 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.452383:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619925.452386:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619925.452392:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50e00/0x1cd7fe5368735c8d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ceb4 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.452408:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f50e00, rc 0) 00000100:00100000:2.0:1788619925.452625:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69d1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571598848:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3615us (4574us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619925.452644:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1283 00000100:00100000:2.0:1788619925.456144:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f75c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589194112:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619925.456186:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a802d33a/0x1cd7fe5368735c86 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cebb expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.456211:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a802d33a/0x1cd7fe5368735c86 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cebb expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.456224:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f72200, rc 0) 00000100:00100000:2.0:1788619925.456291:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69d0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571598976:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8026us (8454us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619925.456301:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1056 00000100:00100000:2.0:1788619925.459914:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619925.459960:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619925.460089:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571599232 00000100:00100000:1.0:1788619925.460099:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571599232 00000100:00100000:1.0:1788619925.460159:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1057 00000100:00100000:1.0:1788619925.460171:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f75c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571599232:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619925.460188:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619925.460206:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000945af99a/0x1cd7fe5368735c94 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cec9 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619925.460272:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64d2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589194368:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619925.460377:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7780 going to sleep for 11 seconds 00000100:00100000:1.0:1788619925.463233:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64d2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589194368:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619925.463271:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000945af99a/0x1cd7fe5368735c94 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cec9 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.463297:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000945af99a/0x1cd7fe5368735c94 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cec9 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.463309:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297400, rc 0) 00000100:00100000:1.0:1788619925.463401:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f75c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571599232:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3232us (3506us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619925.463411:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1057 00000100:00100000:0.0:1788619925.465717:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571599104 00000100:00100000:0.0:1788619925.465731:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571599104 00000100:00100000:1.0:1788619925.468148:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1284 00000100:00100000:1.0:1788619925.468166:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f74e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2355:x1875503571599104:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619925.468191:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619925.468212:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297c00/0x1cd7fe5368735c9b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cec2 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.468255:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735bf3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce21 expref: 9 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619925.468298:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64d2a00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589194496:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619925.468418:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7780 going to sleep for 11 seconds 00000100:00100000:1.0:1788619925.470861:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64d2a00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589194496:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619925.470886:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:1.0:1788619925.470914:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297c00/0x1cd7fe5368735c9b lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cec2 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.470935:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619925.470938:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619925.470944:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297c00/0x1cd7fe5368735c9b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cec2 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.470960:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297c00, rc 0) 00000100:00100000:1.0:1788619925.471024:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f74e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2355:x1875503571599104:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2862us (11063us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619925.471663:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1284 00000100:00100000:3.0:1788619925.483708:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619925.483757:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619925.483819:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571599488 00000100:00100000:0.0:1788619925.483831:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571599488 00002000:00100000:0.0:1788619925.483883:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69d3b80 x1875503571599488/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 520/0 e 0 to 0 dl 1788619936 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619925.483894:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69d3b80 with x1875503571599488 00000100:00100000:0.0:1788619925.483950:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 795 00000100:00100000:0.0:1788619925.483960:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69d3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571599488:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619925.485001:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571599360 00000100:00100000:2.0:1788619925.485009:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571599360 00002000:00100000:2.0:1788619925.485043:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1048576->2093055) req@ffff8e77c69d2a00 x1875503571599360/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 520/0 e 0 to 0 dl 1788619936 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619925.485059:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c69d2a00 with x1875503571599360 ext(1048576->2093055) 00002000:00010000:2.0:1788619925.485064:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c69d2a00 00000100:00100000:2.0:1788619925.485093:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 619 00000100:00100000:2.0:1788619925.485101:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69d2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571599360:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619925.498767:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69d3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571599488:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 14797us (14998us total) trans 0 rc 864256/864256 00000100:00100000:3.0:1788619925.498782:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 795 00002000:00100000:3.0:1788619925.498796:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69d3b80 x1875503571599488/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 520/440 e 0 to 0 dl 1788619936 ref 1 fl Complete:/600/0 rc 864256/864256 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619925.498807:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69d3b80 with x1875503571599488 00000100:00100000:2.0:1788619925.506120:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69d2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571599360:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 21008us (22412us total) trans 0 rc 958464/958464 00000100:00100000:2.0:1788619925.506136:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 619 00002000:00100000:2.0:1788619925.506150:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1048576->2093055) req@ffff8e77c69d2a00 x1875503571599360/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 520/440 e 0 to 0 dl 1788619936 ref 1 fl Complete:/600/0 rc 958464/958464 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619925.506160:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c69d2a00 with x1875503571599360 ext(1048576->2093055) 00002000:00010000:2.0:1788619925.506166:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c69d2a00 00000100:00100000:2.0:1788619925.514258:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619925.514292:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619925.514352:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571599744 00000100:00100000:1.0:1788619925.514364:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571599744 00000100:00100000:0.0:1788619925.514368:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571599616 00000100:00100000:0.0:1788619925.514373:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571599616 00000100:00100000:1.0:1788619925.514425:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1058 00000100:00100000:0.0:1788619925.514425:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1285 00000100:00100000:1.0:1788619925.514435:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f77480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571599744:12345-192.168.201.5@tcp:101:fsx.0 00000100:00100000:0.0:1788619925.514435:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f75500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571599616:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619925.514453:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619925.514453:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619925.514474:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5600/0x1cd7fe5368735ca2 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ced0 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.514512:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735bf3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce21 expref: 9 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619925.514553:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78cdcc1880 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589194624:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619925.514588:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005b4a2f71/0x1cd7fe5368735ca9 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ced7 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619925.514626:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64d2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589194752:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619925.514818:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7780 going to sleep for 11 seconds 00000100:00100000:0.0:1788619925.516271:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590b40 going to sleep for 11 seconds 00000100:00100000:1.0:1788619925.522090:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64d2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589194752:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619925.522130:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005b4a2f71/0x1cd7fe5368735ca9 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ced7 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.522155:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005b4a2f71/0x1cd7fe5368735ca9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ced7 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.522168:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297c00, rc 0) 00000100:00100000:1.0:1788619925.522231:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f77480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571599744:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7796us (7935us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619925.522240:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1058 00000100:00100000:1.0:1788619925.526242:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78cdcc1880 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589194624:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619925.526273:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:1.0:1788619925.526293:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5600/0x1cd7fe5368735ca2 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ced0 expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.526316:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619925.526319:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619925.527513:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5600/0x1cd7fe5368735ca2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ced0 expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.527540:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d5600, rc 0) 00000100:00100000:1.0:1788619925.527799:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f75500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571599616:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 13362us (13545us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619925.527808:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1285 00000100:00100000:2.0:1788619925.537732:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619925.537775:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619925.537869:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571600000 00000100:00100000:1.0:1788619925.537889:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571600000 00000100:00100000:1.0:1788619925.537954:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1059 00000100:00100000:1.0:1788619925.537964:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f75180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571600000:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619925.537982:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619925.538002:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005b4a2f71/0x1cd7fe5368735cb0 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cee5 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619925.538063:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64d2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589194880:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619925.538137:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7780 going to sleep for 11 seconds 00000100:00100000:0.0:1788619925.539017:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571599872 00000100:00100000:0.0:1788619925.539024:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571599872 00000100:00100000:0.0:1788619925.539071:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1286 00000100:00100000:0.0:1788619925.539080:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f77800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2355:x1875503571599872:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619925.539093:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619925.539108:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4000/0x1cd7fe5368735cb7 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cede expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.539142:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735bf3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce21 expref: 9 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619925.539171:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78cdcc3100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589195008:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619925.539233:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c85903c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619925.542053:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64d2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589194880:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619925.542093:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005b4a2f71/0x1cd7fe5368735cb0 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cee5 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.542118:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005b4a2f71/0x1cd7fe5368735cb0 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cee5 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.542131:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297c00, rc 0) 00000100:00100000:1.0:1788619925.542190:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f75180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571600000:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4227us (4467us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619925.542200:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1059 00000100:00100000:0.0:1788619925.543693:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78cdcc3100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589195008:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619925.543718:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:0.0:1788619925.543790:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4000/0x1cd7fe5368735cb7 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cede expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.543809:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619925.543812:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619925.543817:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4000/0x1cd7fe5368735cb7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cede expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.543831:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d4000, rc 0) 00000100:00100000:0.0:1788619925.544102:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f77800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2355:x1875503571599872:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5021us (6324us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619925.544111:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1286 00000100:00100000:2.0:1788619925.546281:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619925.551039:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571600128 00000100:00100000:1.0:1788619925.551054:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571600128 00000100:00100000:1.0:1788619925.551187:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1287 00000100:00100000:1.0:1788619925.551198:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f74a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+12:14908:x1875503571600128:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619925.551221:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619925.551244:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe5368735cbe lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ceec expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.551276:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735bf3 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce21 expref: 9 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.551287:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe5368735cbe lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ceec expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.551348:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735bf3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce21 expref: 9 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.551372:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735bf3 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce21 expref: 9 pid: 5844 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619925.551385:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64d2a00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589195136:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619925.551484:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7780 going to sleep for 11 seconds 00000100:00100000:3.0:1788619925.553921:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619925.554043:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571600256 00000100:00100000:0.0:1788619925.554052:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571600256 00002000:00100000:0.0:1788619925.554103:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69d2680 x1875503571600256/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 488/0 e 0 to 0 dl 1788619936 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619925.554114:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69d2680 with x1875503571600256 00010000:00010000:0.0:1788619925.554122:0:5859:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735bf3 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce21 expref: 10 pid: 5844 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.554136:0:5859:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735bf3 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce21 expref: 10 pid: 5844 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788619925.554144:0:5859:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e77c69d2680 00000100:00100000:0.0:1788619925.554177:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 33 00000100:00100000:0.0:1788619925.554187:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69d2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571600256:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:1.0:1788619925.554442:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64d2a00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589195136:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619925.554480:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe5368735cbe lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ceec expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.554501:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619925.554505:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619925.554509:0:5845:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe5368735cbe lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ceec expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.554519:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297a00, rc 0) 00000100:00100000:1.0:1788619925.554552:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f74a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571600128:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3356us (8281us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619925.554560:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1287 00000004:00010000:0.0:1788619925.560571:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 987136 -> 987136 00000004:00010000:0.0:1788619925.560591:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619925 -> 1788619925 00000004:00010000:0.0:1788619925.560595:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619925 -> 1788619924 00000004:00010000:0.0:1788619925.560598:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619925 -> 1788619925 00000004:00010000:0.0:1788619925.560600:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1158 -> 1158 00000100:00100000:0.0:1788619925.560699:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69d2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571600256:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 6508us (6782us total) trans 4294967754 rc 0/0 00000100:00100000:0.0:1788619925.560711:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 33 00002000:00100000:0.0:1788619925.560723:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69d2680 x1875503571600256/t4294967754(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 488/448 e 0 to 0 dl 1788619936 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619925.560734:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69d2680 with x1875503571600256 00010000:00010000:0.0:1788619925.560742:0:5859:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735bf3 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce21 expref: 12 pid: 5844 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.560765:0:5859:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735bf3 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce21 expref: 12 pid: 5844 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788619925.560772:0:5859:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e77c69d2680 00000100:00100000:2.0:1788619925.566093:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619925.566179:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571600384 00000100:00100000:1.0:1788619925.566190:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571600384 00010000:00010000:1.0:1788619925.566232:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735bf3 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce21 expref: 12 pid: 5844 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619925.566271:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 430 00000100:00100000:1.0:1788619925.566281:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f76d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571600384:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619925.566295:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619925.566350:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735bf3 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce21 expref: 12 pid: 5844 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.566365:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735bf3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce21 expref: 12 pid: 5844 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.566388:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735bf3 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce21 expref: 12 pid: 5844 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.566403:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619925.566408:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe5368735cbe lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ceec expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.566427:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe5368735cbe lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ceec expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.566433:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe5368735cbe lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ceec expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.566474:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe5368735cbe lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ceec expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619925.566487:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64d2a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589195264:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619925.566569:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7780 going to sleep for 11 seconds 00000100:00100000:3.0:1788619925.570064:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619925.570422:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64d2a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589195264:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619925.570446:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619925.570740:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f76d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571600384:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 4458us (4655us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619925.570752:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 430 00000100:00100000:2.0:1788619925.570761:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571600512 00000100:00100000:2.0:1788619925.570768:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571600512 00002000:00100000:2.0:1788619925.570814:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69d1c00 x1875503571600512/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:21/0 lens 488/0 e 0 to 0 dl 1788619936 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619925.570823:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69d1c00 with x1875503571600512 00000100:00100000:2.0:1788619925.570863:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 796 00000100:00100000:2.0:1788619925.570883:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69d1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571600512:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619925.571083:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69d1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571600512:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 210us (1029us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619925.571090:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 796 00002000:00100000:2.0:1788619925.571097:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69d1c00 x1875503571600512/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:21/0 lens 488/4536 e 0 to 0 dl 1788619936 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619925.571105:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69d1c00 with x1875503571600512 00000100:00100000:3.0:1788619925.574061:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619925.574182:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571600640 00000100:00100000:1.0:1788619925.574191:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571600640 00002000:00100000:1.0:1788619925.574235:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69d0e00 x1875503571600640/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:21/0 lens 488/0 e 0 to 0 dl 1788619936 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619925.574245:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69d0e00 with x1875503571600640 00000100:00100000:1.0:1788619925.574290:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 797 00000100:00100000:1.0:1788619925.574300:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69d0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571600640:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619925.574603:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69d0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571600640:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 303us (552us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619925.574611:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 797 00002000:00100000:3.0:1788619925.574618:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69d0e00 x1875503571600640/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:21/0 lens 488/4536 e 0 to 0 dl 1788619936 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619925.574626:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69d0e00 with x1875503571600640 00000100:00100000:3.0:1788619925.578272:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619925.578676:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571600768 00000100:00100000:1.0:1788619925.578686:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571600768 00000100:00100000:3.0:1788619925.579065:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1288 00000100:00100000:3.0:1788619925.579078:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dc8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571600768:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619925.579101:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619925.579122:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70a00/0x1cd7fe5368735cc5 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cef3 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619925.579160:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe5368735cbe lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ceec expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619925.579199:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5dc8000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589195392:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619925.579285:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb540 going to sleep for 11 seconds 00000004:00010000:3.0:1788619925.582433:0:9417:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619924 -> 1788619925 00000100:00100000:3.0:1788619925.582482:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5dc8000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589195392:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619925.582506:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:3.0:1788619925.582526:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70a00/0x1cd7fe5368735cc5 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cef3 expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619925.582548:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619925.582551:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619925.582557:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70a00/0x1cd7fe5368735cc5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cef3 expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619925.582573:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f70a00, rc 0) 00000100:00100000:3.0:1788619925.582643:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dc8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571600768:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3558us (4371us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619925.582653:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1288 00000100:00100000:0.0:1788619925.586247:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619925.586367:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571600896 00000100:00100000:2.0:1788619925.586377:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571600896 00000100:00100000:2.0:1788619925.586434:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1289 00000100:00100000:2.0:1788619925.586445:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571600896:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619925.586464:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619925.586497:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72600/0x1cd7fe5368735ccc lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cefa expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.586536:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe5368735cbe lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ceec expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619925.586577:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f76300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589195520:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619925.586686:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164000 going to sleep for 11 seconds 00000100:00100000:0.0:1788619925.589470:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f76300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589195520:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619925.589496:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:0.0:1788619925.589524:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72600/0x1cd7fe5368735ccc lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cefa expref: 12 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.589545:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619925.589548:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619925.589554:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72600/0x1cd7fe5368735ccc lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cefa expref: 12 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.589569:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f72600, rc 0) 00000100:00100000:0.0:1788619925.589674:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571600896:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3230us (3436us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619925.589682:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1289 00000100:00100000:0.0:1788619925.593267:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619925.593375:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571601024 00000100:00100000:2.0:1788619925.593385:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571601024 00000100:00100000:2.0:1788619925.593443:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1060 00000100:00100000:2.0:1788619925.593453:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571601024:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619925.593480:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619925.593497:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007f9cdd10/0x1cd7fe5368735cd3 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cf01 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619925.593557:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f75f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589195648:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619925.593651:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164540 going to sleep for 11 seconds 00000100:00100000:2.0:1788619925.595708:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f75f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589195648:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619925.595746:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007f9cdd10/0x1cd7fe5368735cd3 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf01 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.595771:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f9cdd10/0x1cd7fe5368735cd3 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf01 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.595783:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f72000, rc 0) 00000100:00100000:2.0:1788619925.595834:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571601024:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2382us (2576us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619925.595844:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1060 00000100:00100000:0.0:1788619925.599518:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619925.599681:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571601152 00000100:00100000:3.0:1788619925.599700:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571601152 00000100:00100000:3.0:1788619925.599755:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1061 00000100:00100000:3.0:1788619925.599766:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571601152:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619925.599785:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619925.599802:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071401568/0x1cd7fe5368735cda lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cf08 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619925.599862:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5dc8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589195776:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619925.600057:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb540 going to sleep for 11 seconds 00000100:00100000:3.0:1788619925.602651:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589195904:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1788619925.602724:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619925.602820:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589195904 00000100:00100000:1.0:1788619925.602828:0:6699:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589195904 00000100:00100000:1.0:1788619925.602876:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 91 00000100:00100000:1.0:1788619925.602886:0:6699:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589195904:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1788619925.602918:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 143065088 pending 1703936 free 3859808256 avail 3855613952 00000100:00100000:1.0:1788619925.602976:0:6699:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589195904:12345-0@lo:13:osp-pre-0-0.0 Request processed in 91us (251us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619925.602983:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 91 00000100:00100000:1.0:1788619925.603029:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589195904:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1788619925.603654:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5dc8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589195776:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619925.603683:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000071401568/0x1cd7fe5368735cda lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf08 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619925.603716:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071401568/0x1cd7fe5368735cda lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf08 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619925.603729:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f70a00, rc 0) 00000100:00100000:3.0:1788619925.603775:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571601152:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4011us (4267us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619925.603782:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1061 00000100:00100000:0.0:1788619925.606859:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619925.606949:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571601280 00000100:00100000:1.0:1788619925.606958:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571601280 00000100:00100000:1.0:1788619925.607052:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1290 00000100:00100000:1.0:1788619925.607062:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571601280:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619925.607083:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619925.607101:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297c00/0x1cd7fe5368735ce1 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cf0f expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.607135:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe5368735cbe lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ceec expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619925.607169:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64d2a00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589196032:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619925.607240:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7780 going to sleep for 11 seconds 00000100:00100000:1.0:1788619925.609619:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64d2a00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589196032:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619925.609652:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:1.0:1788619925.609671:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297c00/0x1cd7fe5368735ce1 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf0f expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.609701:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619925.609704:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619925.609710:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297c00/0x1cd7fe5368735ce1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf0f expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.609722:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297c00, rc 0) 00000100:00100000:1.0:1788619925.609775:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571601280:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2714us (2924us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619925.609783:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1290 00000100:00100000:0.0:1788619925.612230:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619925.613009:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571601408 00000100:00100000:2.0:1788619925.613019:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571601408 00000100:00100000:2.0:1788619925.613080:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1291 00000100:00100000:2.0:1788619925.613090:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571601408:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619925.613111:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619925.613130:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72000/0x1cd7fe5368735ce8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cf16 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.613169:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe5368735cbe lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ceec expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619925.613213:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f75f80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589196160:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619925.613292:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164540 going to sleep for 11 seconds 00000100:00100000:2.0:1788619925.615780:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f75f80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589196160:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619925.615807:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:2.0:1788619925.615825:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72000/0x1cd7fe5368735ce8 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf16 expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.615845:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619925.615848:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619925.615854:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72000/0x1cd7fe5368735ce8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf16 expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.615867:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f72000, rc 0) 00000100:00100000:2.0:1788619925.616132:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571601408:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3042us (3908us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619925.616141:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1291 00000100:00100000:2.0:1788619925.623104:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619925.623189:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571601536 00000100:00100000:0.0:1788619925.623199:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571601536 00000100:00100000:0.0:1788619925.623285:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1292 00000100:00100000:0.0:1788619925.623296:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f75f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571601536:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619925.623317:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619925.623361:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5000/0x1cd7fe5368735cef lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cf1d expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.623400:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe5368735cbe lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ceec expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619925.623442:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78cdcc2300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589196288:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619925.623531:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c85903c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619925.626049:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78cdcc2300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589196288:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619925.626074:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:1.0:1788619925.626092:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5000/0x1cd7fe5368735cef lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf1d expref: 12 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.626113:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619925.626116:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619925.626122:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5000/0x1cd7fe5368735cef lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf1d expref: 12 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.626138:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d5000, rc 0) 00000100:00100000:1.0:1788619925.626228:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f75f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571601536:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2933us (3134us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619925.626237:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1292 00000100:00100000:1.0:1788619925.628758:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619925.628835:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571601664 00000100:00100000:2.0:1788619925.628844:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571601664 00000100:00100000:2.0:1788619925.629054:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1293 00000100:00100000:2.0:1788619925.629065:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64d2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571601664:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619925.629084:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619925.629103:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72a00/0x1cd7fe5368735cf6 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cf24 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.629145:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe5368735cbe lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ceec expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619925.629186:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f77100 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589196416:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619925.629266:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164540 going to sleep for 11 seconds 00000100:00100000:1.0:1788619925.631121:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f77100 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589196416:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619925.631145:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:1.0:1788619925.631160:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72a00/0x1cd7fe5368735cf6 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf24 expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.631179:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619925.631182:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619925.631188:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72a00/0x1cd7fe5368735cf6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf24 expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.631201:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f72a00, rc 0) 00000100:00100000:1.0:1788619925.631255:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64d2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571601664:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2192us (2573us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619925.631264:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1293 00000100:00100000:1.0:1788619925.633730:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619925.633853:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571601792 00000100:00100000:3.0:1788619925.633864:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571601792 00000100:00100000:3.0:1788619925.633925:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1062 00000100:00100000:3.0:1788619925.633936:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64d3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571601792:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619925.633955:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619925.633973:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071401568/0x1cd7fe5368735cfd lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cf2b expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619925.634031:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5dc8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589196544:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619925.634118:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb540 going to sleep for 11 seconds 00000100:00100000:3.0:1788619925.637116:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5dc8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589196544:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619925.637156:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000071401568/0x1cd7fe5368735cfd lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf2b expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619925.637182:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071401568/0x1cd7fe5368735cfd lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf2b expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619925.637196:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f70a00, rc 0) 00000100:00100000:3.0:1788619925.637255:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64d3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571601792:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3320us (3533us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619925.637264:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1062 00000100:00100000:0.0:1788619925.640218:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619925.640311:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571601920 00000100:00100000:1.0:1788619925.640320:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571601920 00000100:00100000:1.0:1788619925.640414:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1063 00000100:00100000:1.0:1788619925.640427:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571601920:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619925.640447:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619925.640479:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6252eaa/0x1cd7fe5368735d04 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cf32 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619925.640544:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64d2680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589196672:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619925.640626:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7780 going to sleep for 11 seconds 00000100:00100000:2.0:1788619925.643070:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64d2680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589196672:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619925.643108:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a6252eaa/0x1cd7fe5368735d04 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf32 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.643132:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6252eaa/0x1cd7fe5368735d04 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf32 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.643145:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297600, rc 0) 00000100:00100000:2.0:1788619925.643203:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571601920:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2779us (3000us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619925.643212:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1063 00000100:00100000:2.0:1788619925.647686:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619925.647835:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571602048 00000100:00100000:3.0:1788619925.647845:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571602048 00000100:00100000:3.0:1788619925.647919:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1064 00000100:00100000:3.0:1788619925.647929:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f77100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571602048:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619925.647948:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619925.647964:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071401568/0x1cd7fe5368735d0b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cf39 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619925.648022:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5dc8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589196800:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619925.648102:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb540 going to sleep for 11 seconds 00000100:00100000:3.0:1788619925.651194:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5dc8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589196800:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619925.651232:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000071401568/0x1cd7fe5368735d0b lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf39 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619925.651257:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071401568/0x1cd7fe5368735d0b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf39 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619925.651271:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f70a00, rc 0) 00000100:00100000:3.0:1788619925.651377:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f77100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571602048:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3448us (3700us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619925.651386:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1064 00000100:00100000:0.0:1788619925.655115:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619925.655252:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571602176 00000100:00100000:3.0:1788619925.655262:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571602176 00000100:00100000:3.0:1788619925.655315:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1065 00000100:00100000:3.0:1788619925.655352:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f54e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571602176:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619925.655370:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619925.655387:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000033c29ece/0x1cd7fe5368735d12 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cf40 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619925.655445:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5dc8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589196928:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619925.655529:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb540 going to sleep for 11 seconds 00000100:00100000:3.0:1788619925.657959:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5dc8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589196928:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619925.658004:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000033c29ece/0x1cd7fe5368735d12 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf40 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619925.658038:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000033c29ece/0x1cd7fe5368735d12 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf40 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619925.658052:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f70600, rc 0) 00000100:00100000:3.0:1788619925.658115:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f54e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571602176:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2791us (3009us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619925.658123:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1065 00000100:00100000:0.0:1788619925.661651:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619925.661823:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571602304 00000100:00100000:3.0:1788619925.661834:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571602304 00000100:00100000:3.0:1788619925.661889:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1294 00000100:00100000:3.0:1788619925.662141:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63f8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571602304:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619925.662163:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619925.662185:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70a00/0x1cd7fe5368735d19 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cf47 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619925.662226:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe5368735cbe lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ceec expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619925.662265:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5dc8000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589197056:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619925.662383:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb540 going to sleep for 11 seconds 00000100:00100000:3.0:1788619925.665868:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5dc8000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589197056:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619925.665895:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:3.0:1788619925.666087:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70a00/0x1cd7fe5368735d19 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf47 expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619925.666115:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619925.666118:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619925.666124:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70a00/0x1cd7fe5368735d19 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf47 expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619925.666139:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f70a00, rc 0) 00000100:00100000:3.0:1788619925.666200:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63f8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571602304:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4302us (4568us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619925.666209:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1294 00000100:00100000:2.0:1788619925.669146:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619925.671108:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571602432 00000100:00100000:1.0:1788619925.671121:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571602432 00000100:00100000:1.0:1788619925.671223:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1295 00000100:00100000:1.0:1788619925.671233:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f76300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571602432:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619925.671252:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619925.671272:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe5368735d20 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cf4e expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.671308:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe5368735cbe lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ceec expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619925.671372:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64d1c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589197184:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619925.671484:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7780 going to sleep for 11 seconds 00000100:00100000:1.0:1788619925.674417:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64d1c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589197184:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619925.674446:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:1.0:1788619925.674475:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe5368735d20 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf4e expref: 12 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.674495:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619925.674498:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619925.674504:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe5368735d20 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf4e expref: 12 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.674518:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297e00, rc 0) 00000100:00100000:1.0:1788619925.674567:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f76300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571602432:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3334us (5430us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619925.674575:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1295 00000100:00100000:2.0:1788619925.678168:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619925.678357:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571602688 00000100:00100000:3.0:1788619925.678368:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571602688 00000100:00100000:3.0:1788619925.678424:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1296 00000100:00100000:3.0:1788619925.678434:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f76a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571602688:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:3.0:1788619925.678496:0:11397:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619925.678525:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368735b2f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cd64 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619925.678544:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619925.678548:0:11397:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788619925.678771:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70a00/0x1cd7fe5368735d27 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619925.678786:0:11397:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50600/0x1cd7fe5368735bc9 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cdf7 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619925.678796:0:11397:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70a00/0x1cd7fe5368735d27 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619925.678828:0:11397:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50600/0x1cd7fe5368735bc9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cdf7 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619925.678842:0:11397:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50600/0x1cd7fe5368735bc9 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cdf7 expref: 10 pid: 5846 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619925.678855:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5dc8000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589197312:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619925.678946:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb540 going to sleep for 11 seconds 00000100:00100000:2.0:1788619925.682478:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619925.682920:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571602816 00000100:00100000:0.0:1788619925.682929:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571602816 00010000:00010000:0.0:1788619925.682971:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50600/0x1cd7fe5368735bc9 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cdf7 expref: 11 pid: 5846 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619925.683017:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 431 00000100:00100000:0.0:1788619925.683027:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f74380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571602816:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619925.683042:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619925.683049:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50600/0x1cd7fe5368735bc9 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cdf7 expref: 11 pid: 5846 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.683069:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50600/0x1cd7fe5368735bc9 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cdf7 expref: 11 pid: 5846 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.683091:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619925.683097:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70a00/0x1cd7fe5368735d27 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.683108:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70a00/0x1cd7fe5368735d27 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.683117:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70a00/0x1cd7fe5368735d27 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.683138:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619925.683187:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f74380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571602816:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 160us (717us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619925.683195:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 431 00010000:00010000:3.0:1788619925.684431:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50600/0x1cd7fe5368735bc9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cdf7 expref: 10 pid: 5846 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619925.684471:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5dc8000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589197312:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619925.684494:0:11397:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70a00/0x1cd7fe5368735d27 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619925.684503:0:11397:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70a00/0x1cd7fe5368735d27 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619925.684508:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70a00/0x1cd7fe5368735d27 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788619925.684626:0:11397:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967755 00000004:00010000:3.0:1788619925.684696:0:11397:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 987136 -> 987136 00000004:00010000:3.0:1788619925.684700:0:11397:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619925 -> 1788619925 00000004:00010000:3.0:1788619925.684702:0:11397:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619925 -> 1788619924 00000004:00010000:3.0:1788619925.684704:0:11397:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619925 -> 1788619925 00000004:00010000:3.0:1788619925.684707:0:11397:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1158 -> 1158 00000004:00010000:3.0:1788619925.684714:0:11397:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000003bfe4b3a reply state 00000000241434d5 transno 4294967756 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70a00/0x1cd7fe5368735d27 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619925.684723:0:11397:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788619925.684813:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f76a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+15:14908:x1875503571602688:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 6380us (6654us total) trans 4294967756 rc 0/0 00000100:00100000:3.0:1788619925.684820:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1296 00000020:00080000:3.0:1788619925.684824:0:11397:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1788619925.687794:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619925.687879:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571602944 00000100:00100000:1.0:1788619925.687889:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571602944 00000100:00100000:1.0:1788619925.687948:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1066 00000100:00100000:1.0:1788619925.687959:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f74700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:14908:x1875503571602944:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619925.687976:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619925.687993:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d990b60e/0x1cd7fe5368735d2e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cf55 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.688024:0:6696:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000011e2507b/0x1cd7fe5368735bde lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1536000->1540095) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce0c expref: 9 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.688032:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=1s v=(6 13) 00010000:00010000:1.0:1788619925.688036:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(7 13) 00010000:00010000:1.0:1788619925.688042:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d990b60e/0x1cd7fe5368735d2e lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf55 expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.688077:0:6696:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000011e2507b/0x1cd7fe5368735bde lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1536000->1540095) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce0c expref: 9 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.688091:0:6696:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000011e2507b/0x1cd7fe5368735bde lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1536000->1540095) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce0c expref: 9 pid: 6697 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619925.688106:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64d1c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589197440:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619925.688187:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7780 going to sleep for 11 seconds 00010000:00010000:0.0:1788619925.688545:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70a00/0x1cd7fe5368735d27 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.688555:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70a00/0x1cd7fe5368735d27 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.688562:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70a00/0x1cd7fe5368735d27 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.688568:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c5f70a00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788619925.688573:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70a00/0x1cd7fe5368735d27 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.688585:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70a00/0x1cd7fe5368735d27 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.688601:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619925.688603:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788619925.688609:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70a00/0x1cd7fe5368735d27 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.688615:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70a00/0x1cd7fe5368735d27 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619925.692488:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64d1c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589197440:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619925.692526:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d990b60e/0x1cd7fe5368735d2e lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf55 expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.692552:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297200, rc 0) 00000100:00100000:1.0:1788619925.692619:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f74700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:14908:x1875503571602944:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4661us (4833us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619925.692627:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1066 00000100:00100000:2.0:1788619925.694202:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619925.695153:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571603072 00000100:00100000:1.0:1788619925.695163:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571603072 00002000:00100000:1.0:1788619925.695214:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1740800->1757183) req@ffff8e78c7430e00 x1875503571603072/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 16872/0 e 0 to 0 dl 1788619936 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619925.695226:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7430e00 with x1875503571603072 ext(1740800->1757183) 00010000:00010000:1.0:1788619925.695235:0:6703:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 0000000011e2507b/0x1cd7fe5368735bde lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1536000->1540095) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce0c expref: 10 pid: 6697 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.695250:0:6703:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 0000000011e2507b/0x1cd7fe5368735bde lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1536000->1540095) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce0c expref: 10 pid: 6697 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788619925.695258:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff8e78c7430e00 00000100:00100000:1.0:1788619925.695292:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 18 00000100:00100000:1.0:1788619925.695301:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7430e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571603072:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:1.0:1788619925.696371:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7430e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571603072:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 1065us (2170us total) trans 4294967477 rc 0/0 00000100:00100000:1.0:1788619925.696381:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 18 00002000:00100000:1.0:1788619925.696389:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1740800->1757183) req@ffff8e78c7430e00 x1875503571603072/t4294967477(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:21/0 lens 16872/448 e 0 to 0 dl 1788619936 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619925.696406:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7430e00 with x1875503571603072 ext(1740800->1757183) 00010000:00010000:1.0:1788619925.696413:0:6703:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 0000000011e2507b/0x1cd7fe5368735bde lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1536000->1540095) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce0c expref: 12 pid: 6697 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.696426:0:6703:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 0000000011e2507b/0x1cd7fe5368735bde lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1536000->1540095) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce0c expref: 12 pid: 6697 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788619925.696434:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff8e78c7430e00 00000100:00100000:3.0:1788619925.703025:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619925.703118:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571603200 00000100:00100000:0.0:1788619925.703128:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571603200 00010000:00010000:0.0:1788619925.703166:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000011e2507b/0x1cd7fe5368735bde lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1536000->1540095) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce0c expref: 12 pid: 6697 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619925.703204:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 432 00000100:00100000:0.0:1788619925.703214:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7561850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2353:x1875503571603200:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619925.703229:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619925.703250:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000011e2507b/0x1cd7fe5368735bde lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1536000->1540095) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce0c expref: 12 pid: 6697 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.703265:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000011e2507b/0x1cd7fe5368735bde lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1536000->1540095) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce0c expref: 12 pid: 6697 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.703290:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011e2507b/0x1cd7fe5368735bde lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1536000->1540095) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ce0c expref: 12 pid: 6697 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.703313:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(7 13) 00010000:00010000:0.0:1788619925.703317:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(7 13) 00010000:00010000:0.0:1788619925.703355:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000d990b60e/0x1cd7fe5368735d2e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf55 expref: 5 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.703366:0:5829:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d990b60e/0x1cd7fe5368735d2e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf55 expref: 5 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.703416:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000d990b60e/0x1cd7fe5368735d2e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf55 expref: 5 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619925.703430:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64aa300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1875503589197568:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619925.703512:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590f00 going to sleep for 11 seconds 00000100:00100000:0.0:1788619925.706801:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64aa300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1875503589197568:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619925.706831:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619925.706904:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7561850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571603200:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3682us (3881us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619925.706916:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 432 00000100:00100000:3.0:1788619925.707916:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619925.709100:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571603328 00000100:00100000:0.0:1788619925.709110:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571603328 00002000:00010000:0.0:1788619925.709148:0:6701:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1696932->18446744073709551615) 00002000:00100000:0.0:1788619925.709152:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5dc8000 with x1875503571603328 ext(1696932->18446744073709551615) 00002000:00010000:0.0:1788619925.709163:0:6701:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5dc8000. 00000100:00100000:0.0:1788619925.709192:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 620 00000100:00100000:0.0:1788619925.709201:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dc8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571603328:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:0.0:1788619925.709511:0:6701:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 2092012 -> 1696932 00002000:00010000:0.0:1788619925.709516:0:6701:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619925 -> 1788619925 00002000:00010000:0.0:1788619925.709518:0:6701:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619925 -> 1788619925 00002000:00010000:0.0:1788619925.709521:0:6701:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619925 -> 1788619925 00002000:00010000:0.0:1788619925.709523:0:6701:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 647 -> 647 00000100:00100000:0.0:1788619925.709578:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dc8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571603328:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 377us (1679us total) trans 4294967478 rc 0/0 00000100:00100000:0.0:1788619925.709586:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 620 00002000:00010000:0.0:1788619925.709590:0:6701:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1696932->18446744073709551615) 00002000:00100000:0.0:1788619925.709592:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5dc8000 with x1875503571603328 ext(1696932->18446744073709551615) 00002000:00010000:0.0:1788619925.709597:0:6701:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5dc8000. 00000100:00100000:3.0:1788619925.716992:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619925.717108:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571603456 00000100:00100000:0.0:1788619925.717121:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571603456 00000100:00100000:0.0:1788619925.717180:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1297 00000100:00100000:0.0:1788619925.717190:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571603456:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619925.717216:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619925.717240:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4e00/0x1cd7fe5368735d35 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cf5c expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.717354:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4400/0x1cd7fe5368735d3c lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.717370:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4400/0x1cd7fe5368735d3c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.717382:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4400/0x1cd7fe5368735d3c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619925.717411:0:9417:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1696932 00000004:00200000:0.0:1788619925.717415:0:9417:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619925.717434:0:9417:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4400/0x1cd7fe5368735d3c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.717473:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4e00/0x1cd7fe5368735d35 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf5c expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.717493:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4400/0x1cd7fe5368735d3c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf5c expref: 15 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.717502:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619925.717505:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619925.717509:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d4400, rc 0) 00000100:00100000:0.0:1788619925.717570:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:14908:x1875503571603456:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 381us (4701us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619925.717578:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1297 00000100:00100000:1.0:1788619925.721269:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619925.721362:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571603584 00000100:00100000:3.0:1788619925.721372:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571603584 00000100:00100000:3.0:1788619925.721444:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 798 00000100:00100000:3.0:1788619925.721464:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64d1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571603584:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:3.0:1788619925.721749:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 987136 -> 987136 00000004:00010000:3.0:1788619925.721753:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619925 -> 1788619925 00000004:00010000:3.0:1788619925.721756:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619924 -> 1788619924 00000004:00010000:3.0:1788619925.721759:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619925 -> 1788619925 00000004:00010000:3.0:1788619925.721761:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1158 -> 1158 00000100:00100000:3.0:1788619925.721828:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64d1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571603584:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 375us (569us total) trans 4294967757 rc 0/0 00000100:00100000:3.0:1788619925.721837:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 798 00000100:00100000:3.0:1788619925.728863:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619925.729255:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571603840 00000100:00100000:0.0:1788619925.729265:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571603840 00000100:00100000:0.0:1788619925.729503:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1298 00000100:00100000:0.0:1788619925.729514:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:14908:x1875503571603840:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:0.0:1788619925.729574:0:5846:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619925.729602:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4400/0x1cd7fe5368735d3c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf5c expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.729619:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619925.729625:0:5846:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788619925.729846:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4e00/0x1cd7fe5368735d43 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.729862:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4e00/0x1cd7fe5368735d43 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.729872:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4e00/0x1cd7fe5368735d43 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788619925.730157:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967758 00000004:00010000:0.0:1788619925.730182:0:5846:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 987136 -> 987136 00000004:00010000:0.0:1788619925.730186:0:5846:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619925 -> 1788619925 00000004:00010000:0.0:1788619925.730189:0:5846:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619924 -> 1788619924 00000004:00010000:0.0:1788619925.730191:0:5846:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619925 -> 1788619925 00000004:00010000:0.0:1788619925.730193:0:5846:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1158 -> 1158 00000004:00010000:0.0:1788619925.730200:0:5846:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000bdb6400e reply state 0000000020f629a0 transno 4294967759 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4e00/0x1cd7fe5368735d43 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619925.730208:0:5846:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788619925.730293:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+18:14908:x1875503571603840:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 780us (1438us total) trans 4294967759 rc 0/0 00000100:00100000:0.0:1788619925.730300:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1298 00000020:00080000:0.0:1788619925.730305:0:5846:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788619925.733181:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4e00/0x1cd7fe5368735d43 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.733196:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4e00/0x1cd7fe5368735d43 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.733203:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4e00/0x1cd7fe5368735d43 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.733210:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c64d4e00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788619925.733216:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4e00/0x1cd7fe5368735d43 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.733223:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4e00/0x1cd7fe5368735d43 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.733239:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619925.733242:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788619925.733246:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4e00/0x1cd7fe5368735d43 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.733252:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4e00/0x1cd7fe5368735d43 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619925.734153:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619925.734359:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571603968 00000100:00100000:1.0:1788619925.734369:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571603968 00002000:00010000:1.0:1788619925.734407:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1158929->18446744073709551615) 00002000:00100000:1.0:1788619925.734412:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a5df80 with x1875503571603968 ext(1158929->18446744073709551615) 00002000:00010000:1.0:1788619925.734422:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a5df80. 00000100:00100000:1.0:1788619925.734448:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 621 00000100:00100000:1.0:1788619925.734464:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571603968:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:1.0:1788619925.734831:0:6703:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1696932 -> 1158929 00002000:00010000:1.0:1788619925.734835:0:6703:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619925 -> 1788619925 00002000:00010000:1.0:1788619925.734838:0:6703:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619925 -> 1788619925 00002000:00010000:1.0:1788619925.734840:0:6703:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619925 -> 1788619925 00002000:00010000:1.0:1788619925.734842:0:6703:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 647 -> 647 00000100:00100000:1.0:1788619925.735141:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571603968:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 439us (748us total) trans 4294967479 rc 0/0 00000100:00100000:1.0:1788619925.735152:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 621 00002000:00010000:1.0:1788619925.735156:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1158929->18446744073709551615) 00002000:00100000:1.0:1788619925.735159:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a5df80 with x1875503571603968 ext(1158929->18446744073709551615) 00002000:00010000:1.0:1788619925.735165:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a5df80. 00000100:00100000:1.0:1788619925.740732:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619925.740821:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571604096 00000100:00100000:0.0:1788619925.740830:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571604096 00000100:00100000:0.0:1788619925.740890:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1299 00000100:00100000:0.0:1788619925.740912:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64d0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:14908:x1875503571604096:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619925.740931:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619925.740952:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4e00/0x1cd7fe5368735d4a lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cf63 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.741039:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4400/0x1cd7fe5368735d51 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.741055:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4400/0x1cd7fe5368735d51 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.741067:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4400/0x1cd7fe5368735d51 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619925.741093:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1158929 00000004:00200000:0.0:1788619925.741098:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619925.741117:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4400/0x1cd7fe5368735d51 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.741142:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4e00/0x1cd7fe5368735d4a lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf63 expref: 12 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.741159:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4400/0x1cd7fe5368735d51 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf63 expref: 11 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.741167:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619925.741170:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619925.741174:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d4400, rc 0) 00000100:00100000:0.0:1788619925.741232:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64d0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571604096:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 334us (509us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619925.741238:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1299 00000100:00100000:3.0:1788619925.746102:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619925.746147:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619925.746239:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571604352 00000100:00100000:2.0:1788619925.746248:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571604352 00000100:00100000:2.0:1788619925.746306:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1067 00000100:00100000:2.0:1788619925.746317:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571604352:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619925.746368:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619925.746386:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002d9637b3/0x1cd7fe5368735d58 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cf71 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619925.746447:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c7433800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589197696:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619925.746534:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164d80 going to sleep for 11 seconds 00000100:00100000:2.0:1788619925.748715:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c7433800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589197696:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619925.748760:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002d9637b3/0x1cd7fe5368735d58 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf71 expref: 12 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.748785:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d9637b3/0x1cd7fe5368735d58 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf71 expref: 12 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.748798:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f73400, rc 0) 00000100:00100000:2.0:1788619925.748851:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571604352:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2535us (2702us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619925.748859:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1067 00000100:00100000:1.0:1788619925.749024:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571604224 00000100:00100000:1.0:1788619925.749031:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571604224 00000100:00100000:1.0:1788619925.749081:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1300 00000100:00100000:1.0:1788619925.749090:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571604224:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619925.749105:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619925.749124:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296e00/0x1cd7fe5368735d5f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cf6a expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.749157:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe5368735cbe lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ceec expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619925.749193:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64d1500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589197824:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619925.749242:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7780 going to sleep for 11 seconds 00000004:00010000:1.0:1788619925.751441:0:5845:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619924 -> 1788619925 00000100:00100000:1.0:1788619925.751481:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64d1500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589197824:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619925.751500:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:1.0:1788619925.751515:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296e00/0x1cd7fe5368735d5f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf6a expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.751532:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619925.751535:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619925.751540:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296e00/0x1cd7fe5368735d5f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf6a expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.751558:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296e00, rc 0) 00000100:00100000:1.0:1788619925.751612:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571604224:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2522us (5519us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619925.751620:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1300 00000100:00100000:3.0:1788619925.759047:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619925.759090:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619925.759313:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571604608 00000100:00100000:2.0:1788619925.759322:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571604608 00000100:00100000:2.0:1788619925.759403:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1068 00000100:00100000:2.0:1788619925.759413:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571604608:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619925.759431:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619925.759449:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002d9637b3/0x1cd7fe5368735d66 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cf7f expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619925.759523:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c7433800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589197952:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619925.759604:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164d80 going to sleep for 11 seconds 00000100:00100000:0.0:1788619925.760361:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571604480 00000100:00100000:0.0:1788619925.760366:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571604480 00000100:00100000:0.0:1788619925.760396:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1301 00000100:00100000:0.0:1788619925.760404:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cdaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571604480:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619925.760415:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619925.760430:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4200/0x1cd7fe5368735d6d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cf78 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.760473:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe5368735cbe lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ceec expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619925.760509:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64aa300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589198080:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619925.760561:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590c00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619925.763411:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c7433800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589197952:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619925.763465:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002d9637b3/0x1cd7fe5368735d66 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf7f expref: 12 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.763491:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d9637b3/0x1cd7fe5368735d66 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf7f expref: 12 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.763506:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f73400, rc 0) 00000100:00100000:2.0:1788619925.763563:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571604608:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4151us (4471us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619925.763572:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1068 00000100:00100000:0.0:1788619925.765235:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64aa300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589198080:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619925.765258:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:0.0:1788619925.765273:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4200/0x1cd7fe5368735d6d lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf78 expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.765289:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619925.765292:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619925.765296:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4200/0x1cd7fe5368735d6d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf78 expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.765309:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d4200, rc 0) 00000100:00100000:0.0:1788619925.765584:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cdaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571604480:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5179us (6545us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619925.765592:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1301 00000100:00100000:3.0:1788619925.768953:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619925.769363:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571604736 00000100:00100000:0.0:1788619925.769372:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571604736 00000100:00100000:0.0:1788619925.769421:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1302 00000100:00100000:0.0:1788619925.769431:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+17:14908:x1875503571604736:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619925.769452:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619925.769485:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5e00/0x1cd7fe5368735d74 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cf86 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.769573:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5c00/0x1cd7fe5368735d7b lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.769588:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5c00/0x1cd7fe5368735d7b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.769600:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5c00/0x1cd7fe5368735d7b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619925.769626:0:9417:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1158929 00000004:00200000:0.0:1788619925.769640:0:9417:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619925.769658:0:9417:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5c00/0x1cd7fe5368735d7b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.769683:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5e00/0x1cd7fe5368735d74 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf86 expref: 19 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.769701:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5c00/0x1cd7fe5368735d7b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf86 expref: 18 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.769709:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619925.769711:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619925.769715:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d5c00, rc 0) 00000100:00100000:0.0:1788619925.770125:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+18:14908:x1875503571604736:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 694us (1181us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619925.770134:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1302 00000100:00100000:1.0:1788619925.776268:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619925.776371:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571604864 00000100:00100000:0.0:1788619925.776381:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571604864 00000100:00100000:1.0:1788619925.776931:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1303 00000100:00100000:1.0:1788619925.776943:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64d1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571604864:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619925.776966:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619925.776987:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296200/0x1cd7fe5368735d82 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cf8d expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.777027:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe5368735cbe lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ceec expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619925.777068:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64d2680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589198208:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619925.777154:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7780 going to sleep for 11 seconds 00000100:00100000:2.0:1788619925.782103:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619925.782190:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571604992 00000100:00100000:3.0:1788619925.782199:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571604992 00000100:00100000:3.0:1788619925.782250:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1069 00000100:00100000:3.0:1788619925.782259:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7433800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571604992:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619925.782276:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619925.782292:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000033c29ece/0x1cd7fe5368735d89 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cf94 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619925.782380:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cd8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589198336:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619925.782473:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb540 going to sleep for 11 seconds 00000100:00100000:1.0:1788619925.784833:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64d2680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589198208:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619925.784860:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:1.0:1788619925.784878:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296200/0x1cd7fe5368735d82 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf8d expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.784913:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619925.784917:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619925.784922:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296200/0x1cd7fe5368735d82 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf8d expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.784943:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296200, rc 0) 00000100:00100000:1.0:1788619925.785006:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64d1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571604864:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8066us (8748us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619925.785014:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1303 00000100:00100000:2.0:1788619925.786591:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cd8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589198336:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619925.786626:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000033c29ece/0x1cd7fe5368735d89 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf94 expref: 12 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.786658:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000033c29ece/0x1cd7fe5368735d89 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf94 expref: 12 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.786669:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f70600, rc 0) 00000100:00100000:2.0:1788619925.786717:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7433800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571604992:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4459us (4624us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619925.786724:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1069 00000100:00100000:2.0:1788619925.796380:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619925.796425:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619925.796457:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571605120 00000100:00100000:0.0:1788619925.796475:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571605120 00000100:00100000:1.0:1788619925.796492:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571605248 00000100:00100000:1.0:1788619925.796497:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571605248 00000100:00100000:0.0:1788619925.796528:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1304 00000100:00100000:1.0:1788619925.796531:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1070 00000100:00100000:1.0:1788619925.796538:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7431880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571605248:12345-192.168.201.5@tcp:101:fsx.0 00000100:00100000:0.0:1788619925.796538:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7433b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571605120:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619925.796555:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619925.796555:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619925.796572:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002abacdb6/0x1cd7fe5368735d90 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cfa2 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.796582:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5e00/0x1cd7fe5368735d97 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cf9b expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.796609:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe5368735cbe lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ceec expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619925.796628:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64d2680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589198464:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619925.796628:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64aa300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589198592:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619925.796721:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590c00 going to sleep for 11 seconds 00000100:00100000:0.0:1788619925.807149:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64aa300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589198592:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619925.807177:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:0.0:1788619925.807198:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5e00/0x1cd7fe5368735d97 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf9b expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.807219:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619925.807221:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619925.807226:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5e00/0x1cd7fe5368735d97 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf9b expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.807241:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d5e00, rc 0) 00000100:00100000:0.0:1788619925.807305:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7433b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571605120:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 10768us (10878us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619925.807314:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1304 00000100:00100000:1.0:1788619925.810578:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7780 going to sleep for 11 seconds 00000100:00100000:1.0:1788619925.810660:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64d2680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589198464:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619925.810694:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002abacdb6/0x1cd7fe5368735d90 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cfa2 expref: 12 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.810716:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002abacdb6/0x1cd7fe5368735d90 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cfa2 expref: 12 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.810731:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296200, rc 0) 00000100:00100000:1.0:1788619925.810889:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7431880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571605248:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 14351us (14516us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619925.811110:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1070 00000100:00100000:0.0:1788619925.824535:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619925.824592:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619925.824776:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571605376 00000100:00100000:3.0:1788619925.824787:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571605376 00000100:00100000:2.0:1788619925.824818:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571605504 00000100:00100000:2.0:1788619925.824822:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571605504 00000100:00100000:3.0:1788619925.824849:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1305 00000100:00100000:2.0:1788619925.824854:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1071 00000100:00100000:3.0:1788619925.824858:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64aa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:2352:x1875503571605376:12345-192.168.201.5@tcp:101:fsx.0 00000100:00100000:2.0:1788619925.824860:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64aaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:2352:x1875503571605504:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619925.824875:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619925.824875:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619925.824892:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000497c2087/0x1cd7fe5368735d9e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cfb0 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619925.824907:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71a00/0x1cd7fe5368735da5 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cfa9 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619925.824934:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe5368735cbe lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ceec expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619925.824949:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c7433100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589198720:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619925.824954:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cdb480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589198848:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619925.831223:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebcc0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619925.831840:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619925.833936:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164d80 going to sleep for 11 seconds 00000100:00100000:3.0:1788619925.835301:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571605632 00000100:00100000:3.0:1788619925.835315:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571605632 00002000:00100000:3.0:1788619925.836850:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0x10:0x0] req@ffff8e77c64d2680 x1875503571605632/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:21/0 lens 488/0 e 0 to 0 dl 1788619936 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619925.836863:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64d2680 with x1875503571605632 00000100:00100000:3.0:1788619925.836912:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 799 00000100:00100000:3.0:1788619925.836922:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64d2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2355:x1875503571605632:12345-192.168.201.5@tcp:4:fsx.0 00002000:00010000:2.0:1788619925.839694:0:7654:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size: 1158929 -> 1860930 00000100:00100000:2.0:1788619925.839723:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c7433100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589198720:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619925.839758:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000497c2087/0x1cd7fe5368735d9e lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cfb0 expref: 12 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.839783:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000497c2087/0x1cd7fe5368735d9e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cfb0 expref: 12 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.839797:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f73200, rc 0) 00000100:00100000:2.0:1788619925.839854:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64aaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:2352:x1875503571605504:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 14993us (15260us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619925.839866:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1071 00000100:00100000:3.0:1788619925.839984:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cdb480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589198848:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619925.839997:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:3.0:1788619925.840011:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71a00/0x1cd7fe5368735da5 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cfa9 expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619925.840025:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619925.840028:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619925.840032:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71a00/0x1cd7fe5368735da5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cfa9 expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619925.840043:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f71a00, rc 0) 00000100:00100000:3.0:1788619925.840080:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64aa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:2352:x1875503571605376:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 15223us (15555us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619925.840087:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1305 00000004:00010000:2.0:1788619925.848117:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x10:0x0] updating lvb size from disk: 0 -> 57344 00000004:00010000:2.0:1788619925.848126:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x10:0x0] updating lvb mtime from disk: 1788619890 -> 1788619922 00000004:00010000:2.0:1788619925.848129:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x10:0x0] updating lvb atime from disk: 1788619890 -> 1788619890 00000004:00010000:2.0:1788619925.848131:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x10:0x0] updating lvb ctime from disk: 1788619890 -> 1788619922 00000004:00010000:2.0:1788619925.848134:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x10:0x0] updating lvb blocks from disk: 2 -> 2 00000100:00100000:3.0:1788619925.853523:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619925.853600:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619925.853686:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571605888 00000100:00100000:3.0:1788619925.853696:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571605888 00000100:00100000:3.0:1788619925.853768:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1072 00000100:00100000:3.0:1788619925.853779:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64abb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571605888:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619925.853798:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619925.853814:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d786ac5b/0x1cd7fe5368735dac lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cfbe expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619925.853874:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cdbb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589198976:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619925.854277:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619925.856449:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571605760 00000100:00100000:0.0:1788619925.856462:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571605760 00000100:00100000:0.0:1788619925.856532:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1306 00000100:00100000:0.0:1788619925.856544:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cdb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571605760:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619925.856565:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619925.856588:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5400/0x1cd7fe5368735db3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cfb7 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.856624:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe5368735cbe lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ceec expref: 20 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619925.856674:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64aa680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589199104:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619925.856742:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590c00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619925.857068:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64d2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2355:x1875503571605632:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 20143us (25228us total) trans 4294967760 rc 0/0 00000100:00100000:2.0:1788619925.857079:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 799 00002000:00100000:2.0:1788619925.857091:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0x10:0x0] req@ffff8e77c64d2680 x1875503571605632/t4294967760(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:21/0 lens 488/448 e 0 to 0 dl 1788619936 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619925.857101:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64d2680 with x1875503571605632 00000100:00100000:2.0:1788619925.860018:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cdbb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589198976:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619925.860058:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d786ac5b/0x1cd7fe5368735dac lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cfbe expref: 12 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.860084:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d786ac5b/0x1cd7fe5368735dac lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cfbe expref: 12 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.860096:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f70c00, rc 0) 00000100:00100000:2.0:1788619925.860228:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64abb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571605888:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6449us (6628us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619925.860237:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1072 00000100:00100000:2.0:1788619925.861200:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64aa680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589199104:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619925.861219:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:2.0:1788619925.861233:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5400/0x1cd7fe5368735db3 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cfb7 expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.861248:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619925.861251:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619925.861255:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5400/0x1cd7fe5368735db3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cfb7 expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.861266:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d5400, rc 0) 00000100:00100000:2.0:1788619925.862974:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cdb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571605760:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6423us (9450us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619925.862986:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1306 00000100:00100000:0.0:1788619925.867950:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619925.868077:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619925.868160:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571606144 00000100:00100000:1.0:1788619925.868171:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571606144 00000100:00100000:1.0:1788619925.868238:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1073 00000100:00100000:1.0:1788619925.868249:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64ab480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2352:x1875503571606144:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619925.868269:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619925.868289:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c5e22f67/0x1cd7fe5368735dba lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cfcc expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619925.868378:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64d3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589199232:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619925.868482:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7780 going to sleep for 11 seconds 00000100:00100000:1.0:1788619925.877226:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64d3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589199232:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619925.877265:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c5e22f67/0x1cd7fe5368735dba lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cfcc expref: 12 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.877290:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c5e22f67/0x1cd7fe5368735dba lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cfcc expref: 12 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.877303:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297000, rc 0) 00000100:00100000:1.0:1788619925.877394:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64ab480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2352:x1875503571606144:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 9146us (9317us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619925.877402:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1073 00000100:00100000:2.0:1788619925.878716:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571606016 00000100:00100000:2.0:1788619925.878727:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571606016 00000100:00100000:2.0:1788619925.878823:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1307 00000100:00100000:2.0:1788619925.878834:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64a8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2352:x1875503571606016:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619925.878853:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619925.878873:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73200/0x1cd7fe5368735dc1 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cfc5 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.878920:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe5368735cbe lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ceec expref: 19 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619925.878961:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c7433100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589199360:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619925.879041:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164d80 going to sleep for 11 seconds 00000100:00100000:2.0:1788619925.882890:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c7433100 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589199360:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619925.882927:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:2.0:1788619925.882946:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73200/0x1cd7fe5368735dc1 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cfc5 expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.882966:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619925.882969:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619925.882975:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73200/0x1cd7fe5368735dc1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cfc5 expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.882989:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f73200, rc 0) 00000100:00100000:2.0:1788619925.883050:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64a8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2352:x1875503571606016:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4217us (15111us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619925.883059:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1307 00000100:00100000:1.0:1788619925.890558:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619925.890594:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619925.890757:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571606400 00000100:00100000:0.0:1788619925.890769:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571606400 00000100:00100000:3.0:1788619925.890876:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571606272 00000100:00100000:3.0:1788619925.890881:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571606272 00000100:00100000:3.0:1788619925.890925:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1308 00000100:00100000:3.0:1788619925.890936:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64d3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571606272:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619925.890957:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619925.890977:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71a00/0x1cd7fe5368735dc8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cfd3 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619925.891017:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe5368735cbe lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ceec expref: 19 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619925.891057:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cda680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589199488:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619925.891137:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebc00 going to sleep for 11 seconds 00000100:00100000:0.0:1788619925.891286:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1074 00000100:00100000:0.0:1788619925.891294:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64d0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571606400:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619925.891302:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619925.891312:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002919f738/0x1cd7fe5368735dcf lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cfda expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619925.891364:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64ab100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589199616:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619925.891410:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590840 going to sleep for 11 seconds 00000100:00100000:1.0:1788619925.895096:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cda680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589199488:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619925.895125:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:1.0:1788619925.895146:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71a00/0x1cd7fe5368735dc8 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cfd3 expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.895167:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619925.895170:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619925.895174:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71a00/0x1cd7fe5368735dc8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cfd3 expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.895185:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f71a00, rc 0) 00000100:00100000:1.0:1788619925.895205:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64ab100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589199616:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619925.895221:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002919f738/0x1cd7fe5368735dcf lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cfda expref: 12 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.895232:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002919f738/0x1cd7fe5368735dcf lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cfda expref: 12 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.895239:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d4600, rc 0) 00000100:00100000:1.0:1788619925.895279:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64d0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571606400:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3986us (4682us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619925.895287:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1074 00000100:00100000:1.0:1788619925.895353:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64d3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571606272:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4392us (4775us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619925.895360:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1308 00000100:00100000:1.0:1788619925.899749:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619925.899825:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571606656 00000100:00100000:3.0:1788619925.899835:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571606656 00000100:00100000:3.0:1788619925.899889:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 37 00000100:00100000:3.0:1788619925.899905:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64d1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+20:14908:x1875503571606656:12345-192.168.201.5@tcp:35:fsx.0 00000100:00100000:3.0:1788619925.900204:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64d1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+21:14908:x1875503571606656:12345-192.168.201.5@tcp:35:fsx.0 Request processed in 306us (463us total) trans 4294967761 rc 0/0 00000100:00100000:3.0:1788619925.900211:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 37 00000100:00100000:3.0:1788619925.904683:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619925.904784:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571606784 00000100:00100000:2.0:1788619925.904794:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571606784 00000100:00100000:2.0:1788619925.904856:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1309 00000100:00100000:2.0:1788619925.904868:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+21:14908:x1875503571606784:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619925.904891:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619925.904976:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73200/0x1cd7fe5368735dd6 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cfe1 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.905046:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72200/0x1cd7fe5368735ddd lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.905060:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72200/0x1cd7fe5368735ddd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.905071:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72200/0x1cd7fe5368735ddd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788619925.905105:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:2.0:1788619925.905109:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1788619925.905128:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72200/0x1cd7fe5368735ddd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.905153:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73200/0x1cd7fe5368735dd6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cfe1 expref: 23 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.905171:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72200/0x1cd7fe5368735ddd lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cfe1 expref: 22 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.905180:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78c7cbbe00) 00010000:00010000:2.0:1788619925.905183:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1788619925.905187:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f72200, rc 0) 00000100:00100000:2.0:1788619925.906732:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+22:14908:x1875503571606784:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 1856us (2049us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619925.906743:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1309 00000100:00100000:1.0:1788619925.915893:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619925.916069:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571607040 00000100:00100000:2.0:1788619925.916079:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571607040 00000100:00100000:2.0:1788619925.916137:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1310 00000100:00100000:2.0:1788619925.916147:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+22:14908:x1875503571607040:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619925.916167:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619925.916188:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72400/0x1cd7fe5368735de4 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cfe8 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.916261:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735deb lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.916281:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735deb lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.916292:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735deb lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.916306:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73e00/0x1cd7fe5368735df2 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x3d6fced7 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.916312:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73e00/0x1cd7fe5368735df2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3d6fced7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.916317:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73e00/0x1cd7fe5368735df2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3d6fced7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.917141:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe5368735cbe lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ceec expref: 23 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.917170:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5800/0x1cd7fe5368735df9 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.917179:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5800/0x1cd7fe5368735df9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.917187:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5800/0x1cd7fe5368735df9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619925.917197:0:5846:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1860930 00010000:00010000:0.0:1788619925.917285:0:5846:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735deb lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.917290:0:5846:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735deb lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.917295:0:5846:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735deb lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.917298:0:5846:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c5f73400 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788619925.917301:0:5846:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735deb lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.917306:0:5846:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735deb lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.917315:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78c7cbbe00) 00010000:00010000:0.0:1788619925.917316:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788619925.917320:0:5846:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735deb lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.919100:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735deb lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.919121:0:5846:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73e00/0x1cd7fe5368735df2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3d6fced7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.919129:0:5846:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73e00/0x1cd7fe5368735df2 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3d6fced7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.919135:0:5846:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73e00/0x1cd7fe5368735df2 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3d6fced7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.919141:0:5846:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c5f73e00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788619925.919145:0:5846:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73e00/0x1cd7fe5368735df2 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3d6fced7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.919152:0:5846:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73e00/0x1cd7fe5368735df2 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3d6fced7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.919165:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x3d6fced7 (ffff8e78c7cbb000) 00010000:00010000:0.0:1788619925.919168:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788619925.919172:0:5846:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73e00/0x1cd7fe5368735df2 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x3d6fced7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.919179:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73e00/0x1cd7fe5368735df2 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x3d6fced7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.919235:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72400/0x1cd7fe5368735de4 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cfe8 expref: 25 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.919249:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5800/0x1cd7fe5368735df9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cfe8 expref: 24 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619925.919256:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619925.919259:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619925.919261:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d5800, rc 0) 00000100:00100000:0.0:1788619925.919719:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+24:14908:x1875503571607040:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3570us (3835us total) trans 4294967762 rc 0/0 00000100:00100000:0.0:1788619925.919727:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1310 00000100:00100000:2.0:1788619925.931624:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619925.931677:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619925.931726:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619925.931868:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571607296 00000100:00100000:3.0:1788619925.931879:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571607296 00000100:00100000:3.0:1788619925.931954:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1075 00000100:00100000:3.0:1788619925.931966:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7431c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571607296:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619925.931985:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619925.932003:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa1562a5/0x1cd7fe5368735e07 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cff6 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619925.932060:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cda300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589199744:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619925.932130:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb840 going to sleep for 11 seconds 00000100:00100000:1.0:1788619925.932462:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571607168 00000100:00100000:1.0:1788619925.932467:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571607168 00000100:00100000:1.0:1788619925.932510:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1311 00000100:00100000:1.0:1788619925.932518:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7433100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571607168:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619925.932529:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619925.932541:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe5368735e0e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cfef expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.932584:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe5368735cbe lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ceec expref: 23 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619925.932611:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df8b1c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589199872:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619925.932655:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7780 going to sleep for 11 seconds 00000100:00100000:1.0:1788619925.935446:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571607424 00000100:00100000:1.0:1788619925.935469:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571607424 00000100:00100000:1.0:1788619925.935542:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 8 00000100:00100000:1.0:1788619925.935553:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e6478a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2354:x1875503571607424:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619925.935739:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619925.935766:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5800/0x1cd7fe5368735df9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cfe8 expref: 24 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.935786:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619925.935791:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619925.935878:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e6478a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+23:2354:x1875503571607424:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 327us (4152us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619925.935889:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 8 00000100:00100000:1.0:1788619925.936968:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cda300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589199744:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619925.937002:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000aa1562a5/0x1cd7fe5368735e07 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cff6 expref: 12 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.937023:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa1562a5/0x1cd7fe5368735e07 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cff6 expref: 12 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.937034:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f71000, rc 0) 00000100:00100000:1.0:1788619925.937244:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7431c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571607296:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5279us (5564us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619925.937252:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1075 00000100:00100000:1.0:1788619925.937307:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df8b1c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589199872:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619925.937321:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:1.0:1788619925.937360:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe5368735e0e lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cfef expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.937374:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619925.937377:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619925.937382:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe5368735e0e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cfef expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.937393:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297e00, rc 0) 00000100:00100000:1.0:1788619925.937416:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7433100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571607168:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4899us (5804us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619925.937426:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1311 00000100:00100000:3.0:1788619925.943528:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619925.943583:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619925.943724:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571607552 00000100:00100000:2.0:1788619925.943734:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571607552 00000100:00100000:2.0:1788619925.943796:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1312 00000100:00100000:2.0:1788619925.943807:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571607552:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619925.943830:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619925.943850:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73200/0x1cd7fe5368735e15 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1cffd expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.943888:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe5368735cbe lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ceec expref: 22 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619925.943947:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e6479500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589200000:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619925.944038:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164540 going to sleep for 11 seconds 00000100:00100000:1.0:1788619925.944165:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571607680 00000100:00100000:1.0:1788619925.944170:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571607680 00000100:00100000:1.0:1788619925.944209:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1076 00000100:00100000:1.0:1788619925.944217:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe13a050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571607680:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619925.944225:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619925.944236:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006901c1d2/0x1cd7fe5368735e1c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d004 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619925.944273:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df8b1c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589200128:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619925.944299:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7780 going to sleep for 11 seconds 00000100:00100000:2.0:1788619925.947281:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e6479500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589200000:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619925.947310:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:2.0:1788619925.947359:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73200/0x1cd7fe5368735e15 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cffd expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.947380:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619925.947383:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619925.947390:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73200/0x1cd7fe5368735e15 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cffd expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619925.947513:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f73200, rc 0) 00000100:00100000:2.0:1788619925.947652:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571607552:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3846us (4132us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619925.947661:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1312 00000100:00100000:1.0:1788619925.951914:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df8b1c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589200128:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619925.951954:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000006901c1d2/0x1cd7fe5368735e1c lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d004 expref: 12 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.951981:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006901c1d2/0x1cd7fe5368735e1c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d004 expref: 12 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619925.951995:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296a00, rc 0) 00000100:00100000:1.0:1788619925.952057:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe13a050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571607680:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7840us (8472us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619925.952066:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1076 00000100:00100000:0.0:1788619925.961566:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619925.961742:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571607808 00000100:00100000:2.0:1788619925.961752:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571607808 00002000:00100000:2.0:1788619925.961801:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c64aad80 x1875503571607808/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:21/0 lens 488/0 e 0 to 0 dl 1788619936 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619925.961814:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64aad80 with x1875503571607808 00000100:00100000:2.0:1788619925.961856:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 800 00000100:00100000:2.0:1788619925.961866:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64aad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+23:2353:x1875503571607808:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619925.962267:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64aad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+23:2353:x1875503571607808:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 402us (710us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619925.962275:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 800 00002000:00100000:2.0:1788619925.962282:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c64aad80 x1875503571607808/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:21/0 lens 488/4536 e 0 to 0 dl 1788619936 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619925.962290:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64aad80 with x1875503571607808 00000100:00100000:2.0:1788619925.966446:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619925.966583:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571607936 00000100:00100000:3.0:1788619925.966593:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571607936 00002000:00100000:3.0:1788619925.966653:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e6479500 x1875503571607936/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:21/0 lens 488/0 e 0 to 0 dl 1788619936 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619925.966663:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e6479500 with x1875503571607936 00000100:00100000:3.0:1788619925.966739:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 801 00000100:00100000:3.0:1788619925.966749:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e6479500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+23:2355:x1875503571607936:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619925.967070:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e6479500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+23:2355:x1875503571607936:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 321us (632us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619925.967078:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 801 00002000:00100000:3.0:1788619925.967086:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e6479500 x1875503571607936/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:21/0 lens 488/4536 e 0 to 0 dl 1788619936 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619925.967094:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e6479500 with x1875503571607936 00000100:00100000:1.0:1788619925.970781:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619925.970959:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571608064 00000100:00100000:2.0:1788619925.970969:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571608064 00002000:00100000:2.0:1788619925.971021:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df8b1c00 x1875503571608064/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:21/0 lens 504/0 e 0 to 0 dl 1788619936 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619925.971032:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df8b1c00 with x1875503571608064 00000100:00100000:2.0:1788619925.971102:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 802 00000100:00100000:2.0:1788619925.971113:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+23:2354:x1875503571608064:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619925.975913:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+23:2354:x1875503571608064:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 4780us (5119us total) trans 0 rc 57344/57344 00000100:00100000:2.0:1788619925.975926:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 802 00002000:00100000:2.0:1788619925.975938:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df8b1c00 x1875503571608064/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:21/0 lens 504/440 e 0 to 0 dl 1788619936 ref 1 fl Complete:/600/0 rc 57344/57344 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619925.975948:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df8b1c00 with x1875503571608064 00000100:00100000:0.0:1788619925.980665:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619925.980752:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571608192 00000100:00100000:3.0:1788619925.980762:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571608192 00002000:00100000:3.0:1788619925.980810:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c64a9c00 x1875503571608192/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:21/0 lens 488/0 e 0 to 0 dl 1788619936 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619925.980822:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64a9c00 with x1875503571608192 00000100:00100000:3.0:1788619925.980859:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 803 00000100:00100000:3.0:1788619925.980869:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64a9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+23:2352:x1875503571608192:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619925.984417:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64a9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+23:2352:x1875503571608192:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 3539us (3752us total) trans 0 rc 36864/36864 00000100:00100000:3.0:1788619925.984430:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 803 00002000:00100000:3.0:1788619925.984443:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c64a9c00 x1875503571608192/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:21/0 lens 488/440 e 0 to 0 dl 1788619936 ref 1 fl Complete:/600/0 rc 36864/36864 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619925.984464:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64a9c00 with x1875503571608192 00000100:00100000:3.0:1788619925.987844:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619925.987970:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571608320 00000100:00100000:2.0:1788619925.987981:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571608320 00002000:00100000:2.0:1788619925.988030:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5cd9180 x1875503571608320/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:21/0 lens 504/0 e 0 to 0 dl 1788619936 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619925.988042:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5cd9180 with x1875503571608320 00000100:00100000:2.0:1788619925.988080:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 804 00000100:00100000:2.0:1788619925.988090:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+23:2354:x1875503571608320:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:0.0:1788619925.992760:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 987136 -> 987136 00000004:00010000:0.0:1788619925.992770:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619925 -> 1788619925 00000004:00010000:0.0:1788619925.992772:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619925 -> 1788619924 00000004:00010000:0.0:1788619925.992775:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619925 -> 1788619925 00000004:00010000:0.0:1788619925.992777:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1158 -> 1158 00000100:00100000:0.0:1788619925.992842:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+25:2354:x1875503571608320:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 4749us (5002us total) trans 4294967763 rc 0/0 00000100:00100000:0.0:1788619925.992853:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 804 00002000:00100000:0.0:1788619925.992866:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5cd9180 x1875503571608320/t4294967763(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:21/0 lens 504/448 e 0 to 0 dl 1788619936 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619925.992875:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5cd9180 with x1875503571608320 00000100:00100000:3.0:1788619925.996284:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619925.996393:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571608448 00000100:00100000:0.0:1788619925.996403:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571608448 00000100:00100000:0.0:1788619925.996541:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1313 00000100:00100000:0.0:1788619925.996552:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cda300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+25:2354:x1875503571608448:12345-192.168.201.5@tcp:16:fsx.0 00000001:00080000:2.0:1788619926.020393:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967763 is committed 00000001:00080000:2.0:1788619926.020419:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967754 is committed 00000100:00100000:0.0:1788619926.020919:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cda300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571608448:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 24361us (24635us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619926.020932:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1313 00000100:00100000:1.0:1788619926.024854:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619926.025015:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571608576 00000100:00100000:2.0:1788619926.025026:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571608576 00000100:00100000:2.0:1788619926.025190:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1314 00000100:00100000:2.0:1788619926.025203:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571608576:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619926.025223:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619926.025245:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73a00/0x1cd7fe5368735e23 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d00b expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.025276:0:9417:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe5368735cbe lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ceec expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.025287:0:9417:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73a00/0x1cd7fe5368735e23 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d00b expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.025319:0:9417:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe5368735cbe lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ceec expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.025361:0:9417:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe5368735cbe lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ceec expref: 13 pid: 5845 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619926.025375:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e6478000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589200256:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619926.025457:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164840 going to sleep for 11 seconds 00000100:00080000:3.0:1788619926.028003:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1788619926.028367:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619926.028672:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571608704 00000100:00100000:3.0:1788619926.028682:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571608704 00010000:00010000:3.0:1788619926.028734:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe5368735cbe lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ceec expref: 14 pid: 5845 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619926.028771:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 433 00000100:00100000:3.0:1788619926.028781:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e647a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571608704:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619926.028796:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000004:00010000:3.0:1788619926.028820:0:5830:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1158 -> 1544 00010000:00010000:3.0:1788619926.028826:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe5368735cbe lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ceec expref: 14 pid: 5845 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.028843:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe5368735cbe lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ceec expref: 14 pid: 5845 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.028865:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619926.028871:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73a00/0x1cd7fe5368735e23 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d00b expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.028884:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73a00/0x1cd7fe5368735e23 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d00b expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.028909:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73a00/0x1cd7fe5368735e23 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d00b expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.028945:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73a00/0x1cd7fe5368735e23 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d00b expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619926.028959:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cda680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589200384:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619926.029031:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb9c0 going to sleep for 11 seconds 00010000:00010000:1.0:1788619926.029305:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe5368735cbe lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ceec expref: 14 pid: 5845 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619926.029321:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e6478000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589200256:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619926.029384:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73a00/0x1cd7fe5368735e23 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d00b expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.029395:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619926.029397:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619926.029400:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f73a00, rc 0) 00000100:00100000:1.0:1788619926.029436:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:14908:x1875503571608576:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4238us (4592us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619926.029443:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1314 00000100:00080000:2.0:1788619926.029746:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788619926.031706:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cda680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589200384:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619926.031731:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619926.031784:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e647a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571608704:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3004us (3423us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619926.031795:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 433 00000100:00100000:3.0:1788619926.033910:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619926.034406:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571608832 00000100:00100000:0.0:1788619926.034415:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571608832 00000100:00100000:0.0:1788619926.034489:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 805 00000100:00100000:0.0:1788619926.034499:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cda680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571608832:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:0.0:1788619926.034798:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 987136 -> 987136 00000004:00010000:0.0:1788619926.034802:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619925 -> 1788619925 00000004:00010000:0.0:1788619926.034805:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619924 -> 1788619924 00000004:00010000:0.0:1788619926.034807:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619925 -> 1788619925 00000004:00010000:0.0:1788619926.034809:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1544 -> 1544 00000100:00100000:0.0:1788619926.034878:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cda680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571608832:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 381us (994us total) trans 4294967764 rc 0/0 00000100:00100000:0.0:1788619926.034886:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 805 00000100:00100000:3.0:1788619926.038226:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619926.038305:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571608960 00000100:00100000:2.0:1788619926.038315:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571608960 00000100:00100000:2.0:1788619926.038401:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1077 00000100:00100000:2.0:1788619926.038412:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cdbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571608960:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619926.038428:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619926.038444:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007f9cdd10/0x1cd7fe5368735e2a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d012 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619926.038507:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e6479180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589200512:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619926.038580:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164cc0 going to sleep for 11 seconds 00000100:00080000:3.0:1788619926.038773:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1788619926.041220:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e6479180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589200512:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619926.041254:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007f9cdd10/0x1cd7fe5368735e2a lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d012 expref: 12 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.041276:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f9cdd10/0x1cd7fe5368735e2a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d012 expref: 12 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.041288:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f72000, rc 0) 00000100:00100000:1.0:1788619926.041377:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cdbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571608960:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2967us (3160us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619926.041386:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1077 00000100:00100000:0.0:1788619926.044214:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619926.044307:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571609088 00000100:00100000:2.0:1788619926.044317:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571609088 00000100:00100000:2.0:1788619926.044395:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1078 00000100:00100000:2.0:1788619926.044404:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64a9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571609088:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619926.044421:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619926.044439:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0aa1c5b/0x1cd7fe5368735e31 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d019 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619926.044507:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e647b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589200640:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619926.044589:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164b40 going to sleep for 11 seconds 00000100:00100000:2.0:1788619926.046834:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e647b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589200640:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619926.046875:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e0aa1c5b/0x1cd7fe5368735e31 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d019 expref: 12 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.046911:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0aa1c5b/0x1cd7fe5368735e31 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d019 expref: 12 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.046930:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f72a00, rc 0) 00000100:00100000:2.0:1788619926.046992:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64a9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571609088:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2590us (2788us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619926.047001:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1078 00000100:00100000:3.0:1788619926.050210:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619926.050323:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571609216 00000100:00100000:1.0:1788619926.050373:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571609216 00002000:00100000:1.0:1788619926.050440:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1114112->1118207) req@ffff8e77c5cd8380 x1875503571609216/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:22/0 lens 488/0 e 0 to 0 dl 1788619937 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619926.050462:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5cd8380 with x1875503571609216 ext(1114112->1118207) 00002000:00010000:1.0:1788619926.050468:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5cd8380 00000100:00100000:1.0:1788619926.050493:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 622 00000100:00100000:1.0:1788619926.050502:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571609216:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619926.050765:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571609216:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 263us (564us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619926.050773:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 622 00002000:00100000:1.0:1788619926.050781:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1114112->1118207) req@ffff8e77c5cd8380 x1875503571609216/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:22/0 lens 488/4536 e 0 to 0 dl 1788619937 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619926.050790:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5cd8380 with x1875503571609216 ext(1114112->1118207) 00002000:00010000:1.0:1788619926.050793:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5cd8380 00000100:00080000:3.0:1788619926.051580:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1788619926.054505:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619926.054853:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571609344 00000100:00100000:2.0:1788619926.054863:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571609344 00002000:00100000:2.0:1788619926.054931:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1134592->1138687) req@ffff8e77c64a9180 x1875503571609344/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:22/0 lens 488/0 e 0 to 0 dl 1788619937 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619926.054943:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64a9180 with x1875503571609344 ext(1134592->1138687) 00002000:00010000:2.0:1788619926.054948:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64a9180 00000100:00100000:2.0:1788619926.054981:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 623 00000100:00100000:2.0:1788619926.054991:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64a9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2352:x1875503571609344:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619926.055255:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64a9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2352:x1875503571609344:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 264us (761us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619926.055262:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 623 00002000:00100000:2.0:1788619926.055269:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1134592->1138687) req@ffff8e77c64a9180 x1875503571609344/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:22/0 lens 488/4536 e 0 to 0 dl 1788619937 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619926.055277:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64a9180 with x1875503571609344 ext(1134592->1138687) 00002000:00010000:2.0:1788619926.055281:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64a9180 00000100:00100000:2.0:1788619926.059080:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619926.059565:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571609472 00000100:00100000:1.0:1788619926.059576:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571609472 00000100:00100000:0.0:1788619926.059839:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1079 00000100:00100000:0.0:1788619926.059849:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e647b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571609472:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619926.059866:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619926.059883:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d6eb124/0x1cd7fe5368735e38 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d020 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619926.060117:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64a8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589200768:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619926.060201:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590d80 going to sleep for 11 seconds 00000100:00100000:3.0:1788619926.062543:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64a8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589200768:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619926.062584:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009d6eb124/0x1cd7fe5368735e38 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d020 expref: 12 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.062609:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d6eb124/0x1cd7fe5368735e38 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d020 expref: 12 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.062622:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d5000, rc 0) 00000100:00100000:3.0:1788619926.062825:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e647b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571609472:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2977us (3753us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619926.062834:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1079 00000100:00100000:1.0:1788619926.065789:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619926.065932:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571609600 00000100:00100000:2.0:1788619926.065942:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571609600 00000100:00100000:2.0:1788619926.065998:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1080 00000100:00100000:2.0:1788619926.066010:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571609600:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619926.066030:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619926.066047:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002cc077c3/0x1cd7fe5368735e3f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d027 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619926.066106:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e647ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589200896:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619926.066188:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164f00 going to sleep for 11 seconds 00000100:00100000:1.0:1788619926.068501:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e647ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589200896:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619926.068540:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002cc077c3/0x1cd7fe5368735e3f lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d027 expref: 12 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.068566:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002cc077c3/0x1cd7fe5368735e3f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d027 expref: 12 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.068579:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f72e00, rc 0) 00000100:00100000:1.0:1788619926.068712:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571609600:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2703us (2933us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619926.068721:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1080 00000100:00100000:3.0:1788619926.071543:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619926.071732:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571609728 00000100:00100000:1.0:1788619926.071741:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571609728 00000100:00100000:1.0:1788619926.071796:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1081 00000100:00100000:1.0:1788619926.071806:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74a9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571609728:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619926.071822:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619926.071839:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff40cdc2/0x1cd7fe5368735e46 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d02e expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619926.072003:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df8b3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589201024:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619926.072090:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78e03f5600 going to sleep for 11 seconds 00000100:00100000:1.0:1788619926.074501:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df8b3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589201024:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619926.074542:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ff40cdc2/0x1cd7fe5368735e46 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d02e expref: 12 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.074569:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff40cdc2/0x1cd7fe5368735e46 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d02e expref: 12 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.074582:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297a00, rc 0) 00000100:00100000:1.0:1788619926.074717:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74a9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571609728:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2912us (3181us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619926.074726:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1081 00000100:00100000:2.0:1788619926.077954:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619926.078129:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571609856 00000100:00100000:1.0:1788619926.078140:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571609856 00000100:00100000:1.0:1788619926.078237:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1082 00000100:00100000:1.0:1788619926.078249:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e647b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2352:x1875503571609856:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619926.078267:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619926.078287:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000945af99a/0x1cd7fe5368735e4d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d035 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619926.078390:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df8b3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589201152:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619926.078489:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78e03f5600 going to sleep for 11 seconds 00000100:00100000:1.0:1788619926.081272:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df8b3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589201152:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619926.081311:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000945af99a/0x1cd7fe5368735e4d lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d035 expref: 12 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.081375:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000945af99a/0x1cd7fe5368735e4d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d035 expref: 12 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.081391:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297400, rc 0) 00000100:00100000:1.0:1788619926.081452:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e647b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2352:x1875503571609856:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3206us (3509us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619926.081470:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1082 00000100:00100000:3.0:1788619926.084824:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619926.085033:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571609984 00000100:00100000:0.0:1788619926.085043:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571609984 00000100:00100000:0.0:1788619926.085105:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1083 00000100:00100000:0.0:1788619926.085115:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74a9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571609984:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619926.085133:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619926.085150:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8566126/0x1cd7fe5368735e54 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d03c expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619926.085211:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64a8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589201280:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619926.085366:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590780 going to sleep for 11 seconds 00000100:00100000:0.0:1788619926.089174:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64a8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589201280:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619926.089212:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b8566126/0x1cd7fe5368735e54 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d03c expref: 12 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.089237:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8566126/0x1cd7fe5368735e54 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d03c expref: 12 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.089251:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d4a00, rc 0) 00000100:00100000:0.0:1788619926.089427:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74a9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571609984:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4313us (4611us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619926.089436:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1083 00000100:00100000:2.0:1788619926.097844:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619926.101808:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571610112 00000100:00100000:1.0:1788619926.101827:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571610112 00000100:00100000:1.0:1788619926.102360:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1084 00000100:00100000:1.0:1788619926.102372:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e6478700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571610112:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619926.102391:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619926.102411:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff40cdc2/0x1cd7fe5368735e5b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d043 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619926.102751:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df8b3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589201408:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619926.102860:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78e03f5600 going to sleep for 11 seconds 00000100:00100000:1.0:1788619926.105872:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df8b3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589201408:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619926.105988:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ff40cdc2/0x1cd7fe5368735e5b lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d043 expref: 12 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.106015:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff40cdc2/0x1cd7fe5368735e5b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d043 expref: 12 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.106029:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297a00, rc 0) 00000100:00100000:1.0:1788619926.106116:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e6478700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571610112:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3746us (8281us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619926.106125:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1084 00000100:00100000:2.0:1788619926.110811:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619926.111035:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571610240 00000100:00100000:0.0:1788619926.111045:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571610240 00000100:00100000:0.0:1788619926.111145:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1315 00000100:00100000:0.0:1788619926.111154:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e647a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571610240:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619926.111176:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619926.111201:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5a00/0x1cd7fe5368735e62 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d04a expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.111235:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73a00/0x1cd7fe5368735e23 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d00b expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619926.111275:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64a8380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589201536:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619926.111523:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590780 going to sleep for 11 seconds 00000004:00010000:2.0:1788619926.114358:0:5845:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619924 -> 1788619925 00000100:00100000:2.0:1788619926.114394:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64a8380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589201536:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619926.114418:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:2.0:1788619926.114438:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5a00/0x1cd7fe5368735e62 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d04a expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.114476:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619926.114479:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619926.114486:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5a00/0x1cd7fe5368735e62 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d04a expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.114502:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d5a00, rc 0) 00000100:00100000:2.0:1788619926.114564:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e647a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571610240:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3411us (3761us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619926.114573:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1315 00000100:00100000:3.0:1788619926.118087:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619926.118173:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571610368 00000100:00100000:0.0:1788619926.118183:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571610368 00000100:00100000:0.0:1788619926.118244:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1316 00000100:00100000:0.0:1788619926.118254:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74a9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571610368:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619926.118272:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619926.118293:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe5368735e69 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d051 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.118320:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73a00/0x1cd7fe5368735e23 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d00b expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.118362:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe5368735e69 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d051 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.118395:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73a00/0x1cd7fe5368735e23 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d00b expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.118408:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73a00/0x1cd7fe5368735e23 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d00b expref: 10 pid: 9417 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619926.118427:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64aaa00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589201664:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619926.118521:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590900 going to sleep for 11 seconds 00000100:00100000:3.0:1788619926.125493:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619926.125646:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571610496 00000100:00100000:2.0:1788619926.125657:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571610496 00010000:00010000:2.0:1788619926.125698:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73a00/0x1cd7fe5368735e23 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d00b expref: 11 pid: 9417 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619926.125734:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 434 00000100:00100000:2.0:1788619926.125744:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74aa680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571610496:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619926.125759:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619926.125785:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73a00/0x1cd7fe5368735e23 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d00b expref: 11 pid: 9417 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.125800:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73a00/0x1cd7fe5368735e23 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d00b expref: 11 pid: 9417 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.125823:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619926.125830:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe5368735e69 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d051 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.125842:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe5368735e69 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d051 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.125852:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe5368735e69 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d051 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.125908:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe5368735e69 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d051 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619926.125923:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e6479c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589201792:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619926.126008:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164480 going to sleep for 11 seconds 00010000:00010000:0.0:1788619926.126676:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73a00/0x1cd7fe5368735e23 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d00b expref: 11 pid: 9417 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619926.126693:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64aaa00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589201664:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619926.126717:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe5368735e69 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d051 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.126728:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619926.126731:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619926.126733:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d4c00, rc 0) 00000100:00100000:0.0:1788619926.126781:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74a9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571610368:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8528us (8705us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619926.126789:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1316 00000100:00100000:3.0:1788619926.132244:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619926.132314:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571610624 00000100:00100000:0.0:1788619926.132323:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571610624 00002000:00100000:0.0:1788619926.132401:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c74aaa00 x1875503571610624/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:22/0 lens 488/0 e 0 to 0 dl 1788619937 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619926.132413:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c74aaa00 with x1875503571610624 00000100:00100000:0.0:1788619926.132445:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 806 00000100:00100000:0.0:1788619926.132468:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74aaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571610624:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619926.134676:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e6479c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589201792:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619926.134701:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619926.134750:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74aa680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571610496:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 9008us (9267us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619926.134760:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 434 00000100:00100000:0.0:1788619926.139211:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74aaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571610624:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 6739us (6964us total) trans 0 rc 65536/65536 00000100:00100000:0.0:1788619926.139229:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 806 00002000:00100000:0.0:1788619926.139242:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c74aaa00 x1875503571610624/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:22/0 lens 488/440 e 0 to 0 dl 1788619937 ref 1 fl Complete:/600/0 rc 65536/65536 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619926.139252:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c74aaa00 with x1875503571610624 00000100:00100000:3.0:1788619926.147655:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619926.147873:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571610752 00000100:00100000:1.0:1788619926.147883:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571610752 00002000:00100000:1.0:1788619926.147944:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c74a8000 x1875503571610752/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:22/0 lens 488/0 e 0 to 0 dl 1788619937 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619926.147955:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c74a8000 with x1875503571610752 00000100:00100000:1.0:1788619926.147994:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 807 00000100:00100000:1.0:1788619926.148003:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74a8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571610752:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619926.151962:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74a8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571610752:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 3949us (4305us total) trans 0 rc 36864/36864 00000100:00100000:1.0:1788619926.151979:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 807 00002000:00100000:1.0:1788619926.151992:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c74a8000 x1875503571610752/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:22/0 lens 488/440 e 0 to 0 dl 1788619937 ref 1 fl Complete:/600/0 rc 36864/36864 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619926.152002:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c74a8000 with x1875503571610752 00000100:00100000:3.0:1788619926.156664:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619926.156820:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571610880 00000100:00100000:0.0:1788619926.156829:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571610880 00002000:00100000:0.0:1788619926.156880:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c64af800 x1875503571610880/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:22/0 lens 488/0 e 0 to 0 dl 1788619937 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619926.156891:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64af800 with x1875503571610880 00000100:00100000:0.0:1788619926.156938:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 808 00000100:00100000:0.0:1788619926.156948:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64af800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571610880:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:0.0:1788619926.162167:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 987136 -> 987136 00000004:00010000:0.0:1788619926.162176:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619925 -> 1788619926 00000004:00010000:0.0:1788619926.162179:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619925 -> 1788619924 00000004:00010000:0.0:1788619926.162181:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619925 -> 1788619926 00000004:00010000:0.0:1788619926.162184:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1544 -> 1544 00000100:00100000:0.0:1788619926.162353:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64af800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571610880:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 5401us (5693us total) trans 4294967765 rc 0/0 00000100:00100000:0.0:1788619926.162365:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 808 00002000:00100000:0.0:1788619926.162378:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c64af800 x1875503571610880/t4294967765(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:22/0 lens 488/448 e 0 to 0 dl 1788619937 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619926.162387:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64af800 with x1875503571610880 00000100:00100000:2.0:1788619926.166213:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619926.166323:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571611008 00000100:00100000:1.0:1788619926.166351:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571611008 00000100:00100000:1.0:1788619926.166442:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1317 00000100:00100000:1.0:1788619926.166573:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e6479c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571611008:12345-192.168.201.5@tcp:16:fsx.0 00000001:00080000:2.0:1788619926.192887:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967765 is committed 00000001:00080000:2.0:1788619926.192918:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967764 is committed 00000100:00100000:0.0:1788619926.193182:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e6479c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571611008:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 26725us (26974us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619926.193197:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1317 00000100:00100000:2.0:1788619926.196124:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619926.196225:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571611136 00000100:00100000:0.0:1788619926.196236:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571611136 00000100:00100000:0.0:1788619926.196290:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1318 00000100:00100000:0.0:1788619926.196301:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e647ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571611136:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619926.196356:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619926.196379:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4a00/0x1cd7fe5368735e70 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d058 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.196417:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe5368735e69 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d051 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619926.196697:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64aaa00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589201920:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619926.196780:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590240 going to sleep for 11 seconds 00000100:00100000:3.0:1788619926.198878:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619926.199044:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571611264 00000100:00100000:1.0:1788619926.199052:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571611264 00000100:00100000:1.0:1788619926.199107:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1085 00000100:00100000:1.0:1788619926.199116:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64adf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571611264:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619926.199130:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619926.199146:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c5e22f67/0x1cd7fe5368735e77 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d05f expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619926.199199:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df8b3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589202048:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619926.199274:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccd80 going to sleep for 11 seconds 00000004:00010000:0.0:1788619926.199425:0:5845:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619924 -> 1788619925 00000100:00100000:0.0:1788619926.199460:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64aaa00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589201920:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619926.199481:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:0.0:1788619926.199497:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4a00/0x1cd7fe5368735e70 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d058 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.199515:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619926.199518:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619926.199523:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4a00/0x1cd7fe5368735e70 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d058 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.199538:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d4a00, rc 0) 00000100:00100000:0.0:1788619926.199567:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e647ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571611136:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3269us (3453us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619926.199573:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1318 00002000:00010000:1.0:1788619926.201826:0:6697:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime: 1788619925 -> 1788619926 00002000:00010000:1.0:1788619926.201836:0:6697:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime: 1788619925 -> 1788619926 00000100:00100000:1.0:1788619926.201858:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df8b3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589202048:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619926.201886:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c5e22f67/0x1cd7fe5368735e77 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d05f expref: 12 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.201924:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c5e22f67/0x1cd7fe5368735e77 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d05f expref: 12 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.201936:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297000, rc 0) 00000100:00100000:1.0:1788619926.201986:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64adf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571611264:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2870us (3116us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619926.201993:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1085 00000100:00100000:2.0:1788619926.206652:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619926.206737:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571611392 00000100:00100000:0.0:1788619926.206746:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571611392 00000100:00100000:0.0:1788619926.206803:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1319 00000100:00100000:0.0:1788619926.206812:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e6479180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571611392:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619926.206833:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619926.206852:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4a00/0x1cd7fe5368735e7e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d066 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.206889:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe5368735e69 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d051 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619926.206940:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64aaa00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589202176:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619926.207021:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590240 going to sleep for 11 seconds 00000100:00100000:3.0:1788619926.209176:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619926.209378:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571611520 00000100:00100000:2.0:1788619926.209387:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571611520 00000100:00100000:2.0:1788619926.209441:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1086 00000100:00100000:2.0:1788619926.209461:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64ac380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571611520:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619926.209477:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619926.209495:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c3458c0e/0x1cd7fe5368735e85 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d06d expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619926.209551:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e6478000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589202304:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619926.209642:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164e40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619926.211631:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64aaa00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589202176:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619926.211658:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:0.0:1788619926.211676:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4a00/0x1cd7fe5368735e7e lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d066 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.211695:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619926.211698:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619926.211703:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4a00/0x1cd7fe5368735e7e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d066 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.211718:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d4a00, rc 0) 00000100:00100000:0.0:1788619926.211776:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e6479180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571611392:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4965us (5135us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619926.211784:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1319 00000100:00100000:2.0:1788619926.214842:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e6478000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589202304:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619926.214880:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c3458c0e/0x1cd7fe5368735e85 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d06d expref: 12 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.214929:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c3458c0e/0x1cd7fe5368735e85 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d06d expref: 12 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.214943:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f73e00, rc 0) 00000100:00100000:2.0:1788619926.215003:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64ac380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571611520:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5554us (5836us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619926.215013:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1086 00000100:00100000:3.0:1788619926.218684:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619926.218730:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619926.218874:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571611648 00000100:00100000:0.0:1788619926.218883:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571611648 00000100:00100000:0.0:1788619926.218953:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1320 00000100:00100000:0.0:1788619926.218964:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64af100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571611648:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619926.218984:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619926.219004:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4a00/0x1cd7fe5368735e8c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d074 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.219042:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe5368735e69 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d051 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619926.219082:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64aaa00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589202432:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619926.219167:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590240 going to sleep for 11 seconds 00000100:00100000:1.0:1788619926.219878:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571611776 00000100:00100000:1.0:1788619926.219884:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571611776 00000100:00100000:1.0:1788619926.220076:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1087 00000100:00100000:1.0:1788619926.220084:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64aed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:2352:x1875503571611776:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619926.220095:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619926.220107:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c5e22f67/0x1cd7fe5368735e93 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d07b expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619926.220146:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df8b3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589202560:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619926.220195:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccd80 going to sleep for 11 seconds 00000100:00100000:0.0:1788619926.223144:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64aaa00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589202432:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619926.223171:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:0.0:1788619926.223189:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4a00/0x1cd7fe5368735e8c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d074 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.223209:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619926.223213:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619926.223218:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4a00/0x1cd7fe5368735e8c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d074 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.223232:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d4a00, rc 0) 00000100:00100000:0.0:1788619926.223291:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64af100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571611648:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4328us (4616us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619926.223300:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1320 00000100:00100000:1.0:1788619926.223844:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df8b3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589202560:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619926.223868:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c5e22f67/0x1cd7fe5368735e93 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d07b expref: 12 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.223897:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c5e22f67/0x1cd7fe5368735e93 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d07b expref: 12 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.223921:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297000, rc 0) 00000100:00100000:1.0:1788619926.223960:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64aed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:2352:x1875503571611776:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3877us (5229us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619926.223968:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1087 00000100:00100000:1.0:1788619926.228049:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619926.228143:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571611904 00000100:00100000:0.0:1788619926.228152:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571611904 00000100:00100000:0.0:1788619926.228206:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1321 00000100:00100000:0.0:1788619926.228216:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571611904:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619926.228238:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619926.228257:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4a00/0x1cd7fe5368735e9a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d082 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.228293:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe5368735e69 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d051 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619926.228359:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64aaa00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589202688:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619926.228443:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590240 going to sleep for 11 seconds 00000100:00100000:2.0:1788619926.234850:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619926.234984:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571612032 00000100:00100000:1.0:1788619926.234993:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571612032 00000100:00100000:1.0:1788619926.235047:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1088 00000100:00100000:1.0:1788619926.235058:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e6478000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571612032:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619926.235077:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619926.235094:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c5e22f67/0x1cd7fe5368735ea1 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d089 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619926.235151:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df8b0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589202816:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619926.235231:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccd80 going to sleep for 11 seconds 00000100:00100000:0.0:1788619926.236144:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64aaa00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589202688:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619926.236164:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:0.0:1788619926.236180:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4a00/0x1cd7fe5368735e9a lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d082 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.236198:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619926.236201:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619926.236206:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4a00/0x1cd7fe5368735e9a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d082 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.236220:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d4a00, rc 0) 00000100:00100000:0.0:1788619926.236266:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571611904:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8050us (8226us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619926.236273:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1321 00000100:00100000:1.0:1788619926.238064:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df8b0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589202816:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619926.238097:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c5e22f67/0x1cd7fe5368735ea1 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d089 expref: 12 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.238120:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c5e22f67/0x1cd7fe5368735ea1 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d089 expref: 12 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.238131:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297000, rc 0) 00000100:00100000:1.0:1788619926.238184:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e6478000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571612032:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3128us (3346us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619926.238192:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1088 00000100:00100000:2.0:1788619926.243085:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619926.243130:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619926.243204:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571612288 00000100:00100000:1.0:1788619926.243214:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571612288 00000100:00100000:1.0:1788619926.243273:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1089 00000100:00100000:1.0:1788619926.243283:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63fd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571612288:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619926.243302:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619926.243321:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003086a8b7/0x1cd7fe5368735ea8 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d097 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619926.243423:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df8b0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589202944:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619926.243509:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccd80 going to sleep for 11 seconds 00000100:00100000:0.0:1788619926.243650:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571612160 00000100:00100000:0.0:1788619926.243654:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571612160 00000100:00100000:0.0:1788619926.243697:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1322 00000100:00100000:0.0:1788619926.243704:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63fc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571612160:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619926.243714:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619926.243727:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5200/0x1cd7fe5368735eaf lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d090 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.243755:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe5368735e69 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d051 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619926.243781:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64aaa00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589203072:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619926.243808:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590240 going to sleep for 11 seconds 00000100:00100000:1.0:1788619926.247224:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df8b0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589202944:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619926.247263:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003086a8b7/0x1cd7fe5368735ea8 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d097 expref: 12 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.247287:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003086a8b7/0x1cd7fe5368735ea8 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d097 expref: 12 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.247301:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296400, rc 0) 00000100:00100000:1.0:1788619926.247400:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63fd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571612288:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4117us (4267us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619926.247409:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1089 00000100:00100000:0.0:1788619926.249238:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64aaa00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589203072:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619926.249262:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:0.0:1788619926.249279:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5200/0x1cd7fe5368735eaf lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d090 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.249298:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619926.249300:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619926.249306:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5200/0x1cd7fe5368735eaf lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d090 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.249320:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d5200, rc 0) 00000100:00100000:0.0:1788619926.249527:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63fc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571612160:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5822us (6449us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619926.249535:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1322 00000100:00100000:1.0:1788619926.253168:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619926.253260:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571612416 00000100:00100000:2.0:1788619926.253272:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571612416 00000100:00100000:2.0:1788619926.253360:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1090 00000100:00100000:2.0:1788619926.253372:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571612416:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619926.253390:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619926.253408:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d73b257/0x1cd7fe5368735eb6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d09e expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.253442:0:6696:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000d990b60e/0x1cd7fe5368735d2e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf55 expref: 7 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.253615:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(7 13) 00010000:00010000:2.0:1788619926.253619:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(8 13) 00010000:00010000:2.0:1788619926.253634:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d73b257/0x1cd7fe5368735eb6 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1343488->1347583] (req 1343488->1347583) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d09e expref: 12 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.253670:0:6696:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000d990b60e/0x1cd7fe5368735d2e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf55 expref: 7 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.253685:0:6696:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000d990b60e/0x1cd7fe5368735d2e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf55 expref: 7 pid: 6696 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619926.253699:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63ff480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589203200:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619926.253790:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164e40 going to sleep for 11 seconds 00000100:00100000:3.0:1788619926.256556:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619926.256617:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571612544 00000100:00100000:0.0:1788619926.256635:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571612544 00002000:00100000:0.0:1788619926.256684:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1114112->1860929) req@ffff8e77c64ac700 x1875503571612544/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:22/0 lens 504/0 e 0 to 0 dl 1788619937 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619926.256697:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64ac700 with x1875503571612544 ext(1114112->1860929) 00010000:00010000:0.0:1788619926.256706:0:6701:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000d990b60e/0x1cd7fe5368735d2e lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf55 expref: 8 pid: 6696 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.256724:0:6701:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000d990b60e/0x1cd7fe5368735d2e lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf55 expref: 8 pid: 6696 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788619926.256732:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff8e77c64ac700 00000100:00100000:0.0:1788619926.256754:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 19 00000100:00100000:0.0:1788619926.256763:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64ac700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571612544:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:2.0:1788619926.260256:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64ac700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+10:2355:x1875503571612544:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 3479us (3694us total) trans 4294967480 rc 0/0 00000100:00100000:2.0:1788619926.260273:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 19 00002000:00100000:2.0:1788619926.260287:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1114112->1860929) req@ffff8e77c64ac700 x1875503571612544/t4294967480(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:22/0 lens 504/448 e 0 to 0 dl 1788619937 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619926.260299:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64ac700 with x1875503571612544 ext(1114112->1860929) 00010000:00010000:2.0:1788619926.260310:0:6701:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000d990b60e/0x1cd7fe5368735d2e lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf55 expref: 10 pid: 6696 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.260355:0:6701:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000d990b60e/0x1cd7fe5368735d2e lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf55 expref: 10 pid: 6696 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788619926.260364:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff8e77c64ac700 00000100:00100000:3.0:1788619926.264252:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619926.264388:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63ff480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589203200:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619926.264426:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001d73b257/0x1cd7fe5368735eb6 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1343488->1347583] (req 1343488->1347583) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d09e expref: 12 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.264458:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f73600, rc 0) 00000100:00100000:1.0:1788619926.264521:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:14908:x1875503571612416:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 11152us (11363us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619926.264530:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1090 00000100:00100000:0.0:1788619926.264945:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571612672 00000100:00100000:0.0:1788619926.264953:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571612672 00010000:00010000:0.0:1788619926.264989:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000d990b60e/0x1cd7fe5368735d2e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf55 expref: 10 pid: 6696 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619926.265025:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 435 00000100:00100000:0.0:1788619926.265033:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6859500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+10:2353:x1875503571612672:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619926.265047:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619926.265070:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d990b60e/0x1cd7fe5368735d2e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf55 expref: 10 pid: 6696 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.265084:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d990b60e/0x1cd7fe5368735d2e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf55 expref: 10 pid: 6696 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.265110:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d990b60e/0x1cd7fe5368735d2e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf55 expref: 10 pid: 6696 timeout: 518 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.265126:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(8 13) 00010000:00010000:0.0:1788619926.265131:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(8 13) 00010000:00010000:0.0:1788619926.265139:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000001d73b257/0x1cd7fe5368735eb6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1343488->1347583) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d09e expref: 11 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.265149:0:5829:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d73b257/0x1cd7fe5368735eb6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1343488->1347583) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d09e expref: 11 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.265182:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000001d73b257/0x1cd7fe5368735eb6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1343488->1347583) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d09e expref: 11 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619926.265195:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64aaa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1875503589203328:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619926.265255:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590900 going to sleep for 11 seconds 00000100:00100000:0.0:1788619926.268608:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64aaa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1875503589203328:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619926.268645:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619926.268701:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6859500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2353:x1875503571612672:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3667us (4457us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619926.268715:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 435 00000100:00100000:1.0:1788619926.269921:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619926.270034:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571612800 00000100:00100000:3.0:1788619926.270042:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571612800 00002000:00100000:3.0:1788619926.270089:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1343488->1347583) req@ffff8e77c63f8a80 x1875503571612800/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:22/0 lens 488/0 e 0 to 0 dl 1788619937 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619926.270101:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c63f8a80 with x1875503571612800 ext(1343488->1347583) 00002000:00010000:3.0:1788619926.270106:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c63f8a80 00000100:00100000:3.0:1788619926.270145:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 624 00000100:00100000:3.0:1788619926.270155:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63f8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2352:x1875503571612800:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619926.270430:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63f8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2352:x1875503571612800:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 275us (514us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619926.270438:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 624 00002000:00100000:3.0:1788619926.270444:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1343488->1347583) req@ffff8e77c63f8a80 x1875503571612800/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:22/0 lens 488/4536 e 0 to 0 dl 1788619937 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619926.270465:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c63f8a80 with x1875503571612800 ext(1343488->1347583) 00002000:00010000:3.0:1788619926.270469:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c63f8a80 00000100:00100000:0.0:1788619926.273983:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619926.274301:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571612928 00000100:00100000:2.0:1788619926.274311:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571612928 00002000:00100000:2.0:1788619926.274388:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1347584->1351679) req@ffff8e77c64aaa00 x1875503571612928/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:22/0 lens 488/0 e 0 to 0 dl 1788619937 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619926.274400:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64aaa00 with x1875503571612928 ext(1347584->1351679) 00002000:00010000:2.0:1788619926.274407:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64aaa00 00000100:00100000:2.0:1788619926.274437:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 625 00000100:00100000:2.0:1788619926.274455:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64aaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2353:x1875503571612928:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619926.274692:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64aaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2353:x1875503571612928:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 246us (719us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619926.274700:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 625 00002000:00100000:2.0:1788619926.274707:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1347584->1351679) req@ffff8e77c64aaa00 x1875503571612928/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:22/0 lens 488/4536 e 0 to 0 dl 1788619937 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619926.274715:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64aaa00 with x1875503571612928 ext(1347584->1351679) 00002000:00010000:2.0:1788619926.274719:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64aaa00 00000100:00100000:0.0:1788619926.278413:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619926.278498:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571613056 00000100:00100000:1.0:1788619926.278506:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571613056 00002000:00100000:1.0:1788619926.278553:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1351680->1355775) req@ffff8e77c64abb80 x1875503571613056/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:22/0 lens 488/0 e 0 to 0 dl 1788619937 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619926.278565:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64abb80 with x1875503571613056 ext(1351680->1355775) 00002000:00010000:1.0:1788619926.278570:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64abb80 00000100:00100000:1.0:1788619926.278602:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 626 00000100:00100000:1.0:1788619926.278612:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64abb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2353:x1875503571613056:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619926.278910:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64abb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2353:x1875503571613056:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 291us (500us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619926.278918:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 626 00002000:00100000:1.0:1788619926.278925:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1351680->1355775) req@ffff8e77c64abb80 x1875503571613056/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:22/0 lens 488/4536 e 0 to 0 dl 1788619937 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619926.278934:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64abb80 with x1875503571613056 ext(1351680->1355775) 00002000:00010000:1.0:1788619926.278937:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64abb80 00000100:00100000:1.0:1788619926.282534:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619926.282634:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571613184 00000100:00100000:3.0:1788619926.282645:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571613184 00002000:00100000:3.0:1788619926.282692:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1355776->1359871) req@ffff8e77c69fb100 x1875503571613184/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:22/0 lens 488/0 e 0 to 0 dl 1788619937 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619926.282704:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c69fb100 with x1875503571613184 ext(1355776->1359871) 00002000:00010000:3.0:1788619926.282709:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c69fb100 00000100:00100000:3.0:1788619926.282774:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 627 00000100:00100000:3.0:1788619926.282784:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571613184:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619926.283044:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571613184:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 260us (523us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619926.283052:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 627 00002000:00100000:3.0:1788619926.283059:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1355776->1359871) req@ffff8e77c69fb100 x1875503571613184/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:22/0 lens 488/4536 e 0 to 0 dl 1788619937 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619926.283067:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c69fb100 with x1875503571613184 ext(1355776->1359871) 00002000:00010000:3.0:1788619926.283071:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c69fb100 00000100:00100000:0.0:1788619926.285800:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619926.285879:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571613312 00000100:00100000:2.0:1788619926.285887:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571613312 00002000:00100000:2.0:1788619926.285938:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1359872->1363967) req@ffff8e77c64ab480 x1875503571613312/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:22/0 lens 488/0 e 0 to 0 dl 1788619937 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619926.285950:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64ab480 with x1875503571613312 ext(1359872->1363967) 00002000:00010000:2.0:1788619926.285955:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64ab480 00000100:00100000:2.0:1788619926.285989:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 628 00000100:00100000:2.0:1788619926.285997:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64ab480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571613312:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619926.286212:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64ab480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571613312:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 214us (419us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619926.286219:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 628 00002000:00100000:2.0:1788619926.286226:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1359872->1363967) req@ffff8e77c64ab480 x1875503571613312/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:22/0 lens 488/4536 e 0 to 0 dl 1788619937 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619926.286234:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64ab480 with x1875503571613312 ext(1359872->1363967) 00002000:00010000:2.0:1788619926.286237:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64ab480 00000100:00100000:0.0:1788619926.289496:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619926.289670:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571613440 00000100:00100000:3.0:1788619926.289680:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571613440 00002000:00100000:3.0:1788619926.289727:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1363968->1368063) req@ffff8e77c64ab100 x1875503571613440/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:22/0 lens 488/0 e 0 to 0 dl 1788619937 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619926.289737:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64ab100 with x1875503571613440 ext(1363968->1368063) 00002000:00010000:3.0:1788619926.289742:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64ab100 00000100:00100000:3.0:1788619926.289777:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 629 00000100:00100000:3.0:1788619926.289787:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64ab100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571613440:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619926.290059:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64ab100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571613440:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 272us (573us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619926.290066:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 629 00002000:00100000:3.0:1788619926.290072:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1363968->1368063) req@ffff8e77c64ab100 x1875503571613440/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:22/0 lens 488/4536 e 0 to 0 dl 1788619937 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619926.290080:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64ab100 with x1875503571613440 ext(1363968->1368063) 00002000:00010000:3.0:1788619926.290083:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64ab100 00000100:00100000:0.0:1788619926.294761:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619926.294851:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571613568 00000100:00100000:3.0:1788619926.294864:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571613568 00002000:00100000:3.0:1788619926.294934:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1368064->1372159) req@ffff8e77c64aa680 x1875503571613568/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:22/0 lens 488/0 e 0 to 0 dl 1788619937 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619926.294952:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64aa680 with x1875503571613568 ext(1368064->1372159) 00002000:00010000:3.0:1788619926.294957:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64aa680 00000100:00100000:3.0:1788619926.294996:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 630 00000100:00100000:3.0:1788619926.295007:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64aa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571613568:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619926.295269:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64aa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571613568:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 262us (518us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619926.295277:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 630 00002000:00100000:3.0:1788619926.295284:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1368064->1372159) req@ffff8e77c64aa680 x1875503571613568/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:22/0 lens 488/4536 e 0 to 0 dl 1788619937 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619926.295293:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64aa680 with x1875503571613568 ext(1368064->1372159) 00002000:00010000:3.0:1788619926.295297:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64aa680 00000100:00100000:0.0:1788619926.298682:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619926.298821:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571613696 00000100:00100000:2.0:1788619926.298830:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571613696 00002000:00100000:2.0:1788619926.298878:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1343488->1372159) req@ffff8e77c64a8380 x1875503571613696/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:22/0 lens 488/0 e 0 to 0 dl 1788619937 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619926.298901:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64a8380 with x1875503571613696 ext(1343488->1372159) 00002000:00010000:2.0:1788619926.298907:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64a8380 00000100:00100000:2.0:1788619926.298934:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 631 00000100:00100000:2.0:1788619926.298943:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64a8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571613696:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:2.0:1788619926.302383:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64a8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+14:2354:x1875503571613696:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 3430us (3700us total) trans 4294967481 rc 0/0 00000100:00100000:2.0:1788619926.302397:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 631 00002000:00100000:2.0:1788619926.302411:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1343488->1372159) req@ffff8e77c64a8380 x1875503571613696/t4294967481(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:22/0 lens 488/448 e 0 to 0 dl 1788619937 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619926.302421:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64a8380 with x1875503571613696 ext(1343488->1372159) 00002000:00010000:2.0:1788619926.302426:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64a8380 00000100:00100000:0.0:1788619926.305150:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619926.305222:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571613824 00000100:00100000:3.0:1788619926.305231:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571613824 00000100:00100000:3.0:1788619926.305284:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1091 00000100:00100000:3.0:1788619926.305293:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64a8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+14:2355:x1875503571613824:12345-192.168.201.5@tcp:16:fsx.0 00010000:00010000:2.0:1788619926.314558:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1788619926.329764:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967478 is committed 00000001:00080000:2.0:1788619926.329777:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967476 is committed 00000001:00080000:2.0:1788619926.329787:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967479 is committed 00000001:00080000:2.0:1788619926.329791:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967477 is committed 00000001:00080000:2.0:1788619926.329795:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967481 is committed 00000001:00080000:2.0:1788619926.329801:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967480 is committed 00000100:00100000:1.0:1788619926.329994:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64a8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571613824:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 24697us (24849us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619926.330003:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1091 00000100:00100000:3.0:1788619926.332239:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619926.332297:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571613952 00000100:00100000:1.0:1788619926.332304:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571613952 00000100:00100000:1.0:1788619926.332357:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1092 00000100:00100000:1.0:1788619926.332365:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c685b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571613952:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619926.332379:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619926.332393:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098665f62/0x1cd7fe5368735ebd lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d0a5 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619926.332456:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69d2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589203456:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619926.332691:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc9c0 going to sleep for 11 seconds 00002000:00010000:1.0:1788619926.334032:0:6698:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619925 -> 1788619926 00000100:00100000:1.0:1788619926.334053:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69d2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589203456:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619926.334075:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000098665f62/0x1cd7fe5368735ebd lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d0a5 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.334093:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098665f62/0x1cd7fe5368735ebd lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d0a5 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.334102:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297800, rc 0) 00000100:00100000:1.0:1788619926.334133:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c685b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571613952:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 1770us (1900us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619926.334141:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1092 00000100:00100000:0.0:1788619926.336093:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619926.336160:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571614080 00000100:00100000:1.0:1788619926.336167:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571614080 00000100:00100000:1.0:1788619926.336231:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1093 00000100:00100000:1.0:1788619926.336239:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69ff100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571614080:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619926.336252:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619926.336266:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe5368735ec4 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d0ac expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619926.336309:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69d2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589203584:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619926.336383:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619926.338088:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69d2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589203584:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619926.338117:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe5368735ec4 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d0ac expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.338134:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe5368735ec4 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d0ac expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.338144:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296e00, rc 0) 00000100:00100000:1.0:1788619926.338213:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69ff100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571614080:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 1974us (2125us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619926.338218:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1093 00000100:00100000:0.0:1788619926.340181:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619926.340403:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571614208 00000100:00100000:1.0:1788619926.340410:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571614208 00000100:00100000:1.0:1788619926.340535:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1323 00000100:00100000:1.0:1788619926.340543:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69ffb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571614208:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619926.340559:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619926.340574:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296200/0x1cd7fe5368735ecb lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d0b3 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.340601:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe5368735e69 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d051 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619926.340645:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69d2d80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589203712:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619926.340713:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc9c0 going to sleep for 11 seconds 00000004:00010000:1.0:1788619926.342319:0:9417:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619925 -> 1788619926 00000100:00100000:1.0:1788619926.342483:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69d2d80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589203712:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619926.342505:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:1.0:1788619926.342524:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296200/0x1cd7fe5368735ecb lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d0b3 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.342542:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619926.342545:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619926.342550:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296200/0x1cd7fe5368735ecb lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d0b3 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.342562:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296200, rc 0) 00000100:00100000:1.0:1788619926.342605:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69ffb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571614208:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2063us (2429us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619926.342612:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1323 00000100:00100000:2.0:1788619926.344109:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619926.344166:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571614336 00000100:00100000:0.0:1788619926.344173:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571614336 00000100:00100000:0.0:1788619926.344214:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1324 00000100:00100000:0.0:1788619926.344220:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63fea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571614336:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619926.344234:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619926.344249:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4800/0x1cd7fe5368735ed2 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d0ba expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.344272:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe5368735e69 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d051 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619926.344299:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69fe680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589203840:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619926.344378:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590900 going to sleep for 11 seconds 00000100:00100000:2.0:1788619926.350554:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69fe680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589203840:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619926.350585:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:2.0:1788619926.350607:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4800/0x1cd7fe5368735ed2 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d0ba expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.350639:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619926.350642:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619926.350649:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4800/0x1cd7fe5368735ed2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d0ba expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.350666:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d4800, rc 0) 00000100:00100000:2.0:1788619926.350725:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63fea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571614336:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6503us (6619us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619926.350734:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1324 00000100:00100000:2.0:1788619926.353045:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619926.353092:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571614464 00000100:00100000:1.0:1788619926.353102:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571614464 00000100:00100000:1.0:1788619926.353159:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1094 00000100:00100000:1.0:1788619926.353168:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571614464:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619926.353185:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619926.353200:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe5368735ed9 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d0c1 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619926.353245:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69d2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589203968:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619926.353308:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619926.354715:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69d2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589203968:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619926.354740:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe5368735ed9 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d0c1 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.354757:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076349659/0x1cd7fe5368735ed9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d0c1 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.354767:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296e00, rc 0) 00000100:00100000:1.0:1788619926.354804:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571614464:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 1637us (1764us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619926.354812:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1094 00000100:00100000:0.0:1788619926.356786:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619926.357043:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571614592 00000100:00100000:1.0:1788619926.357050:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571614592 00002000:00100000:1.0:1788619926.357088:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69fdc00 x1875503571614592/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:22/0 lens 488/0 e 0 to 0 dl 1788619937 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619926.357096:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69fdc00 with x1875503571614592 00000100:00100000:1.0:1788619926.357123:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 809 00000100:00100000:1.0:1788619926.357129:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571614592:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619926.359209:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571614592:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 2072us (2422us total) trans 0 rc 65536/65536 00000100:00100000:1.0:1788619926.359221:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 809 00002000:00100000:1.0:1788619926.359232:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69fdc00 x1875503571614592/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:22/0 lens 488/440 e 0 to 0 dl 1788619937 ref 1 fl Complete:/600/0 rc 65536/65536 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619926.359241:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69fdc00 with x1875503571614592 00000100:00100000:0.0:1788619926.362660:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619926.362753:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571614720 00000100:00100000:1.0:1788619926.362765:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571614720 00002000:00100000:1.0:1788619926.362813:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69fca80 x1875503571614720/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:22/0 lens 488/0 e 0 to 0 dl 1788619937 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619926.362825:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69fca80 with x1875503571614720 00000100:00100000:1.0:1788619926.362917:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 810 00000100:00100000:1.0:1788619926.362927:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571614720:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:1.0:1788619926.366221:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 987136 -> 987136 00000004:00010000:1.0:1788619926.366232:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619926 -> 1788619926 00000004:00010000:1.0:1788619926.366234:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619926 -> 1788619924 00000004:00010000:1.0:1788619926.366237:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619926 -> 1788619926 00000004:00010000:1.0:1788619926.366239:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1544 -> 1544 00000100:00100000:1.0:1788619926.366323:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571614720:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 3392us (3667us total) trans 4294967766 rc 0/0 00000100:00100000:1.0:1788619926.366371:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 810 00002000:00100000:1.0:1788619926.366386:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69fca80 x1875503571614720/t4294967766(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:22/0 lens 488/448 e 0 to 0 dl 1788619937 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619926.366396:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69fca80 with x1875503571614720 00000100:00100000:2.0:1788619926.369138:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619926.369250:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571614848 00000100:00100000:0.0:1788619926.369260:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571614848 00000100:00100000:0.0:1788619926.369322:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1325 00000100:00100000:0.0:1788619926.369461:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63ff480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571614848:12345-192.168.201.5@tcp:16:fsx.0 00000100:00100000:1.0:1788619926.396307:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63ff480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571614848:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 26837us (27167us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619926.396323:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1325 00000001:00080000:3.0:1788619926.396415:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967766 is committed 00000100:00100000:0.0:1788619926.399398:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619926.399607:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571614976 00000100:00100000:1.0:1788619926.399617:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571614976 00000100:00100000:1.0:1788619926.399702:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1326 00000100:00100000:1.0:1788619926.399712:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fe680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571614976:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619926.399732:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619926.399752:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296e00/0x1cd7fe5368735ee0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d0c8 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.399789:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe5368735e69 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d051 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619926.399831:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69d2d80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589204096:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619926.400002:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc300 going to sleep for 11 seconds 00000004:00010000:1.0:1788619926.402113:0:9417:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619924 -> 1788619926 00000100:00100000:1.0:1788619926.402142:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69d2d80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589204096:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619926.402162:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:1.0:1788619926.402179:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296e00/0x1cd7fe5368735ee0 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d0c8 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.402198:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619926.402201:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619926.402206:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296e00/0x1cd7fe5368735ee0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d0c8 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.402220:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296e00, rc 0) 00000100:00100000:1.0:1788619926.402306:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fe680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571614976:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2595us (2916us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619926.402314:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1326 00000100:00100000:2.0:1788619926.405120:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619926.405193:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571615104 00000100:00100000:0.0:1788619926.405203:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571615104 00000100:00100000:0.0:1788619926.405299:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1327 00000100:00100000:0.0:1788619926.405310:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571615104:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619926.405361:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619926.405384:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4a00/0x1cd7fe5368735ee7 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d0cf expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.405428:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe5368735e69 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d051 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619926.405483:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f77b80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589204224:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619926.405572:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590840 going to sleep for 11 seconds 00000100:00100000:2.0:1788619926.407720:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f77b80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589204224:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619926.407744:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:2.0:1788619926.407761:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4a00/0x1cd7fe5368735ee7 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d0cf expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.407781:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619926.407784:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619926.407789:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4a00/0x1cd7fe5368735ee7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d0cf expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.407808:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d4a00, rc 0) 00000100:00100000:2.0:1788619926.408102:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571615104:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2792us (2989us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619926.408111:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1327 00000100:00100000:1.0:1788619926.410594:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619926.410689:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571615232 00000100:00100000:3.0:1788619926.410699:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571615232 00000100:00100000:3.0:1788619926.410780:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1095 00000100:00100000:3.0:1788619926.410790:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69d2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571615232:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619926.410808:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619926.410827:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1c6e115/0x1cd7fe5368735eee lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d0d6 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619926.410886:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6858380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589204352:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619926.411026:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeba80 going to sleep for 11 seconds 00000100:00100000:2.0:1788619926.413615:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6858380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589204352:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619926.413665:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b1c6e115/0x1cd7fe5368735eee lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d0d6 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.413691:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1c6e115/0x1cd7fe5368735eee lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d0d6 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.413705:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f70200, rc 0) 00000100:00100000:2.0:1788619926.413803:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69d2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571615232:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3015us (3218us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619926.413812:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1095 00000100:00100000:0.0:1788619926.417052:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619926.417159:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571615360 00000100:00100000:2.0:1788619926.417169:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571615360 00000100:00100000:2.0:1788619926.417272:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1096 00000100:00100000:2.0:1788619926.417283:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f75c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571615360:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619926.417301:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619926.417317:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002dd9059c/0x1cd7fe5368735ef5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d0dd expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619926.417408:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a58a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589204480:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619926.417488:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164240 going to sleep for 11 seconds 00000100:00100000:2.0:1788619926.419635:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a58a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589204480:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619926.419673:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002dd9059c/0x1cd7fe5368735ef5 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d0dd expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.419698:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002dd9059c/0x1cd7fe5368735ef5 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d0dd expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.419712:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f73c00, rc 0) 00000100:00100000:2.0:1788619926.419769:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f75c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571615360:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2488us (2727us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619926.419778:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1096 00000100:00100000:0.0:1788619926.422679:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619926.422780:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571615488 00000100:00100000:3.0:1788619926.422790:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571615488 00002000:00100000:3.0:1788619926.422847:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1552384->1556479) req@ffff8e77c5f77480 x1875503571615488/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:22/0 lens 488/0 e 0 to 0 dl 1788619937 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619926.422858:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5f77480 with x1875503571615488 ext(1552384->1556479) 00002000:00010000:3.0:1788619926.422864:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5f77480 00000100:00100000:3.0:1788619926.422957:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 632 00000100:00100000:3.0:1788619926.422967:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f77480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571615488:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619926.423212:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f77480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571615488:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 246us (542us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619926.423220:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 632 00002000:00100000:3.0:1788619926.423227:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1552384->1556479) req@ffff8e77c5f77480 x1875503571615488/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:22/0 lens 488/4536 e 0 to 0 dl 1788619937 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619926.423236:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5f77480 with x1875503571615488 ext(1552384->1556479) 00002000:00010000:3.0:1788619926.423239:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5f77480 00000100:00100000:0.0:1788619926.428474:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619926.428773:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571615616 00000100:00100000:2.0:1788619926.428785:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571615616 00002000:00100000:2.0:1788619926.428839:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1560576->1564671) req@ffff8e77c5f75180 x1875503571615616/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:22/0 lens 488/0 e 0 to 0 dl 1788619937 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619926.428850:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5f75180 with x1875503571615616 ext(1560576->1564671) 00002000:00010000:2.0:1788619926.428856:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5f75180 00000100:00100000:2.0:1788619926.428945:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 633 00000100:00100000:2.0:1788619926.428956:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f75180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571615616:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619926.429249:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f75180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571615616:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 294us (785us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619926.429256:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 633 00002000:00100000:2.0:1788619926.429263:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1560576->1564671) req@ffff8e77c5f75180 x1875503571615616/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:22/0 lens 488/4536 e 0 to 0 dl 1788619937 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619926.429271:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5f75180 with x1875503571615616 ext(1560576->1564671) 00002000:00010000:2.0:1788619926.429276:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5f75180 00000100:00100000:3.0:1788619926.433154:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619926.433387:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571615744 00000100:00100000:1.0:1788619926.433398:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571615744 00000100:00100000:1.0:1788619926.433468:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1328 00000100:00100000:1.0:1788619926.433479:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6858380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571615744:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619926.433499:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619926.433520:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296600/0x1cd7fe5368735efc lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d0e4 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.433557:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe5368735e69 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d051 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619926.433598:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69d2300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589204608:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619926.433688:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc300 going to sleep for 11 seconds 00000100:00100000:0.0:1788619926.436119:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69d2300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589204608:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619926.436146:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:0.0:1788619926.436166:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296600/0x1cd7fe5368735efc lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d0e4 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.436187:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619926.436190:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619926.436197:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296600/0x1cd7fe5368735efc lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d0e4 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.436215:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296600, rc 0) 00000100:00100000:0.0:1788619926.436323:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6858380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571615744:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2844us (3175us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619926.436377:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1328 00000100:00100000:3.0:1788619926.439194:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619926.439295:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571615872 00000100:00100000:1.0:1788619926.439305:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571615872 00000100:00100000:1.0:1788619926.439423:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1329 00000100:00100000:1.0:1788619926.439434:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e03100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571615872:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619926.439465:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619926.439489:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297800/0x1cd7fe5368735f03 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d0eb expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.439527:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe5368735e69 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d051 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619926.439567:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69d3100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589204736:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619926.439651:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccc00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619926.441856:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69d3100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589204736:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619926.441895:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:2.0:1788619926.441923:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297800/0x1cd7fe5368735f03 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d0eb expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.441944:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619926.441947:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619926.441953:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297800/0x1cd7fe5368735f03 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d0eb expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.441967:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297800, rc 0) 00000100:00100000:2.0:1788619926.442022:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e03100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571615872:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2590us (2838us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619926.442031:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1329 00000100:00100000:0.0:1788619926.445033:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619926.445188:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571616000 00000100:00100000:2.0:1788619926.445198:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571616000 00000100:00100000:2.0:1788619926.445246:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1097 00000100:00100000:2.0:1788619926.445257:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f77100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571616000:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619926.445275:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619926.445293:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002dd9059c/0x1cd7fe5368735f0a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d0f2 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619926.445382:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a58a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589204864:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619926.445467:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164240 going to sleep for 11 seconds 00000100:00100000:2.0:1788619926.447193:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a58a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589204864:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619926.447226:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002dd9059c/0x1cd7fe5368735f0a lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d0f2 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.447250:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002dd9059c/0x1cd7fe5368735f0a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d0f2 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.447262:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f73c00, rc 0) 00000100:00100000:2.0:1788619926.447307:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f77100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571616000:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2052us (2284us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619926.447315:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1097 00000100:00100000:0.0:1788619926.453644:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619926.453778:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571616128 00000100:00100000:1.0:1788619926.453789:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571616128 00002000:00100000:1.0:1788619926.453842:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5f74700 x1875503571616128/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:22/0 lens 488/0 e 0 to 0 dl 1788619937 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619926.453854:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5f74700 with x1875503571616128 00000100:00100000:1.0:1788619926.453893:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 811 00000100:00100000:1.0:1788619926.453904:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f74700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571616128:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:1.0:1788619926.457871:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 987136 -> 987136 00000004:00010000:1.0:1788619926.457879:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619926 -> 1788619926 00000004:00010000:1.0:1788619926.457881:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619926 -> 1788619924 00000004:00010000:1.0:1788619926.457883:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619926 -> 1788619926 00000004:00010000:1.0:1788619926.457885:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1544 -> 1544 00000100:00100000:1.0:1788619926.457963:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f74700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571616128:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 4052us (4321us total) trans 4294967767 rc 0/0 00000100:00100000:1.0:1788619926.457975:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 811 00002000:00100000:1.0:1788619926.457988:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5f74700 x1875503571616128/t4294967767(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:22/0 lens 488/448 e 0 to 0 dl 1788619937 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619926.457998:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5f74700 with x1875503571616128 00000100:00100000:0.0:1788619926.460294:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619926.460558:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571616256 00000100:00100000:2.0:1788619926.460569:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571616256 00000100:00100000:2.0:1788619926.460673:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1330 00000100:00100000:2.0:1788619926.460685:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f76d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571616256:12345-192.168.201.5@tcp:16:fsx.0 00000001:00080000:3.0:1788619926.484714:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967767 is committed 00000100:00100000:0.0:1788619926.484882:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f76d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571616256:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 24195us (24593us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619926.484904:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1330 00000100:00100000:0.0:1788619926.487857:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619926.487971:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571616384 00000100:00100000:2.0:1788619926.487981:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571616384 00000100:00100000:2.0:1788619926.488040:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1331 00000100:00100000:2.0:1788619926.488051:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f74380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571616384:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619926.488073:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619926.488094:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73c00/0x1cd7fe5368735f11 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d0f9 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.488131:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe5368735e69 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d051 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619926.488174:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a58a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589204992:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619926.488258:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81649c0 going to sleep for 11 seconds 00000004:00010000:2.0:1788619926.490090:0:5846:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619924 -> 1788619926 00000100:00100000:2.0:1788619926.490124:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a58a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589204992:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619926.490150:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:2.0:1788619926.490168:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73c00/0x1cd7fe5368735f11 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d0f9 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.490189:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619926.490193:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619926.490199:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73c00/0x1cd7fe5368735f11 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d0f9 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.490213:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f73c00, rc 0) 00000100:00100000:2.0:1788619926.490371:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f74380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571616384:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2273us (2475us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619926.490381:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1331 00000100:00100000:0.0:1788619926.493480:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619926.493610:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571616512 00000100:00100000:2.0:1788619926.493629:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571616512 00000100:00100000:2.0:1788619926.493691:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1332 00000100:00100000:2.0:1788619926.493701:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f77b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571616512:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619926.493722:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619926.493741:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73800/0x1cd7fe5368735f18 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d100 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.493781:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe5368735e69 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d051 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619926.493820:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a58a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589205120:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619926.493916:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81649c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619926.496046:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a58a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589205120:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619926.496073:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:2.0:1788619926.496092:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73800/0x1cd7fe5368735f18 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d100 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.496113:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619926.496116:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619926.496121:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73800/0x1cd7fe5368735f18 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d100 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.496137:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f73800, rc 0) 00000100:00100000:2.0:1788619926.496196:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f77b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571616512:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2495us (2728us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619926.496204:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1332 00000100:00100000:3.0:1788619926.498688:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619926.498807:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571616768 00000100:00100000:2.0:1788619926.498818:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571616768 00000100:00100000:2.0:1788619926.498873:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1333 00000100:00100000:2.0:1788619926.498889:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e01880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571616768:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:2.0:1788619926.498955:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619926.498983:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5c00/0x1cd7fe5368735d7b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf86 expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.499006:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619926.499012:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788619926.499278:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73200/0x1cd7fe5368735f1f lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.499294:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4400/0x1cd7fe5368735d51 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf63 expref: 8 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.499302:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73200/0x1cd7fe5368735f1f lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.499384:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4400/0x1cd7fe5368735d51 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf63 expref: 8 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.499397:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4400/0x1cd7fe5368735d51 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf63 expref: 8 pid: 5844 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619926.499412:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a58a80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589205248:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619926.499501:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81649c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619926.502778:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a58a80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589205248:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619926.502808:0:5845:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73200/0x1cd7fe5368735f1f lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619926.503547:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619926.503801:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571616896 00000100:00100000:1.0:1788619926.503810:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571616896 00010000:00010000:1.0:1788619926.503850:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4400/0x1cd7fe5368735d51 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf63 expref: 9 pid: 5844 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619926.503895:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 436 00000100:00100000:1.0:1788619926.503912:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e03800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571616896:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619926.503926:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619926.503933:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4400/0x1cd7fe5368735d51 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf63 expref: 9 pid: 5844 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.503946:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4400/0x1cd7fe5368735d51 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf63 expref: 9 pid: 5844 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.503969:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4400/0x1cd7fe5368735d51 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cf63 expref: 9 pid: 5844 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.503986:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619926.503991:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73200/0x1cd7fe5368735f1f lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.504002:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73200/0x1cd7fe5368735f1f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.504010:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73200/0x1cd7fe5368735f1f lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.504041:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619926.504098:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e03800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+8:2355:x1875503571616896:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 194us (553us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619926.504105:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 436 00010000:00010000:0.0:1788619926.504174:0:5845:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73200/0x1cd7fe5368735f1f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.504183:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73200/0x1cd7fe5368735f1f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788619926.504298:0:5845:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967768 00000004:00010000:0.0:1788619926.504381:0:5845:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 987136 -> 987136 00000004:00010000:0.0:1788619926.504384:0:5845:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619926 -> 1788619926 00000004:00010000:0.0:1788619926.504387:0:5845:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619926 -> 1788619924 00000004:00010000:0.0:1788619926.504389:0:5845:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619926 -> 1788619926 00000004:00010000:0.0:1788619926.504391:0:5845:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1544 -> 1544 00000004:00010000:0.0:1788619926.504399:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000081304283 reply state 0000000019973c45 transno 4294967769 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73200/0x1cd7fe5368735f1f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619926.504409:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788619926.504480:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e01880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:14908:x1875503571616768:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 5598us (5801us total) trans 4294967769 rc 0/0 00000100:00100000:0.0:1788619926.504487:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1333 00000020:00080000:0.0:1788619926.504491:0:5845:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1788619926.507708:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00010000:00010000:2.0:1788619926.507831:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73200/0x1cd7fe5368735f1f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.507842:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73200/0x1cd7fe5368735f1f lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.507850:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73200/0x1cd7fe5368735f1f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.507856:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c5f73200 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:2.0:1788619926.507860:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73200/0x1cd7fe5368735f1f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.507868:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73200/0x1cd7fe5368735f1f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.507892:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619926.507895:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788619926.507900:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73200/0x1cd7fe5368735f1f lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.507919:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73200/0x1cd7fe5368735f1f lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619926.508017:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571617024 00000100:00100000:1.0:1788619926.508027:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571617024 00000100:00100000:1.0:1788619926.508081:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1098 00000100:00100000:1.0:1788619926.508090:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:14908:x1875503571617024:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619926.508107:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619926.508122:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002abacdb6/0x1cd7fe5368735f26 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d107 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.508151:0:6696:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000001d73b257/0x1cd7fe5368735eb6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1343488->1347583) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d09e expref: 5 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.508160:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=1s v=(8 13) 00010000:00010000:1.0:1788619926.508164:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(9 13) 00010000:00010000:1.0:1788619926.508169:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002abacdb6/0x1cd7fe5368735f26 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d107 expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.508202:0:6696:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000001d73b257/0x1cd7fe5368735eb6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1343488->1347583) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d09e expref: 5 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.508216:0:6696:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000001d73b257/0x1cd7fe5368735eb6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1343488->1347583) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d09e expref: 5 pid: 6696 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619926.508230:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69d0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589205376:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619926.508423:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc600 going to sleep for 11 seconds 00000100:00100000:1.0:1788619926.512877:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619926.512957:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571617152 00000100:00100000:3.0:1788619926.512966:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571617152 00002000:00100000:3.0:1788619926.513013:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1552384->1564671) req@ffff8e77c69d3100 x1875503571617152/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:22/0 lens 12776/0 e 0 to 0 dl 1788619937 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619926.513024:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c69d3100 with x1875503571617152 ext(1552384->1564671) 00010000:00010000:3.0:1788619926.513032:0:6702:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000001d73b257/0x1cd7fe5368735eb6 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1343488->1347583) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d09e expref: 6 pid: 6696 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.513050:0:6702:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000001d73b257/0x1cd7fe5368735eb6 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1343488->1347583) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d09e expref: 6 pid: 6696 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788619926.513057:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff8e77c69d3100 00000100:00100000:3.0:1788619926.513091:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 20 00000100:00100000:3.0:1788619926.513100:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69d3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571617152:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:3.0:1788619926.513629:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69d3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+8:2354:x1875503571617152:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 520us (753us total) trans 4294967482 rc 0/0 00000100:00100000:3.0:1788619926.513638:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 20 00002000:00100000:3.0:1788619926.513645:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1552384->1564671) req@ffff8e77c69d3100 x1875503571617152/t4294967482(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:22/0 lens 12776/448 e 0 to 0 dl 1788619937 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619926.513654:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c69d3100 with x1875503571617152 ext(1552384->1564671) 00010000:00010000:3.0:1788619926.513660:0:6702:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000001d73b257/0x1cd7fe5368735eb6 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1343488->1347583) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d09e expref: 8 pid: 6696 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.513673:0:6702:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000001d73b257/0x1cd7fe5368735eb6 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1343488->1347583) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d09e expref: 8 pid: 6696 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788619926.513686:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff8e77c69d3100 00000100:00100000:2.0:1788619926.514191:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69d0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589205376:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619926.514218:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002abacdb6/0x1cd7fe5368735f26 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d107 expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.514234:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296200, rc 0) 00000100:00100000:2.0:1788619926.514256:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:14908:x1875503571617024:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6168us (6558us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619926.514263:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1098 00000100:00100000:1.0:1788619926.517202:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619926.517276:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571617280 00000100:00100000:3.0:1788619926.517284:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571617280 00010000:00010000:3.0:1788619926.517318:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000001d73b257/0x1cd7fe5368735eb6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1343488->1347583) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d09e expref: 8 pid: 6696 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619926.517406:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 437 00000100:00100000:3.0:1788619926.517415:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69d2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+8:2354:x1875503571617280:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619926.520735:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619926.520783:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000001d73b257/0x1cd7fe5368735eb6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1343488->1347583) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d09e expref: 8 pid: 6696 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.520804:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000001d73b257/0x1cd7fe5368735eb6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1343488->1347583) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d09e expref: 8 pid: 6696 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.520830:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d73b257/0x1cd7fe5368735eb6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1343488->1347583) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d09e expref: 8 pid: 6696 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.520845:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(9 13) 00010000:00010000:3.0:1788619926.520850:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(9 13) 00010000:00010000:3.0:1788619926.520858:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000002abacdb6/0x1cd7fe5368735f26 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d107 expref: 5 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.520868:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002abacdb6/0x1cd7fe5368735f26 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d107 expref: 5 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.520937:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000002abacdb6/0x1cd7fe5368735f26 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d107 expref: 5 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619926.520955:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900e01c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589205504:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619926.521051:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb540 going to sleep for 11 seconds 00000100:00100000:3.0:1788619926.524562:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900e01c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589205504:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619926.524588:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619926.525074:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69d2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571617280:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 7658us (7879us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619926.525086:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 437 00000100:00100000:2.0:1788619926.525280:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619926.525879:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571617408 00000100:00100000:3.0:1788619926.525897:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571617408 00002000:00010000:3.0:1788619926.525933:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1550849->18446744073709551615) 00002000:00100000:3.0:1788619926.525937:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a58a80 with x1875503571617408 ext(1550849->18446744073709551615) 00002000:00010000:3.0:1788619926.525948:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a58a80. 00000100:00100000:3.0:1788619926.525984:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 634 00000100:00100000:3.0:1788619926.525992:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571617408:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:3.0:1788619926.526274:0:6703:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1860930 -> 1550849 00002000:00010000:3.0:1788619926.526279:0:6703:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619926 -> 1788619926 00002000:00010000:3.0:1788619926.526281:0:6703:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619926 -> 1788619925 00002000:00010000:3.0:1788619926.526284:0:6703:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619926 -> 1788619926 00002000:00010000:3.0:1788619926.526286:0:6703:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 647 -> 391 00000100:00100000:3.0:1788619926.526357:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571617408:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 364us (1077us total) trans 4294967483 rc 0/0 00000100:00100000:3.0:1788619926.526365:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 634 00002000:00010000:3.0:1788619926.526369:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1550849->18446744073709551615) 00002000:00100000:3.0:1788619926.526371:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a58a80 with x1875503571617408 ext(1550849->18446744073709551615) 00002000:00010000:3.0:1788619926.526376:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a58a80. 00000100:00100000:2.0:1788619926.529690:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619926.529785:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571617536 00000100:00100000:0.0:1788619926.529795:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571617536 00000100:00100000:0.0:1788619926.529892:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1334 00000100:00100000:0.0:1788619926.529904:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a59f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+8:14908:x1875503571617536:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619926.529925:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619926.529945:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5400/0x1cd7fe5368735f2d lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d10e expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.530035:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4600/0x1cd7fe5368735f34 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.530052:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4600/0x1cd7fe5368735f34 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.530064:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4600/0x1cd7fe5368735f34 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619926.530092:0:5846:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1550849 00000004:00200000:0.0:1788619926.530098:0:5846:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619926.530118:0:5846:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4600/0x1cd7fe5368735f34 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.530147:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5400/0x1cd7fe5368735f2d lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d10e expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.530166:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4600/0x1cd7fe5368735f34 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d10e expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.530175:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619926.530179:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619926.530183:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d4600, rc 0) 00000100:00100000:0.0:1788619926.530247:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a59f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571617536:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 346us (566us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619926.530253:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1334 00000100:00100000:2.0:1788619926.533157:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619926.533259:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571617664 00000100:00100000:1.0:1788619926.533268:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571617664 00000100:00100000:1.0:1788619926.533363:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1335 00000100:00100000:1.0:1788619926.533374:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a58380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571617664:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619926.533393:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619926.533413:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297200/0x1cd7fe5368735f3b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d115 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.533456:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe5368735e69 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d051 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619926.533496:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69d0380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589205632:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619926.533576:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc000 going to sleep for 11 seconds 00000100:00100000:2.0:1788619926.535922:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619926.536073:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571617792 00000100:00100000:3.0:1788619926.536080:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571617792 00000100:00100000:3.0:1788619926.536132:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1099 00000100:00100000:3.0:1788619926.536142:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571617792:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619926.536158:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619926.536181:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e9c3b213/0x1cd7fe5368735f42 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d11c expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619926.536240:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900e01c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589205760:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619926.536323:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb540 going to sleep for 11 seconds 00000004:00010000:1.0:1788619926.537293:0:9417:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619924 -> 1788619926 00000100:00100000:1.0:1788619926.537318:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69d0380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589205632:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619926.537369:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:1.0:1788619926.537386:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297200/0x1cd7fe5368735f3b lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d115 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.537404:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619926.537407:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619926.537412:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297200/0x1cd7fe5368735f3b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d115 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.537426:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297200, rc 0) 00000100:00100000:1.0:1788619926.537573:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a58380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571617664:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4200us (4425us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619926.537581:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1335 00002000:00010000:3.0:1788619926.541227:0:7654:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619925 -> 1788619926 00000100:00100000:3.0:1788619926.541257:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900e01c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589205760:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619926.541291:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e9c3b213/0x1cd7fe5368735f42 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d11c expref: 8 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.541315:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e9c3b213/0x1cd7fe5368735f42 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d11c expref: 8 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.541359:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f71200, rc 0) 00000100:00100000:3.0:1788619926.541480:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571617792:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5339us (5564us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619926.541491:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1099 00000100:00100000:2.0:1788619926.544759:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619926.544842:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619926.545100:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571617920 00000100:00100000:0.0:1788619926.545110:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571617920 00000100:00100000:0.0:1788619926.545202:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1336 00000100:00100000:0.0:1788619926.545213:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a59500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571617920:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619926.545233:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619926.545252:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5600/0x1cd7fe5368735f49 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d123 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.545286:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe5368735e69 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d051 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619926.545327:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f5180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589205888:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619926.545409:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590600 going to sleep for 11 seconds 00000100:00100000:1.0:1788619926.545495:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571618048 00000100:00100000:1.0:1788619926.545500:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571618048 00000100:00100000:1.0:1788619926.545530:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1100 00000100:00100000:1.0:1788619926.545538:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571618048:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619926.545547:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619926.545559:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008079352d/0x1cd7fe5368735f50 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d12a expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619926.545594:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69d0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589206016:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619926.545637:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc000 going to sleep for 11 seconds 00000100:00100000:1.0:1788619926.548421:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69d0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589206016:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619926.548472:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008079352d/0x1cd7fe5368735f50 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d12a expref: 8 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.548497:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008079352d/0x1cd7fe5368735f50 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d12a expref: 8 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.548509:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296c00, rc 0) 00000100:00100000:1.0:1788619926.548570:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571618048:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3032us (3821us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619926.548578:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1100 00000100:00100000:2.0:1788619926.549723:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f5180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589205888:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619926.549746:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:2.0:1788619926.549763:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5600/0x1cd7fe5368735f49 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d123 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.549779:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619926.549782:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619926.549787:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5600/0x1cd7fe5368735f49 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d123 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.549800:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d5600, rc 0) 00000100:00100000:2.0:1788619926.549856:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a59500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571617920:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4646us (5014us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619926.549864:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1336 00000100:00100000:0.0:1788619926.554089:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619926.554207:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571618176 00000100:00100000:2.0:1788619926.554220:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571618176 00000100:00100000:2.0:1788619926.554290:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1337 00000100:00100000:2.0:1788619926.554301:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571618176:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619926.554349:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619926.554375:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735f57 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d131 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.554407:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe5368735e69 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d051 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.554419:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735f57 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d131 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.554466:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe5368735e69 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d051 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.554480:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe5368735e69 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d051 expref: 14 pid: 5846 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619926.554496:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a5aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589206144:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619926.554587:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81649c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619926.558297:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619926.558472:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a5aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589206144:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619926.558508:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735f57 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d131 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.558528:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619926.558532:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619926.558536:0:5846:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735f57 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d131 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.558547:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f73400, rc 0) 00000100:00100000:2.0:1788619926.558608:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:14908:x1875503571618176:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4308us (4527us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619926.558625:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1337 00000100:00100000:0.0:1788619926.558713:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571618304 00000100:00100000:0.0:1788619926.558720:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571618304 00010000:00010000:0.0:1788619926.558755:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe5368735e69 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d051 expref: 15 pid: 5846 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619926.558792:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 438 00000100:00100000:0.0:1788619926.558801:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7563c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571618304:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619926.558813:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619926.558838:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe5368735e69 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d051 expref: 15 pid: 5846 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.558851:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe5368735e69 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d051 expref: 15 pid: 5846 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.558871:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe5368735e69 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d051 expref: 15 pid: 5846 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.558898:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619926.558912:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735f57 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d131 expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.558922:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735f57 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d131 expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.558931:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735f57 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d131 expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.558966:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735f57 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d131 expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619926.558985:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f6300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589206272:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619926.559037:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590840 going to sleep for 11 seconds 00000100:00100000:0.0:1788619926.562298:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619926.562431:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571618432 00000100:00100000:2.0:1788619926.562451:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571618432 00002000:00100000:2.0:1788619926.562504:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e67f4700 x1875503571618432/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:22/0 lens 488/0 e 0 to 0 dl 1788619937 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619926.562515:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e67f4700 with x1875503571618432 00000100:00100000:2.0:1788619926.562554:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 812 00000100:00100000:2.0:1788619926.562566:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571618432:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619926.562840:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571618432:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 276us (551us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619926.562846:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 812 00002000:00100000:2.0:1788619926.562851:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e67f4700 x1875503571618432/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:22/0 lens 488/4536 e 0 to 0 dl 1788619937 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619926.562856:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e67f4700 with x1875503571618432 00000100:00100000:1.0:1788619926.563506:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f6300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589206272:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619926.563527:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619926.563568:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7563c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571618304:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 4768us (5282us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619926.563576:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 438 00000100:00100000:0.0:1788619926.566717:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619926.566865:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571618560 00000100:00100000:1.0:1788619926.566874:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571618560 00002000:00100000:1.0:1788619926.566933:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e67f7100 x1875503571618560/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:22/0 lens 488/0 e 0 to 0 dl 1788619937 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619926.566943:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e67f7100 with x1875503571618560 00000100:00100000:1.0:1788619926.566984:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 813 00000100:00100000:1.0:1788619926.566993:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571618560:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619926.567221:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571618560:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 228us (513us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619926.567228:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 813 00002000:00100000:1.0:1788619926.567234:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e67f7100 x1875503571618560/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:22/0 lens 488/4536 e 0 to 0 dl 1788619937 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619926.567242:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e67f7100 with x1875503571618560 00000100:00100000:0.0:1788619926.571169:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619926.571464:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571618688 00000100:00100000:2.0:1788619926.571474:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571618688 00000100:00100000:2.0:1788619926.571536:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1101 00000100:00100000:2.0:1788619926.571547:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571618688:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619926.571565:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619926.571582:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000644e85bb/0x1cd7fe5368735f5e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d138 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619926.571650:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a5aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589206400:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619926.571722:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164f00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619926.573785:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a5aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589206400:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619926.573820:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000644e85bb/0x1cd7fe5368735f5e lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d138 expref: 8 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.573851:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000644e85bb/0x1cd7fe5368735f5e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d138 expref: 8 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.573864:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f72400, rc 0) 00000100:00100000:2.0:1788619926.573952:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571618688:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2406us (2794us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619926.573960:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1101 00000100:00100000:0.0:1788619926.577145:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619926.577233:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571618816 00000100:00100000:2.0:1788619926.577241:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571618816 00000100:00100000:2.0:1788619926.577344:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1102 00000100:00100000:2.0:1788619926.577353:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571618816:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619926.577370:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619926.577389:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c3458c0e/0x1cd7fe5368735f65 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d13f expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619926.577459:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a5aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589206528:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619926.577539:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164f00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619926.579583:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a5aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589206528:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619926.579636:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c3458c0e/0x1cd7fe5368735f65 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d13f expref: 8 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.579659:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c3458c0e/0x1cd7fe5368735f65 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d13f expref: 8 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.579679:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f73e00, rc 0) 00000100:00100000:2.0:1788619926.579727:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571618816:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2374us (2592us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619926.579736:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1102 00000100:00100000:0.0:1788619926.582636:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619926.582723:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571618944 00000100:00100000:3.0:1788619926.582733:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571618944 00000100:00100000:3.0:1788619926.582797:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1103 00000100:00100000:3.0:1788619926.582807:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571618944:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619926.582826:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619926.582843:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e9c3b213/0x1cd7fe5368735f6c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d146 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619926.582925:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900e01c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589206656:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619926.583009:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb540 going to sleep for 11 seconds 00000100:00100000:3.0:1788619926.585123:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900e01c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589206656:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619926.585159:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e9c3b213/0x1cd7fe5368735f6c lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d146 expref: 8 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.585183:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e9c3b213/0x1cd7fe5368735f6c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d146 expref: 8 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.585197:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f71200, rc 0) 00000100:00100000:3.0:1788619926.585250:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571618944:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2444us (2634us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619926.585259:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1103 00000100:00100000:0.0:1788619926.588206:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619926.588313:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571619072 00000100:00100000:1.0:1788619926.588350:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571619072 00000100:00100000:1.0:1788619926.588408:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1104 00000100:00100000:1.0:1788619926.588419:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571619072:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619926.588437:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619926.588467:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003086a8b7/0x1cd7fe5368735f73 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d14d expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619926.588531:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69d2300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589206784:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619926.588613:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc300 going to sleep for 11 seconds 00000100:00100000:1.0:1788619926.590757:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69d2300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589206784:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619926.590798:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003086a8b7/0x1cd7fe5368735f73 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d14d expref: 8 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.590823:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003086a8b7/0x1cd7fe5368735f73 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d14d expref: 8 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.590836:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296400, rc 0) 00000100:00100000:1.0:1788619926.591048:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571619072:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2630us (2851us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619926.591058:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1104 00000100:00100000:0.0:1788619926.593852:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619926.594174:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571619200 00000100:00100000:1.0:1788619926.594184:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571619200 00000100:00100000:1.0:1788619926.594280:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1338 00000100:00100000:1.0:1788619926.594291:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe060000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571619200:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619926.594315:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619926.594363:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe5368735f7a lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d154 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.594458:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe5368735f81 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.594478:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe5368735f81 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.594490:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe5368735f81 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788619926.594520:0:9417:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1550849 00000004:00200000:1.0:1788619926.594525:0:9417:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788619926.594545:0:9417:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe5368735f81 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.594572:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe5368735f7a lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d154 expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.594592:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe5368735f81 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d154 expref: 15 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.594600:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619926.594603:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788619926.594606:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297600, rc 0) 00000100:00100000:1.0:1788619926.594676:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe060000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:14908:x1875503571619200:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 387us (834us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619926.594682:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1338 00000100:00100000:0.0:1788619926.597305:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619926.597706:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571619328 00000100:00100000:1.0:1788619926.597715:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571619328 00000100:00100000:1.0:1788619926.597807:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1339 00000100:00100000:1.0:1788619926.597818:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe061f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571619328:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619926.597840:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619926.597859:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296a00/0x1cd7fe5368735f88 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d15b expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.597913:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735f57 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d131 expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619926.597952:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69d2300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589206912:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619926.598036:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccc00 going to sleep for 11 seconds 00000100:00100000:1.0:1788619926.600271:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69d2300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589206912:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619926.600297:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:1.0:1788619926.600316:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296a00/0x1cd7fe5368735f88 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d15b expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.600353:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619926.600357:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619926.600363:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296a00/0x1cd7fe5368735f88 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d15b expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.600378:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296a00, rc 0) 00000100:00100000:1.0:1788619926.600481:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe061f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571619328:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2663us (3186us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619926.600490:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1339 00000100:00100000:0.0:1788619926.603599:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619926.603707:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571619456 00000100:00100000:2.0:1788619926.603717:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571619456 00000100:00100000:2.0:1788619926.603802:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1340 00000100:00100000:2.0:1788619926.603812:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe061180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571619456:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619926.603831:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619926.603849:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73e00/0x1cd7fe5368735f8f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d162 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.603894:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735f57 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d131 expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619926.604041:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a5aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589207040:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619926.604118:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164f00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619926.606626:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a5aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589207040:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619926.606652:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:2.0:1788619926.606670:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73e00/0x1cd7fe5368735f8f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d162 expref: 16 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.606692:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619926.606695:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619926.606703:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73e00/0x1cd7fe5368735f8f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d162 expref: 16 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.606718:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f73e00, rc 0) 00000100:00100000:2.0:1788619926.606818:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe061180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571619456:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3006us (3227us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619926.606828:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1340 00000100:00100000:0.0:1788619926.610558:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619926.610667:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571619584 00000100:00100000:3.0:1788619926.610677:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571619584 00000100:00100000:3.0:1788619926.610778:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 814 00000100:00100000:3.0:1788619926.610788:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe061500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571619584:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:3.0:1788619926.611245:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 987136 -> 987136 00000004:00010000:3.0:1788619926.611250:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619926 -> 1788619926 00000004:00010000:3.0:1788619926.611252:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619926 -> 1788619926 00000004:00010000:3.0:1788619926.611254:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619926 -> 1788619926 00000004:00010000:3.0:1788619926.611256:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1544 -> 1544 00000100:00100000:3.0:1788619926.611534:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe061500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571619584:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 747us (984us total) trans 4294967770 rc 0/0 00000100:00100000:3.0:1788619926.611544:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 814 00000100:00100000:1.0:1788619926.618491:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619926.618740:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571619712 00000100:00100000:3.0:1788619926.618751:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571619712 00000100:00100000:3.0:1788619926.618816:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1341 00000100:00100000:3.0:1788619926.618828:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69d2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571619712:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619926.618850:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619926.618873:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71200/0x1cd7fe5368735f96 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d169 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.619139:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735f57 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d131 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619926.619183:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900e01c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589207168:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619926.619269:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb540 going to sleep for 11 seconds 00000100:00100000:3.0:1788619926.623518:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900e01c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589207168:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619926.623547:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:3.0:1788619926.623567:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71200/0x1cd7fe5368735f96 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d169 expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.623590:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619926.623593:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619926.623598:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71200/0x1cd7fe5368735f96 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d169 expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.623613:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f71200, rc 0) 00000100:00100000:3.0:1788619926.624053:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69d2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571619712:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5225us (5571us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619926.624063:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1341 00000100:00100000:0.0:1788619926.626772:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619926.626846:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571619840 00000100:00100000:1.0:1788619926.626856:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571619840 00000100:00100000:1.0:1788619926.627244:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1342 00000100:00100000:1.0:1788619926.627256:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6532300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571619840:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619926.627277:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619926.627296:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296a00/0x1cd7fe5368735f9d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d170 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.627357:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735f57 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d131 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619926.627400:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69d0380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589207296:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619926.627612:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccc00 going to sleep for 11 seconds 00000100:00100000:3.0:1788619926.629988:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69d0380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589207296:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619926.630016:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:3.0:1788619926.630036:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296a00/0x1cd7fe5368735f9d lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d170 expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.630057:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619926.630060:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619926.630066:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296a00/0x1cd7fe5368735f9d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d170 expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.630082:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296a00, rc 0) 00000100:00100000:3.0:1788619926.630143:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6532300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571619840:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2890us (3379us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619926.630152:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1342 00000100:00100000:0.0:1788619926.632867:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619926.633057:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571619968 00000100:00100000:2.0:1788619926.633067:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571619968 00000100:00100000:2.0:1788619926.633128:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1343 00000100:00100000:2.0:1788619926.633138:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6533100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571619968:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619926.633158:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619926.633185:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73e00/0x1cd7fe5368735fa4 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d177 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.633229:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735f57 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d131 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619926.633275:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a5aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589207424:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619926.633384:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164f00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619926.636078:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a5aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589207424:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619926.636106:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:2.0:1788619926.636128:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73e00/0x1cd7fe5368735fa4 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d177 expref: 16 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.636150:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619926.636153:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619926.636160:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73e00/0x1cd7fe5368735fa4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d177 expref: 16 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.636174:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f73e00, rc 0) 00000100:00100000:2.0:1788619926.636237:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6533100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571619968:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3099us (3378us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619926.636246:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1343 00000100:00100000:1.0:1788619926.639522:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619926.639624:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571620096 00000100:00100000:0.0:1788619926.639633:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571620096 00000100:00100000:0.0:1788619926.639690:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1344 00000100:00100000:0.0:1788619926.639700:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69d0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:14908:x1875503571620096:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619926.639719:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619926.639740:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4400/0x1cd7fe5368735fab lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d17e expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.639768:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735f57 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d131 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.639778:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4400/0x1cd7fe5368735fab lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d17e expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.639812:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735f57 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d131 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.639825:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735f57 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d131 expref: 10 pid: 5846 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619926.639839:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6530380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589207552:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619926.640062:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590300 going to sleep for 11 seconds 00000100:00100000:0.0:1788619926.643223:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619926.643355:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6530380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589207552:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619926.643394:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4400/0x1cd7fe5368735fab lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d17e expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.643413:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619926.643416:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619926.643420:0:5844:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4400/0x1cd7fe5368735fab lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d17e expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.643430:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d4400, rc 0) 00000100:00100000:2.0:1788619926.643494:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69d0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:14908:x1875503571620096:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3794us (3979us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619926.643503:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1344 00000100:00100000:1.0:1788619926.643640:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571620224 00000100:00100000:1.0:1788619926.643647:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571620224 00002000:00100000:1.0:1788619926.643692:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6532a00 x1875503571620224/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:22/0 lens 488/0 e 0 to 0 dl 1788619937 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619926.643702:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6532a00 with x1875503571620224 00010000:00010000:1.0:1788619926.643709:0:5857:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735f57 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d131 expref: 11 pid: 5846 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.643721:0:5857:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735f57 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d131 expref: 11 pid: 5846 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788619926.643728:0:5857:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e77c6532a00 00000100:00100000:1.0:1788619926.643758:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 34 00000100:00100000:1.0:1788619926.643767:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6532a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571620224:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:2.0:1788619926.647557:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 987136 -> 987136 00000004:00010000:2.0:1788619926.647566:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619926 -> 1788619926 00000004:00010000:2.0:1788619926.647569:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619926 -> 1788619926 00000004:00010000:2.0:1788619926.647571:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619926 -> 1788619926 00000004:00010000:2.0:1788619926.647574:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1544 -> 1544 00000100:00100000:2.0:1788619926.647654:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6532a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571620224:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 3883us (4435us total) trans 4294967771 rc 0/0 00000100:00100000:2.0:1788619926.647665:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 34 00002000:00100000:2.0:1788619926.647679:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6532a00 x1875503571620224/t4294967771(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:22/0 lens 488/448 e 0 to 0 dl 1788619937 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619926.647689:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6532a00 with x1875503571620224 00010000:00010000:2.0:1788619926.647698:0:5857:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735f57 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d131 expref: 13 pid: 5846 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.647713:0:5857:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735f57 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d131 expref: 13 pid: 5846 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788619926.647720:0:5857:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e77c6532a00 00000100:00100000:0.0:1788619926.651418:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619926.651513:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571620352 00000100:00100000:3.0:1788619926.651524:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571620352 00010000:00010000:3.0:1788619926.651569:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735f57 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d131 expref: 13 pid: 5846 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619926.651611:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 439 00000100:00100000:3.0:1788619926.651629:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6531f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+13:2353:x1875503571620352:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619926.651643:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619926.651669:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735f57 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d131 expref: 13 pid: 5846 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.651682:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735f57 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d131 expref: 13 pid: 5846 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.651706:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73400/0x1cd7fe5368735f57 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d131 expref: 13 pid: 5846 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.651723:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619926.651729:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4400/0x1cd7fe5368735fab lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d17e expref: 15 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.651741:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4400/0x1cd7fe5368735fab lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d17e expref: 15 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.651750:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4400/0x1cd7fe5368735fab lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d17e expref: 15 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.651786:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4400/0x1cd7fe5368735fab lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d17e expref: 15 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619926.651803:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900e01c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589207680:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619926.651892:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb540 going to sleep for 11 seconds 00000100:00100000:0.0:1788619926.654774:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619926.655111:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900e01c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589207680:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619926.655134:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619926.655193:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6531f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2353:x1875503571620352:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3565us (3784us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619926.655203:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 439 00000100:00100000:1.0:1788619926.655290:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571620480 00000100:00100000:1.0:1788619926.655298:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571620480 00002000:00100000:1.0:1788619926.655378:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6532680 x1875503571620480/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:22/0 lens 488/0 e 0 to 0 dl 1788619937 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619926.655390:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6532680 with x1875503571620480 00000100:00100000:1.0:1788619926.655410:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 815 00000100:00100000:1.0:1788619926.655420:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6532680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571620480:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619926.662004:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6532680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571620480:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 6576us (7228us total) trans 0 rc 65536/65536 00000100:00100000:3.0:1788619926.662019:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 815 00002000:00100000:3.0:1788619926.662032:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6532680 x1875503571620480/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:22/0 lens 488/440 e 0 to 0 dl 1788619937 ref 1 fl Complete:/600/0 rc 65536/65536 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619926.662043:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6532680 with x1875503571620480 00000100:00100000:0.0:1788619926.667204:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619926.667311:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571620608 00000100:00100000:2.0:1788619926.667322:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571620608 00002000:00100000:2.0:1788619926.667410:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6531c00 x1875503571620608/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:22/0 lens 488/0 e 0 to 0 dl 1788619937 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619926.667420:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6531c00 with x1875503571620608 00000100:00100000:2.0:1788619926.667464:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 816 00000100:00100000:2.0:1788619926.667474:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6531c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571620608:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619926.671605:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6531c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571620608:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 4121us (4401us total) trans 0 rc 36864/36864 00000100:00100000:2.0:1788619926.671629:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 816 00002000:00100000:2.0:1788619926.671641:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6531c00 x1875503571620608/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:22/0 lens 488/440 e 0 to 0 dl 1788619937 ref 1 fl Complete:/600/0 rc 36864/36864 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619926.671650:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6531c00 with x1875503571620608 00000100:00100000:0.0:1788619926.675276:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619926.675366:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571620736 00000100:00100000:3.0:1788619926.675376:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571620736 00002000:00100000:3.0:1788619926.675420:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6530a80 x1875503571620736/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:22/0 lens 488/0 e 0 to 0 dl 1788619937 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619926.675431:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6530a80 with x1875503571620736 00000100:00100000:3.0:1788619926.675479:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 817 00000100:00100000:3.0:1788619926.675488:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6530a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2352:x1875503571620736:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:3.0:1788619926.679131:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 987136 -> 987136 00000004:00010000:3.0:1788619926.679139:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619926 -> 1788619926 00000004:00010000:3.0:1788619926.679142:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619926 -> 1788619926 00000004:00010000:3.0:1788619926.679145:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619926 -> 1788619926 00000004:00010000:3.0:1788619926.679147:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1544 -> 1544 00000100:00100000:3.0:1788619926.679220:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6530a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2352:x1875503571620736:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 3728us (3949us total) trans 4294967772 rc 0/0 00000100:00100000:3.0:1788619926.679232:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 817 00002000:00100000:3.0:1788619926.679245:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6530a80 x1875503571620736/t4294967772(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:22/0 lens 488/448 e 0 to 0 dl 1788619937 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619926.679254:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6530a80 with x1875503571620736 00000100:00100000:0.0:1788619926.682134:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619926.682219:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571620864 00000100:00100000:3.0:1788619926.682228:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571620864 00000100:00100000:3.0:1788619926.682283:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1345 00000100:00100000:3.0:1788619926.682292:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6530380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2354:x1875503571620864:12345-192.168.201.5@tcp:16:fsx.0 00000100:00100000:3.0:1788619926.713747:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6530380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2354:x1875503571620864:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 31443us (31609us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619926.713762:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1345 00000001:00080000:2.0:1788619926.713851:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967769 is committed 00000001:00080000:2.0:1788619926.713864:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967771 is committed 00000001:00080000:2.0:1788619926.713876:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967772 is committed 00000100:00100000:3.0:1788619926.718107:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619926.718888:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571620992 00000100:00100000:2.0:1788619926.718898:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571620992 00000100:00100000:2.0:1788619926.718960:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1346 00000100:00100000:2.0:1788619926.718971:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e01c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571620992:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619926.718998:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619926.719017:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73e00/0x1cd7fe5368735fb2 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d185 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.719054:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4400/0x1cd7fe5368735fab lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d17e expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619926.719094:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a5aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589207808:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619926.719175:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164840 going to sleep for 11 seconds 00000100:00100000:2.0:1788619926.719944:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619926.720069:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571621120 00000100:00100000:3.0:1788619926.720075:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571621120 00000100:00100000:3.0:1788619926.720117:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1105 00000100:00100000:3.0:1788619926.720125:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5026a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+7:2352:x1875503571621120:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619926.720135:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619926.720148:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e9c3b213/0x1cd7fe5368735fb9 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d18c expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619926.720189:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900e03480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589207936:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619926.720254:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb540 going to sleep for 11 seconds 00000100:00100000:3.0:1788619926.724729:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900e03480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589207936:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619926.724766:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e9c3b213/0x1cd7fe5368735fb9 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d18c expref: 8 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.724790:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e9c3b213/0x1cd7fe5368735fb9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d18c expref: 8 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.724804:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f71200, rc 0) 00000100:00100000:3.0:1788619926.724870:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5026a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+7:2352:x1875503571621120:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4745us (4927us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619926.724879:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1105 00000004:00010000:2.0:1788619926.726653:0:5846:0:(mdt_lvb.c:201:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks: 1544 -> 1800 00000100:00100000:2.0:1788619926.726681:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a5aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589207808:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619926.726704:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:2.0:1788619926.726721:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73e00/0x1cd7fe5368735fb2 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d185 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.726737:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619926.726740:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619926.726745:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73e00/0x1cd7fe5368735fb2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d185 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.726763:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f73e00, rc 0) 00000100:00100000:2.0:1788619926.726819:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e01c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571620992:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7849us (8720us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619926.726827:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1346 00000100:00100000:0.0:1788619926.730692:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619926.730728:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619926.730845:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571621376 00000100:00100000:2.0:1788619926.730856:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571621376 00000100:00100000:2.0:1788619926.730921:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1106 00000100:00100000:2.0:1788619926.730931:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2352:x1875503571621376:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619926.730950:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619926.730967:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001544d1a8/0x1cd7fe5368735fc0 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d19a expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619926.731027:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5024a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589208064:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619926.731102:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164840 going to sleep for 11 seconds 00000100:00100000:3.0:1788619926.731591:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571621248 00000100:00100000:3.0:1788619926.731597:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571621248 00000100:00100000:3.0:1788619926.731652:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1347 00000100:00100000:3.0:1788619926.731660:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571621248:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619926.731670:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619926.731684:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70400/0x1cd7fe5368735fc7 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d193 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.731711:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4400/0x1cd7fe5368735fab lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d17e expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619926.731737:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900e03480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589208192:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619926.731787:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb540 going to sleep for 11 seconds 00000100:00100000:3.0:1788619926.737385:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900e03480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589208192:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619926.737423:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 987136 00010000:00010000:3.0:1788619926.737445:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70400/0x1cd7fe5368735fc7 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d193 expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.740962:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619926.740967:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619926.740977:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70400/0x1cd7fe5368735fc7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d193 expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.740999:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f70400, rc 0) 00000100:00100000:3.0:1788619926.741072:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571621248:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 9411us (10342us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619926.741081:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1347 00000100:00100000:2.0:1788619926.741922:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5024a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589208064:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619926.741961:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001544d1a8/0x1cd7fe5368735fc0 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d19a expref: 8 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.741979:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001544d1a8/0x1cd7fe5368735fc0 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d19a expref: 8 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.741989:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f72800, rc 0) 00000100:00100000:2.0:1788619926.742035:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2352:x1875503571621376:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 11105us (11353us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619926.742045:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1106 00000100:00100000:0.0:1788619926.752064:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619926.752109:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571621632 00000100:00100000:3.0:1788619926.752118:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571621632 00000100:00100000:3.0:1788619926.752236:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1348 00000100:00100000:3.0:1788619926.752247:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571621632:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:3.0:1788619926.752314:0:9417:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619926.752368:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4600/0x1cd7fe5368735f34 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d10e expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.752386:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619926.752391:0:9417:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788619926.752674:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71200/0x1cd7fe5368735fce lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.752689:0:9417:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe5368735f81 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d154 expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.752698:0:9417:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71200/0x1cd7fe5368735fce lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.752740:0:9417:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe5368735f81 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d154 expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.752754:0:9417:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe5368735f81 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d154 expref: 13 pid: 9417 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619926.752766:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900e03480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589208320:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619926.752845:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb540 going to sleep for 11 seconds 00000100:00100000:0.0:1788619926.760930:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619926.761084:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571621760 00000100:00100000:2.0:1788619926.761094:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571621760 00010000:00010000:2.0:1788619926.761135:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe5368735f81 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d154 expref: 14 pid: 9417 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619926.761177:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 440 00000100:00100000:2.0:1788619926.761187:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571621760:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619926.761214:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619926.761221:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe5368735f81 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d154 expref: 14 pid: 9417 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.761237:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe5368735f81 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d154 expref: 14 pid: 9417 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.761270:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619926.761276:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71200/0x1cd7fe5368735fce lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.761287:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71200/0x1cd7fe5368735fce lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.761295:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71200/0x1cd7fe5368735fce lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.761313:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619926.761388:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571621760:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 201us (468us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619926.762094:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 440 00010000:00010000:3.0:1788619926.764125:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe5368735f81 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d154 expref: 13 pid: 9417 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619926.764148:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900e03480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589208320:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619926.764172:0:9417:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71200/0x1cd7fe5368735fce lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.764183:0:9417:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71200/0x1cd7fe5368735fce lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.764189:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71200/0x1cd7fe5368735fce lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788619926.765090:0:9417:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967773 00000004:00010000:3.0:1788619926.765143:0:9417:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 987136 -> 987136 00000004:00010000:3.0:1788619926.765147:0:9417:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619926 -> 1788619926 00000004:00010000:3.0:1788619926.765151:0:9417:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619926 -> 1788619926 00000004:00010000:3.0:1788619926.765153:0:9417:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619926 -> 1788619926 00000004:00010000:3.0:1788619926.765155:0:9417:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1800 -> 1800 00000004:00010000:3.0:1788619926.765164:0:9417:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000838276b7 reply state 0000000045e44689 transno 4294967774 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71200/0x1cd7fe5368735fce lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619926.765174:0:9417:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788619926.766413:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571621632:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 14158us (14349us total) trans 4294967774 rc 0/0 00000100:00100000:2.0:1788619926.766424:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1348 00000020:00080000:2.0:1788619926.766430:0:9417:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:2.0:1788619926.772217:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71200/0x1cd7fe5368735fce lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.772235:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71200/0x1cd7fe5368735fce lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.772243:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71200/0x1cd7fe5368735fce lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.772248:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c5f71200 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:2.0:1788619926.772253:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71200/0x1cd7fe5368735fce lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.772383:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71200/0x1cd7fe5368735fce lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.772695:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619926.772699:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1788619926.772904:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00010000:00010000:2.0:1788619926.773117:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71200/0x1cd7fe5368735fce lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.773128:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71200/0x1cd7fe5368735fce lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619926.773370:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571621888 00000100:00100000:3.0:1788619926.773379:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571621888 00000100:00100000:3.0:1788619926.773435:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1349 00000100:00100000:3.0:1788619926.773446:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:14908:x1875503571621888:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619926.774565:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619926.774590:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70400/0x1cd7fe5368735fd5 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d1a1 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.775193:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4400/0x1cd7fe5368735fab lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d17e expref: 12 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.775205:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70400/0x1cd7fe5368735fd5 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1a1 expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.775251:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4400/0x1cd7fe5368735fab lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d17e expref: 12 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.775267:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4400/0x1cd7fe5368735fab lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d17e expref: 12 pid: 5844 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619926.775283:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900e03480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589208448:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619926.775671:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb540 going to sleep for 11 seconds 00000100:00100000:2.0:1788619926.778958:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900e03480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589208448:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619926.778992:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70400/0x1cd7fe5368735fd5 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1a1 expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.779012:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619926.779014:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619926.779019:0:5846:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70400/0x1cd7fe5368735fd5 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1a1 expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.779029:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f70400, rc 0) 00000100:00100000:2.0:1788619926.779082:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571621888:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5637us (6181us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619926.779090:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1349 00000100:00100000:3.0:1788619926.779309:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619926.779951:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571622016 00000100:00100000:1.0:1788619926.779960:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571622016 00010000:00010000:1.0:1788619926.779994:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4400/0x1cd7fe5368735fab lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d17e expref: 13 pid: 5844 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619926.780023:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 441 00000100:00100000:1.0:1788619926.780032:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e00700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571622016:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619926.780045:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619926.780069:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4400/0x1cd7fe5368735fab lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d17e expref: 13 pid: 5844 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.780081:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4400/0x1cd7fe5368735fab lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d17e expref: 13 pid: 5844 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.780102:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4400/0x1cd7fe5368735fab lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d17e expref: 13 pid: 5844 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.780116:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619926.780121:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70400/0x1cd7fe5368735fd5 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1a1 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.780130:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70400/0x1cd7fe5368735fd5 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1a1 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.780138:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70400/0x1cd7fe5368735fd5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1a1 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.780172:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70400/0x1cd7fe5368735fd5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1a1 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619926.780186:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e6478a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589208576:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619926.780254:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78e03f5600 going to sleep for 11 seconds 00000100:00100000:1.0:1788619926.784607:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e6478a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589208576:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619926.784831:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619926.784985:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e00700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2355:x1875503571622016:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 4952us (5674us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619926.784996:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 441 00000100:00100000:0.0:1788619926.785824:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619926.786519:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571622144 00000100:00100000:2.0:1788619926.786531:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571622144 00000100:00100000:2.0:1788619926.786586:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1107 00000100:00100000:2.0:1788619926.786595:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:14908:x1875503571622144:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619926.786621:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619926.786640:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001544d1a8/0x1cd7fe5368735fdc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d1a8 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.786677:0:6696:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000002abacdb6/0x1cd7fe5368735f26 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d107 expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.786687:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(9 13) 00010000:00010000:2.0:1788619926.786691:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(10 13) 00010000:00010000:2.0:1788619926.786697:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001544d1a8/0x1cd7fe5368735fdc lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1a8 expref: 8 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.786728:0:6696:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000002abacdb6/0x1cd7fe5368735f26 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d107 expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.786742:0:6696:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000002abacdb6/0x1cd7fe5368735f26 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d107 expref: 6 pid: 6696 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619926.786756:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5024a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589208704:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619926.786859:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164840 going to sleep for 11 seconds 00000100:00100000:2.0:1788619926.792713:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5024a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589208704:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619926.792750:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001544d1a8/0x1cd7fe5368735fdc lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1a8 expref: 8 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.792770:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f72800, rc 0) 00000100:00100000:2.0:1788619926.792838:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+8:14908:x1875503571622144:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6242us (7014us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619926.792847:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1107 00000100:00100000:2.0:1788619926.800047:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619926.805124:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571622272 00000100:00100000:1.0:1788619926.805141:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571622272 00010000:00010000:1.0:1788619926.805191:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000002abacdb6/0x1cd7fe5368735f26 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d107 expref: 7 pid: 6696 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619926.805817:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 442 00000100:00100000:1.0:1788619926.805830:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5024a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2352:x1875503571622272:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619926.805846:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619926.805872:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000002abacdb6/0x1cd7fe5368735f26 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d107 expref: 7 pid: 6696 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.805905:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000002abacdb6/0x1cd7fe5368735f26 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d107 expref: 7 pid: 6696 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.805929:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002abacdb6/0x1cd7fe5368735f26 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d107 expref: 7 pid: 6696 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.805947:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(10 13) 00010000:00010000:1.0:1788619926.805951:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(10 13) 00010000:00010000:1.0:1788619926.805960:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000001544d1a8/0x1cd7fe5368735fdc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1a8 expref: 7 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.805971:0:13330:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001544d1a8/0x1cd7fe5368735fdc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1a8 expref: 7 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.806009:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000001544d1a8/0x1cd7fe5368735fdc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1a8 expref: 7 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619926.806023:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e6478a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589208832:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619926.806119:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78e03f5600 going to sleep for 11 seconds 00000100:00100000:1.0:1788619926.820595:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619926.820668:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619926.820794:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571622400 00000100:00100000:3.0:1788619926.820803:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571622400 00002000:00010000:3.0:1788619926.820846:0:5857:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:3.0:1788619926.820851:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e647a300 with x1875503571622400 00000100:00100000:3.0:1788619926.820889:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 818 00000100:00100000:3.0:1788619926.820900:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e647a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571622400:12345-192.168.201.5@tcp:10:fsx.0 00000004:00010000:3.0:1788619926.821210:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 987136 -> 428565 00000004:00010000:3.0:1788619926.821215:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619926 -> 1788619926 00000004:00010000:3.0:1788619926.821217:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619926 -> 1788619926 00000004:00010000:3.0:1788619926.821220:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619926 -> 1788619926 00000004:00010000:3.0:1788619926.821221:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1800 -> 1800 00000100:00100000:3.0:1788619926.821299:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e647a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571622400:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 400us (712us total) trans 4294967775 rc 0/0 00000100:00100000:3.0:1788619926.821305:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 818 00002000:00010000:3.0:1788619926.821309:0:5857:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:3.0:1788619926.821311:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e647a300 with x1875503571622400 00000100:00100000:2.0:1788619926.821581:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e6478a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589208832:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619926.821602:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619926.821644:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5024a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571622272:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 15817us (21606us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619926.821652:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 442 00000100:00100000:0.0:1788619926.821896:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571622528 00000100:00100000:0.0:1788619926.821902:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571622528 00002000:00010000:0.0:1788619926.821927:0:6701:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:0.0:1788619926.821932:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c74aa680 with x1875503571622528 ext(0->18446744073709551615) 00002000:00010000:0.0:1788619926.821939:0:6701:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c74aa680. 00000100:00100000:0.0:1788619926.821963:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 635 00000100:00100000:0.0:1788619926.821971:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74aa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+8:2355:x1875503571622528:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:0.0:1788619926.822178:0:6701:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1550849 -> 0 00002000:00010000:0.0:1788619926.822181:0:6701:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619926 -> 1788619926 00002000:00010000:0.0:1788619926.822183:0:6701:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619926 -> 1788619925 00002000:00010000:0.0:1788619926.822186:0:6701:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619926 -> 1788619926 00002000:00010000:0.0:1788619926.822188:0:6701:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 391 -> 391 00000100:00100000:0.0:1788619926.822227:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74aa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571622528:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 256us (1558us total) trans 4294967484 rc 0/0 00000100:00100000:0.0:1788619926.822234:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 635 00002000:00010000:0.0:1788619926.822237:0:6701:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:0.0:1788619926.822240:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c74aa680 with x1875503571622528 ext(0->18446744073709551615) 00002000:00010000:0.0:1788619926.822245:0:6701:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c74aa680. 00000100:00100000:1.0:1788619926.825657:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619926.825740:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571622656 00000100:00100000:3.0:1788619926.825750:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571622656 00000100:00100000:3.0:1788619926.825807:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1350 00000100:00100000:3.0:1788619926.825816:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6859500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:14908:x1875503571622656:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619926.825839:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619926.825858:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70e00/0x1cd7fe5368735fe3 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d1af expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.825940:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71e00/0x1cd7fe5368735fea lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.825956:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71e00/0x1cd7fe5368735fea lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.825967:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71e00/0x1cd7fe5368735fea lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788619926.825995:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 428565 00000004:00200000:3.0:1788619926.826000:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788619926.826018:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71e00/0x1cd7fe5368735fea lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.826046:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70e00/0x1cd7fe5368735fe3 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1af expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.826064:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71e00/0x1cd7fe5368735fea lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1af expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.826073:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619926.826075:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788619926.826079:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f71e00, rc 0) 00000100:00100000:3.0:1788619926.826138:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6859500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+13:14908:x1875503571622656:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 323us (489us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619926.826146:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1350 00000100:00100000:2.0:1788619926.829650:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619926.829783:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571622784 00000100:00100000:1.0:1788619926.829793:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571622784 00000100:00100000:1.0:1788619926.829844:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1351 00000100:00100000:1.0:1788619926.829853:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5027100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2355:x1875503571622784:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619926.829871:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619926.829892:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296200/0x1cd7fe5368735ff1 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d1b6 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.829927:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70400/0x1cd7fe5368735fd5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1a1 expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619926.829967:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b17b80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589208960:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619926.830046:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78e03f53c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619926.835574:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619926.835746:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571622912 00000100:00100000:1.0:1788619926.835756:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571622912 00000100:00100000:1.0:1788619926.835824:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1108 00000100:00100000:1.0:1788619926.835834:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5025880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571622912:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619926.835851:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619926.835869:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd485a32/0x1cd7fe5368735ff8 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d1bd expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619926.835933:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b17800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589209088:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619926.836017:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78e03f5600 going to sleep for 11 seconds 00002000:00010000:3.0:1788619926.841849:0:6698:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619925 -> 1788619926 00000100:00100000:3.0:1788619926.841888:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b17800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589209088:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619926.841925:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000cd485a32/0x1cd7fe5368735ff8 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1bd expref: 7 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.841958:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd485a32/0x1cd7fe5368735ff8 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1bd expref: 7 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.841974:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296600, rc 0) 00000100:00100000:3.0:1788619926.842047:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5025880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571622912:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6214us (6483us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619926.842057:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1108 00000100:00100000:2.0:1788619926.849057:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b17b80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589208960:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619926.849093:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 428565 00010000:00010000:2.0:1788619926.849115:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296200/0x1cd7fe5368735ff1 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1b6 expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.849139:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619926.849143:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619926.849148:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296200/0x1cd7fe5368735ff1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1b6 expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.849164:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296200, rc 0) 00000100:00100000:2.0:1788619926.849250:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5027100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2355:x1875503571622784:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 19397us (19608us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619926.849268:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1351 00000100:00100000:1.0:1788619926.856694:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619926.856789:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571623040 00000100:00100000:2.0:1788619926.856799:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571623040 00000100:00100000:2.0:1788619926.856857:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1352 00000100:00100000:2.0:1788619926.856867:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b15c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:14908:x1875503571623040:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619926.856888:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619926.856906:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72000/0x1cd7fe5368735fff lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d1c4 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.856936:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70400/0x1cd7fe5368735fd5 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1a1 expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.856946:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72000/0x1cd7fe5368735fff lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1c4 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.856977:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70400/0x1cd7fe5368735fd5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1a1 expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.856989:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70400/0x1cd7fe5368735fd5 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1a1 expref: 12 pid: 5846 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619926.857002:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5027480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589209216:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619926.857083:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164840 going to sleep for 11 seconds 00000100:00100000:0.0:1788619926.866515:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619926.867126:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571623168 00000100:00100000:3.0:1788619926.867136:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571623168 00010000:00010000:3.0:1788619926.867178:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70400/0x1cd7fe5368735fd5 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1a1 expref: 13 pid: 5846 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619926.867232:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 443 00000100:00100000:3.0:1788619926.867241:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff6300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+13:2355:x1875503571623168:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619926.867254:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619926.867288:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70400/0x1cd7fe5368735fd5 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1a1 expref: 13 pid: 5846 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.867302:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70400/0x1cd7fe5368735fd5 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1a1 expref: 13 pid: 5846 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.867368:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619926.867376:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72000/0x1cd7fe5368735fff lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1c4 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.867394:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72000/0x1cd7fe5368735fff lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1c4 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.867403:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72000/0x1cd7fe5368735fff lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1c4 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.867445:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72000/0x1cd7fe5368735fff lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1c4 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619926.867458:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900e01180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589209344:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619926.867536:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb9c0 going to sleep for 11 seconds 00010000:00010000:2.0:1788619926.867960:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70400/0x1cd7fe5368735fd5 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1a1 expref: 13 pid: 5846 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619926.867979:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5027480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589209216:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619926.868003:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72000/0x1cd7fe5368735fff lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1c4 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.868014:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619926.868016:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619926.868020:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f72000, rc 0) 00000100:00100000:2.0:1788619926.868063:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b15c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571623040:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 11197us (11380us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619926.868071:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1352 00000100:00100000:3.0:1788619926.878703:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900e01180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589209344:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619926.878729:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619926.878827:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff6300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571623168:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 11586us (12321us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619926.878838:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 443 00000100:00100000:0.0:1788619926.887757:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619926.887883:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571623296 00000100:00100000:3.0:1788619926.887893:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571623296 00002000:00100000:3.0:1788619926.887941:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5ff5500 x1875503571623296/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:22/0 lens 488/0 e 0 to 0 dl 1788619937 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619926.887953:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5ff5500 with x1875503571623296 00000100:00100000:3.0:1788619926.888038:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 819 00000100:00100000:3.0:1788619926.888047:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571623296:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619926.906763:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571623296:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 18705us (19003us total) trans 0 rc 65536/65536 00000100:00100000:1.0:1788619926.906778:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 819 00002000:00100000:1.0:1788619926.906793:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5ff5500 x1875503571623296/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:22/0 lens 488/440 e 0 to 0 dl 1788619937 ref 1 fl Complete:/600/0 rc 65536/65536 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619926.906805:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5ff5500 with x1875503571623296 00000100:00100000:3.0:1788619926.910024:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619926.910133:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571623424 00000100:00100000:1.0:1788619926.910143:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571623424 00002000:00100000:1.0:1788619926.910191:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900e01180 x1875503571623424/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:22/0 lens 488/0 e 0 to 0 dl 1788619937 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619926.910268:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900e01180 with x1875503571623424 00000100:00100000:1.0:1788619926.910310:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 820 00000100:00100000:1.0:1788619926.910320:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e01180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571623424:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619926.918969:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e01180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571623424:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 8640us (8945us total) trans 0 rc 36864/36864 00000100:00100000:1.0:1788619926.918985:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 820 00002000:00100000:1.0:1788619926.918997:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900e01180 x1875503571623424/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:22/0 lens 488/440 e 0 to 0 dl 1788619937 ref 1 fl Complete:/600/0 rc 36864/36864 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619926.919007:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900e01180 with x1875503571623424 00000100:00100000:1.0:1788619926.922712:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619926.923056:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571623552 00000100:00100000:3.0:1788619926.923066:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571623552 00000100:00100000:3.0:1788619926.923120:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1353 00000100:00100000:3.0:1788619926.923129:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b14380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571623552:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619926.923147:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619926.923166:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71c00/0x1cd7fe5368736006 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d1cb expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.923253:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70e00/0x1cd7fe536873600d lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.923278:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70e00/0x1cd7fe536873600d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.923289:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70e00/0x1cd7fe536873600d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788619926.923314:0:11397:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 428565 00000004:00200000:3.0:1788619926.923318:0:11397:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788619926.923374:0:11397:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70e00/0x1cd7fe536873600d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.923416:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71c00/0x1cd7fe5368736006 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1cb expref: 15 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.923435:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70e00/0x1cd7fe536873600d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1cb expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619926.923443:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619926.923446:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788619926.923623:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f70e00, rc 0) 00000100:00100000:2.0:1788619926.924808:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b14380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571623552:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 1671us (2199us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619926.924818:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1353 00000100:00100000:0.0:1788619926.927792:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619926.927853:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571623680 00000100:00100000:3.0:1788619926.927860:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571623680 00000100:00100000:3.0:1788619926.927913:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1109 00000100:00100000:3.0:1788619926.927923:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571623680:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619926.927940:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619926.927957:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d234829/0x1cd7fe5368736014 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d1d2 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619926.928009:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900e03480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589209472:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619926.928077:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619926.934286:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900e03480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589209472:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619926.934458:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009d234829/0x1cd7fe5368736014 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1d2 expref: 7 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.934487:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d234829/0x1cd7fe5368736014 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1d2 expref: 7 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.934502:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f70800, rc 0) 00000100:00100000:0.0:1788619926.934569:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571623680:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6647us (6783us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619926.934582:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1109 00000100:00100000:1.0:1788619926.940790:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619926.940874:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571623808 00000100:00100000:0.0:1788619926.940884:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571623808 00000100:00100000:0.0:1788619926.940949:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1110 00000100:00100000:0.0:1788619926.940959:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b16d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571623808:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619926.940978:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619926.940994:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096deb5d6/0x1cd7fe536873601b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d1d9 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619926.941061:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589209600:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619926.941154:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590540 going to sleep for 11 seconds 00000100:00100000:0.0:1788619926.944558:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589209600:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619926.944601:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000096deb5d6/0x1cd7fe536873601b lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1d9 expref: 7 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.944644:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096deb5d6/0x1cd7fe536873601b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1d9 expref: 7 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619926.944661:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d5c00, rc 0) 00000100:00100000:0.0:1788619926.944725:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b16d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571623808:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3767us (3944us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619926.944734:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1110 00000100:00100000:2.0:1788619926.948253:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619926.948362:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571623936 00000100:00100000:0.0:1788619926.948371:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571623936 00000100:00100000:2.0:1788619926.951382:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1111 00000100:00100000:2.0:1788619926.951408:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5027480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571623936:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619926.951428:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619926.951448:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002cc077c3/0x1cd7fe5368736022 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d1e0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619926.951513:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5026300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589209728:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619926.951624:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164840 going to sleep for 11 seconds 00000100:00100000:2.0:1788619926.962547:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5026300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589209728:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619926.962590:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002cc077c3/0x1cd7fe5368736022 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1e0 expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.962632:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002cc077c3/0x1cd7fe5368736022 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1e0 expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619926.962646:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f72e00, rc 0) 00000100:00100000:2.0:1788619926.962716:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5027480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571623936:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 11320us (14471us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619926.962724:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1111 00000100:00100000:1.0:1788619926.985071:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619926.985536:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571624064 00000100:00100000:3.0:1788619926.985547:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571624064 00000100:00100000:3.0:1788619926.985626:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1112 00000100:00100000:3.0:1788619926.985638:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b14a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571624064:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619926.985657:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619926.985673:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afc8360d/0x1cd7fe5368736029 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d1e7 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619926.985727:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900e03800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589209856:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619926.985811:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb840 going to sleep for 11 seconds 00000100:00100000:1.0:1788619926.993712:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900e03800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589209856:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619926.993758:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000afc8360d/0x1cd7fe5368736029 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1e7 expref: 7 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.993786:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afc8360d/0x1cd7fe5368736029 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1e7 expref: 7 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619926.993806:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f71c00, rc 0) 00000100:00100000:1.0:1788619926.993890:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b14a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571624064:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8253us (8828us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619926.993899:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1112 00000100:00100000:1.0:1788619926.999067:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619927.001145:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571624320 00000100:00100000:2.0:1788619927.001159:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571624320 00000100:00100000:2.0:1788619927.001239:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1354 00000100:00100000:2.0:1788619927.001249:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b17b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+12:14908:x1875503571624320:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:2.0:1788619927.001303:0:9417:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619927.001365:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71e00/0x1cd7fe5368735fea lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1af expref: 12 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.001385:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619927.001390:0:9417:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788619927.001799:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72e00/0x1cd7fe5368736030 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.001815:0:9417:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70e00/0x1cd7fe536873600d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1cb expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.001824:0:9417:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72e00/0x1cd7fe5368736030 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.001854:0:9417:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70e00/0x1cd7fe536873600d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1cb expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.001868:0:9417:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70e00/0x1cd7fe536873600d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1cb expref: 13 pid: 11397 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619927.002713:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5026300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589209984:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619927.002816:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164840 going to sleep for 11 seconds 00000100:00100000:2.0:1788619927.009884:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619927.010471:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571624448 00000100:00100000:0.0:1788619927.010481:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571624448 00010000:00010000:0.0:1788619927.010533:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70e00/0x1cd7fe536873600d lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1cb expref: 14 pid: 11397 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619927.010578:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 444 00000100:00100000:0.0:1788619927.010589:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5024000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571624448:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619927.010622:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619927.010631:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70e00/0x1cd7fe536873600d lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1cb expref: 14 pid: 11397 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.010645:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70e00/0x1cd7fe536873600d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1cb expref: 14 pid: 11397 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.010669:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619927.010674:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72e00/0x1cd7fe5368736030 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.010684:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72e00/0x1cd7fe5368736030 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.010693:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72e00/0x1cd7fe5368736030 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.010713:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619927.010774:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5024000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571624448:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 187us (908us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619927.010782:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 444 00010000:00010000:1.0:1788619927.010832:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70e00/0x1cd7fe536873600d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1cb expref: 13 pid: 11397 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619927.010850:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5026300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589209984:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619927.010873:0:9417:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72e00/0x1cd7fe5368736030 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.010892:0:9417:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72e00/0x1cd7fe5368736030 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.010896:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72e00/0x1cd7fe5368736030 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788619927.011024:0:9417:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967776 00000004:00010000:1.0:1788619927.011074:0:9417:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 428565 -> 428565 00000004:00010000:1.0:1788619927.011077:0:9417:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619926 -> 1788619926 00000004:00010000:1.0:1788619927.011079:0:9417:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619926 -> 1788619926 00000004:00010000:1.0:1788619927.011081:0:9417:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619926 -> 1788619926 00000004:00010000:1.0:1788619927.011083:0:9417:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1800 -> 1800 00000004:00010000:1.0:1788619927.011090:0:9417:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000fa9418c3 reply state 00000000d5f50a7d transno 4294967777 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72e00/0x1cd7fe5368736030 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619927.011104:0:9417:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788619927.011172:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b17b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+14:14908:x1875503571624320:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 9924us (12114us total) trans 4294967777 rc 0/0 00000100:00100000:1.0:1788619927.011180:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1354 00000020:00080000:1.0:1788619927.011184:0:9417:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1788619927.016412:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00010000:00010000:0.0:1788619927.017872:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72e00/0x1cd7fe5368736030 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.017900:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72e00/0x1cd7fe5368736030 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.017908:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72e00/0x1cd7fe5368736030 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.017914:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c5f72e00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788619927.017920:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72e00/0x1cd7fe5368736030 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.017928:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72e00/0x1cd7fe5368736030 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.017946:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619927.017949:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788619927.017955:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72e00/0x1cd7fe5368736030 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.017962:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72e00/0x1cd7fe5368736030 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619927.019425:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571624576 00000100:00100000:2.0:1788619927.019441:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571624576 00000100:00100000:2.0:1788619927.019522:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1355 00000100:00100000:2.0:1788619927.019533:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+13:14908:x1875503571624576:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619927.019558:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619927.019579:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72a00/0x1cd7fe5368736037 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d1ee expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.019634:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72000/0x1cd7fe5368735fff lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1c4 expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.019646:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72a00/0x1cd7fe5368736037 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1ee expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.019683:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72000/0x1cd7fe5368735fff lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1c4 expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.019698:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72000/0x1cd7fe5368735fff lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1c4 expref: 12 pid: 5846 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619927.019713:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5024380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589210112:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619927.019807:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81643c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619927.027497:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5024380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589210112:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619927.027537:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72a00/0x1cd7fe5368736037 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1ee expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.027558:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619927.027561:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619927.027566:0:5846:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72a00/0x1cd7fe5368736037 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1ee expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.027577:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f72a00, rc 0) 00000100:00100000:2.0:1788619927.027741:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+14:14908:x1875503571624576:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8209us (11351us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619927.027750:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1355 00000100:00100000:3.0:1788619927.038243:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619927.039788:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571624704 00000100:00100000:1.0:1788619927.039802:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571624704 00010000:00010000:1.0:1788619927.039852:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72000/0x1cd7fe5368735fff lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1c4 expref: 13 pid: 5846 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619927.039901:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 445 00000100:00100000:1.0:1788619927.039912:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcacc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571624704:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619927.039933:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619927.039962:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72000/0x1cd7fe5368735fff lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1c4 expref: 13 pid: 5846 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.039976:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72000/0x1cd7fe5368735fff lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1c4 expref: 13 pid: 5846 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.039999:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72000/0x1cd7fe5368735fff lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1c4 expref: 13 pid: 5846 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.040018:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619927.040024:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72a00/0x1cd7fe5368736037 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1ee expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.040041:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72a00/0x1cd7fe5368736037 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1ee expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.040051:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72a00/0x1cd7fe5368736037 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1ee expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.040088:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72a00/0x1cd7fe5368736037 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1ee expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619927.040102:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b17800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589210240:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619927.040194:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78e03f53c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619927.043780:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619927.043857:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619927.043957:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b17800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589210240:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619927.043982:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619927.044044:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcacc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2355:x1875503571624704:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 4133us (5812us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619927.044055:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 445 00000100:00100000:3.0:1788619927.044312:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571624960 00000100:00100000:3.0:1788619927.044320:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571624960 00002000:00010000:3.0:1788619927.044394:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:3.0:1788619927.044399:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fd93b480 with x1875503571624960 ext(0->18446744073709551615) 00002000:00010000:3.0:1788619927.044410:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fd93b480. 00000100:00100000:3.0:1788619927.044432:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 636 00000100:00100000:3.0:1788619927.044441:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd93b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571624960:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:3.0:1788619927.044682:0:6702:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:3.0:1788619927.044686:0:6702:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619926 -> 1788619926 00002000:00010000:3.0:1788619927.044689:0:6702:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619926 -> 1788619925 00002000:00010000:3.0:1788619927.044691:0:6702:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619926 -> 1788619926 00002000:00010000:3.0:1788619927.044693:0:6702:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 391 -> 391 00000100:00100000:3.0:1788619927.044729:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd93b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571624960:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 289us (959us total) trans 4294967485 rc 0/0 00000100:00100000:3.0:1788619927.044735:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 636 00002000:00010000:3.0:1788619927.044738:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:3.0:1788619927.044741:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fd93b480 with x1875503571624960 ext(0->18446744073709551615) 00002000:00010000:3.0:1788619927.044745:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fd93b480. 00000100:00100000:2.0:1788619927.045015:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571624832 00000100:00100000:2.0:1788619927.045021:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571624832 00002000:00010000:2.0:1788619927.045042:0:5859:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:2.0:1788619927.045046:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fd93a300 with x1875503571624832 00000100:00100000:2.0:1788619927.045076:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 821 00000100:00100000:2.0:1788619927.045083:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd93a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+14:2355:x1875503571624832:12345-192.168.201.5@tcp:10:fsx.0 00000004:00010000:2.0:1788619927.045198:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 428565 -> 541014 00000004:00010000:2.0:1788619927.045201:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619926 -> 1788619926 00000004:00010000:2.0:1788619927.045204:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619926 -> 1788619926 00000004:00010000:2.0:1788619927.045206:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619926 -> 1788619926 00000004:00010000:2.0:1788619927.045209:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1800 -> 1800 00000100:00100000:2.0:1788619927.045252:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd93a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+15:2355:x1875503571624832:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 168us (1393us total) trans 4294967778 rc 0/0 00000100:00100000:2.0:1788619927.045259:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 821 00002000:00010000:2.0:1788619927.045262:0:5859:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:2.0:1788619927.045264:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fd93a300 with x1875503571624832 00000100:00100000:1.0:1788619927.050661:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619927.050825:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571625088 00000100:00100000:0.0:1788619927.050836:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571625088 00000100:00100000:0.0:1788619927.050896:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1356 00000100:00100000:0.0:1788619927.050905:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd93b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+12:14908:x1875503571625088:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619927.050925:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619927.050947:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5e00/0x1cd7fe536873603e lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d1f5 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.051044:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe5368736045 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.051060:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe5368736045 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.051071:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe5368736045 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619927.051101:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 541014 00000004:00200000:0.0:1788619927.051105:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619927.051123:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe5368736045 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.051147:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5e00/0x1cd7fe536873603e lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1f5 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.051167:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe5368736045 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1f5 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.051176:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619927.051178:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619927.051182:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d4c00, rc 0) 00000100:00100000:0.0:1788619927.051248:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd93b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571625088:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 344us (598us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619927.051255:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1356 00000100:00100000:1.0:1788619927.065097:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619927.065201:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571625216 00000100:00100000:2.0:1788619927.065211:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571625216 00000100:00100000:2.0:1788619927.065272:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1357 00000100:00100000:2.0:1788619927.065283:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd93bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571625216:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619927.065302:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619927.065322:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72c00/0x1cd7fe536873604c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d1fc expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.065405:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72a00/0x1cd7fe5368736037 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1ee expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619927.065453:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5024380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589210368:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619927.065536:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81643c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619927.073318:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619927.073478:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571625344 00000100:00100000:3.0:1788619927.073488:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571625344 00000100:00100000:3.0:1788619927.073544:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1113 00000100:00100000:3.0:1788619927.073554:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd938a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571625344:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619927.073573:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619927.073591:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7d29fbb/0x1cd7fe5368736053 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d203 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619927.073668:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900e00700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589210496:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619927.073746:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebb40 going to sleep for 11 seconds 00000100:00100000:2.0:1788619927.074699:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5024380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589210368:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619927.074723:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 541014 00010000:00010000:2.0:1788619927.074739:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72c00/0x1cd7fe536873604c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1fc expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.074759:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619927.074762:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619927.074767:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72c00/0x1cd7fe536873604c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1fc expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.074780:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f72c00, rc 0) 00000100:00100000:2.0:1788619927.074832:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd93bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571625216:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 9552us (9744us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619927.074840:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1357 00002000:00010000:3.0:1788619927.079855:0:7654:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619925 -> 1788619926 00000100:00100000:3.0:1788619927.079898:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900e00700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589210496:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619927.079937:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c7d29fbb/0x1cd7fe5368736053 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d203 expref: 7 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619927.079961:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7d29fbb/0x1cd7fe5368736053 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d203 expref: 7 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619927.079975:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f71400, rc 0) 00000100:00100000:3.0:1788619927.080032:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd938a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571625344:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6479us (6726us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619927.080041:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1113 00000100:00100000:3.0:1788619927.083181:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619927.083289:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571625472 00000100:00100000:2.0:1788619927.083298:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571625472 00000100:00100000:2.0:1788619927.083378:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1358 00000100:00100000:2.0:1788619927.083389:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e00700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571625472:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619927.083410:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619927.083430:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73000/0x1cd7fe536873605a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d20a expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.083895:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72a00/0x1cd7fe5368736037 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1ee expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619927.083939:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5024380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589210624:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619927.084061:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81643c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619927.093457:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619927.093875:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571625600 00000100:00100000:3.0:1788619927.093886:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571625600 00000100:00100000:3.0:1788619927.093949:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1114 00000100:00100000:3.0:1788619927.093960:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571625600:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619927.093978:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619927.093994:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7d29fbb/0x1cd7fe5368736061 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d211 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619927.094060:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900e03800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589210752:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619927.094142:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebb40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619927.098624:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5024380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589210624:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619927.098653:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 541014 00010000:00010000:0.0:1788619927.098673:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73000/0x1cd7fe536873605a lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d20a expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.098696:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619927.098700:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619927.098706:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73000/0x1cd7fe536873605a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d20a expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.098721:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f73000, rc 0) 00000100:00100000:0.0:1788619927.098787:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e00700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571625472:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 15400us (15615us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619927.098796:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1358 00000100:00100000:3.0:1788619927.099453:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900e03800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589210752:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619927.099477:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c7d29fbb/0x1cd7fe5368736061 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d211 expref: 7 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619927.099498:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7d29fbb/0x1cd7fe5368736061 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d211 expref: 7 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619927.099508:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f71400, rc 0) 00000100:00100000:3.0:1788619927.099537:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571625600:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5579us (6098us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619927.099544:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1114 00000100:00100000:1.0:1788619927.102228:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619927.102274:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619927.102371:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571625728 00000100:00100000:3.0:1788619927.102381:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571625728 00000100:00100000:3.0:1788619927.102434:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1359 00000100:00100000:3.0:1788619927.102452:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571625728:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619927.102473:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619927.102493:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70000/0x1cd7fe5368736068 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d218 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619927.102534:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72a00/0x1cd7fe5368736037 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1ee expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619927.102574:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900e03800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589210880:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619927.102815:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebb40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619927.102898:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571625856 00000100:00100000:0.0:1788619927.102903:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571625856 00000100:00100000:0.0:1788619927.102941:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1115 00000100:00100000:0.0:1788619927.102949:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571625856:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619927.102957:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619927.102968:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000df72df5a/0x1cd7fe536873606f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d21f expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619927.103004:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be7100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589211008:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619927.103035:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590900 going to sleep for 11 seconds 00000100:00100000:2.0:1788619927.105367:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be7100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589211008:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619927.105407:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000df72df5a/0x1cd7fe536873606f lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d21f expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.105433:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000df72df5a/0x1cd7fe536873606f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d21f expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.105516:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d5200, rc 0) 00000100:00100000:3.0:1788619927.105696:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900e03800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589210880:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619927.105710:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 541014 00010000:00010000:3.0:1788619927.105722:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70000/0x1cd7fe5368736068 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d218 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619927.105739:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619927.105741:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619927.105746:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70000/0x1cd7fe5368736068 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d218 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619927.105756:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f70000, rc 0) 00000100:00100000:2.0:1788619927.107738:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571625856:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4776us (5505us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619927.107751:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1115 00000100:00100000:3.0:1788619927.107846:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571625728:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5401us (5568us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619927.107853:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1359 00000100:00100000:2.0:1788619927.112633:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619927.112724:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571625984 00000100:00100000:1.0:1788619927.112734:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571625984 00000100:00100000:1.0:1788619927.112797:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1360 00000100:00100000:1.0:1788619927.112808:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5024700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571625984:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619927.112829:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619927.112853:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe5368736076 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d226 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.112884:0:9417:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72a00/0x1cd7fe5368736037 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1ee expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.112895:0:9417:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe5368736076 lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d226 expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.113004:0:9417:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72a00/0x1cd7fe5368736037 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1ee expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.113017:0:9417:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72a00/0x1cd7fe5368736037 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1ee expref: 14 pid: 5846 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619927.113030:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c75f5500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589211136:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619927.113109:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78e03f53c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619927.118063:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c75f5500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589211136:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619927.118100:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe5368736076 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d226 expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.118119:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619927.118123:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619927.118128:0:9417:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe5368736076 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d226 expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.118138:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297a00, rc 0) 00000100:00100000:3.0:1788619927.118191:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619927.118251:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571626112 00000100:00100000:1.0:1788619927.118259:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571626112 00010000:00010000:1.0:1788619927.118294:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72a00/0x1cd7fe5368736037 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1ee expref: 15 pid: 5846 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619927.118351:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 446 00000100:00100000:1.0:1788619927.118361:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e03800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+15:2352:x1875503571626112:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619927.118375:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619927.118410:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72a00/0x1cd7fe5368736037 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1ee expref: 15 pid: 5846 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.118428:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72a00/0x1cd7fe5368736037 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1ee expref: 15 pid: 5846 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.118460:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72a00/0x1cd7fe5368736037 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1ee expref: 15 pid: 5846 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.118473:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619927.118478:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe5368736076 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d226 expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.118486:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe5368736076 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d226 expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.118495:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe5368736076 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d226 expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.118527:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe5368736076 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d226 expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619927.118539:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c75f6a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589211264:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619927.118852:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5024700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571625984:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6045us (6229us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619927.118860:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1360 00000100:00100000:3.0:1788619927.129592:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619927.130618:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571626240 00000100:00100000:0.0:1788619927.130631:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571626240 00002000:00100000:0.0:1788619927.130687:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900e03480 x1875503571626240/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:23/0 lens 488/0 e 0 to 0 dl 1788619938 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619927.130700:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900e03480 with x1875503571626240 00000100:00100000:0.0:1788619927.130731:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 822 00000100:00100000:0.0:1788619927.130742:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e03480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571626240:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619927.133521:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78e03f53c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619927.133570:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c75f6a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589211264:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619927.133591:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619927.133669:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e03800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+14:2352:x1875503571626112:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 15309us (15478us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619927.133684:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 446 00000100:00100000:1.0:1788619927.137756:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e03480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571626240:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 7004us (8161us total) trans 0 rc 65536/65536 00000100:00100000:1.0:1788619927.137770:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 822 00002000:00100000:1.0:1788619927.137783:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900e03480 x1875503571626240/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:23/0 lens 488/440 e 0 to 0 dl 1788619938 ref 1 fl Complete:/600/0 rc 65536/65536 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619927.137793:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900e03480 with x1875503571626240 00000100:00100000:3.0:1788619927.141923:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619927.142024:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571626368 00000100:00100000:2.0:1788619927.142033:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571626368 00002000:00100000:2.0:1788619927.142081:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fefd4e00 x1875503571626368/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:23/0 lens 488/0 e 0 to 0 dl 1788619938 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619927.142093:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fefd4e00 with x1875503571626368 00000100:00100000:2.0:1788619927.142175:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 823 00000100:00100000:2.0:1788619927.142185:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571626368:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619927.148286:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571626368:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 6089us (6361us total) trans 0 rc 36864/36864 00000100:00100000:3.0:1788619927.148301:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 823 00002000:00100000:3.0:1788619927.148314:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fefd4e00 x1875503571626368/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:23/0 lens 488/440 e 0 to 0 dl 1788619938 ref 1 fl Complete:/600/0 rc 36864/36864 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619927.148370:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fefd4e00 with x1875503571626368 00000100:00100000:3.0:1788619927.156383:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619927.156805:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571626496 00000100:00100000:0.0:1788619927.156815:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571626496 00000100:00100000:0.0:1788619927.156890:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1361 00000100:00100000:0.0:1788619927.156900:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+14:14908:x1875503571626496:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619927.156923:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619927.156944:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4600/0x1cd7fe536873607d lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d22d expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.157032:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5000/0x1cd7fe5368736084 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.157048:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5000/0x1cd7fe5368736084 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.157059:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5000/0x1cd7fe5368736084 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619927.157086:0:5846:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 541014 00000004:00200000:0.0:1788619927.157091:0:5846:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619927.157111:0:5846:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5000/0x1cd7fe5368736084 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.157138:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4600/0x1cd7fe536873607d lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d22d expref: 16 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.157157:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5000/0x1cd7fe5368736084 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d22d expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.157167:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619927.157170:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619927.157174:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d5000, rc 0) 00000100:00100000:0.0:1788619927.157244:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+15:14908:x1875503571626496:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 345us (932us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619927.157251:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1361 00000100:00100000:0.0:1788619927.160317:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619927.160830:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571626624 00000100:00100000:3.0:1788619927.160840:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571626624 00000100:00100000:3.0:1788619927.161000:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1362 00000100:00100000:3.0:1788619927.161012:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c3257850 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+15:2353:x1875503571626624:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619927.161036:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619927.161057:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71600/0x1cd7fe536873608b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d234 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619927.161093:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71600/0x1cd7fe536873608b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d234 expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788619927.161104:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 541014 00010000:00010000:3.0:1788619927.161113:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71600/0x1cd7fe536873608b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d234 expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619927.161127:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619927.161131:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619927.161135:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f71600, rc 0) 00000100:00100000:3.0:1788619927.161200:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c3257850 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+16:2353:x1875503571626624:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 190us (894us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619927.161207:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1362 00000100:00100000:2.0:1788619927.165885:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619927.165986:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571626880 00000100:00100000:1.0:1788619927.165997:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571626880 00000100:00100000:1.0:1788619927.166057:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1363 00000100:00100000:1.0:1788619927.166067:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5027b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571626880:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:1.0:1788619927.166124:0:9417:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619927.166152:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe5368736045 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1f5 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.166175:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619927.166180:0:9417:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788619927.167873:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe5368736092 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.167899:0:9417:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5000/0x1cd7fe5368736084 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d22d expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.167910:0:9417:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe5368736092 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.167953:0:9417:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5000/0x1cd7fe5368736084 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d22d expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.167968:0:9417:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5000/0x1cd7fe5368736084 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d22d expref: 15 pid: 5846 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619927.167983:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c75f6a00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589211392:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619927.168082:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78e03f53c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619927.171954:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619927.172133:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571627008 00000100:00100000:3.0:1788619927.172143:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571627008 00010000:00010000:3.0:1788619927.172184:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5000/0x1cd7fe5368736084 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d22d expref: 16 pid: 5846 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619927.172242:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 447 00000100:00100000:3.0:1788619927.172253:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571627008:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619927.172269:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619927.172277:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5000/0x1cd7fe5368736084 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d22d expref: 16 pid: 5846 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619927.172291:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5000/0x1cd7fe5368736084 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d22d expref: 16 pid: 5846 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619927.172318:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619927.172322:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe5368736092 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619927.172366:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe5368736092 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x50290000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619927.172374:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe5368736092 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x50290000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619927.172598:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619927.172736:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571627008:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 484us (792us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619927.172744:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 447 00010000:00010000:2.0:1788619927.172928:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5000/0x1cd7fe5368736084 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d22d expref: 15 pid: 5846 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619927.172947:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c75f6a00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589211392:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619927.172970:0:9417:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe5368736092 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.172979:0:9417:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe5368736092 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.172985:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe5368736092 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788619927.173115:0:9417:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967779 00000004:00010000:2.0:1788619927.173164:0:9417:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 541014 -> 541014 00000004:00010000:2.0:1788619927.173168:0:9417:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619926 -> 1788619927 00000004:00010000:2.0:1788619927.173170:0:9417:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619926 -> 1788619926 00000004:00010000:2.0:1788619927.173173:0:9417:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619926 -> 1788619927 00000004:00010000:2.0:1788619927.173175:0:9417:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1800 -> 1800 00000004:00010000:2.0:1788619927.173182:0:9417:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000314ce95a reply state 000000001d0d1955 transno 4294967780 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe5368736092 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619927.173204:0:9417:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788619927.173270:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5027b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:14908:x1875503571626880:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 7205us (7405us total) trans 4294967780 rc 0/0 00000100:00100000:2.0:1788619927.173278:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1363 00000020:00080000:2.0:1788619927.173281:0:9417:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788619927.178130:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe5368736092 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.178147:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe5368736092 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.178156:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe5368736092 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.178162:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78e0297e00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788619927.178167:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe5368736092 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.178175:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe5368736092 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.178200:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619927.178203:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788619927.178208:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe5368736092 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.178215:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297e00/0x1cd7fe5368736092 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619927.178265:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619927.178315:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571627136 00000100:00100000:0.0:1788619927.178347:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571627136 00000100:00100000:0.0:1788619927.178401:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1364 00000100:00100000:0.0:1788619927.178411:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5027800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:14908:x1875503571627136:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619927.178440:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619927.178443:0:5846:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:0.0:1788619927.178477:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297a00/0x1cd7fe5368736076 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d226 expref: 15 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.178487:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619927.178490:0:5846:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788619927.178503:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4a00/0x1cd7fe5368736099 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d23b expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.178525:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71600/0x1cd7fe536873608b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d234 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.178535:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4a00/0x1cd7fe5368736099 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d23b expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.178564:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71600/0x1cd7fe536873608b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d234 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.178583:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71600/0x1cd7fe536873608b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d234 expref: 14 pid: 5844 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619927.178596:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be4a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589211520:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619927.178680:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590a80 going to sleep for 11 seconds 00000100:00100000:2.0:1788619927.181772:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619927.181804:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571627264 00000100:00100000:2.0:1788619927.181814:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571627264 00010000:00010000:2.0:1788619927.181856:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71600/0x1cd7fe536873608b lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d234 expref: 15 pid: 5844 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619927.181897:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 448 00000100:00100000:2.0:1788619927.181906:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5025180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+15:2355:x1875503571627264:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619927.181922:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619927.181946:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71600/0x1cd7fe536873608b lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d234 expref: 15 pid: 5844 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.181966:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71600/0x1cd7fe536873608b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d234 expref: 15 pid: 5844 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.181987:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619927.181994:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4a00/0x1cd7fe5368736099 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d23b expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.182006:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4a00/0x1cd7fe5368736099 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d23b expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.182017:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4a00/0x1cd7fe5368736099 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d23b expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.182060:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4a00/0x1cd7fe5368736099 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d23b expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619927.182075:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5025f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589211648:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619927.182154:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164780 going to sleep for 11 seconds 00010000:00010000:1.0:1788619927.182710:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71600/0x1cd7fe536873608b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d234 expref: 15 pid: 5844 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619927.182728:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be4a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589211520:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619927.182752:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4a00/0x1cd7fe5368736099 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d23b expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.182764:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619927.182767:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619927.182771:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d4a00, rc 0) 00000100:00100000:1.0:1788619927.182816:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5027800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:14908:x1875503571627136:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4406us (4554us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619927.182824:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1364 00000100:00100000:0.0:1788619927.186399:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5025f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589211648:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619927.186427:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619927.186495:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5025180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+14:2355:x1875503571627264:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 4589us (4732us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619927.186508:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 448 00000100:00100000:0.0:1788619927.188147:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619927.188224:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571627392 00000100:00100000:1.0:1788619927.188232:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571627392 00000100:00100000:1.0:1788619927.188288:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1116 00000100:00100000:1.0:1788619927.188298:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:14908:x1875503571627392:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619927.188315:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619927.188357:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006901c1d2/0x1cd7fe53687360a0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d242 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.188387:0:6698:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000001544d1a8/0x1cd7fe5368735fdc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1a8 expref: 9 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.188396:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(10 13) 00010000:00010000:1.0:1788619927.188400:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(11 13) 00010000:00010000:1.0:1788619927.188406:0:6698:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006901c1d2/0x1cd7fe53687360a0 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d242 expref: 7 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.188504:0:6698:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000001544d1a8/0x1cd7fe5368735fdc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1a8 expref: 9 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.188519:0:6698:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000001544d1a8/0x1cd7fe5368735fdc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1a8 expref: 9 pid: 6696 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619927.188532:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c75f4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589211776:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619927.188602:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78e03f5600 going to sleep for 11 seconds 00000100:00100000:1.0:1788619927.196572:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c75f4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589211776:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619927.196620:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000006901c1d2/0x1cd7fe53687360a0 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d242 expref: 7 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.196642:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296a00, rc 0) 00000100:00100000:1.0:1788619927.196706:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:14908:x1875503571627392:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8408us (8567us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619927.196715:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1116 00000100:00100000:2.0:1788619927.200888:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619927.201103:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571627520 00000100:00100000:0.0:1788619927.201113:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571627520 00010000:00010000:0.0:1788619927.201152:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000001544d1a8/0x1cd7fe5368735fdc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1a8 expref: 10 pid: 6696 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619927.201284:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 449 00000100:00100000:0.0:1788619927.201295:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5024e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571627520:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619927.201310:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619927.201364:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000001544d1a8/0x1cd7fe5368735fdc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1a8 expref: 10 pid: 6696 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.201378:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000001544d1a8/0x1cd7fe5368735fdc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1a8 expref: 10 pid: 6696 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.201409:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001544d1a8/0x1cd7fe5368735fdc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d1a8 expref: 10 pid: 6696 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.201426:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(11 13) 00010000:00010000:0.0:1788619927.201439:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(11 13) 00010000:00010000:0.0:1788619927.201448:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000006901c1d2/0x1cd7fe53687360a0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d242 expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.201460:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006901c1d2/0x1cd7fe53687360a0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d242 expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.201496:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000006901c1d2/0x1cd7fe53687360a0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d242 expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619927.201509:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be5f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589211904:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619927.201584:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590240 going to sleep for 11 seconds 00000100:00100000:1.0:1788619927.204028:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be5f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589211904:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619927.204054:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619927.204102:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5024e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571627520:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 2809us (3234us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619927.204112:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 449 00000100:00100000:2.0:1788619927.206613:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619927.206699:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571627648 00000100:00100000:3.0:1788619927.206710:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571627648 00002000:00010000:3.0:1788619927.206751:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1083295->18446744073709551615) 00002000:00100000:3.0:1788619927.206755:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5025f80 with x1875503571627648 ext(1083295->18446744073709551615) 00002000:00010000:3.0:1788619927.206768:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5025f80. 00000100:00100000:3.0:1788619927.206803:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 637 00000100:00100000:3.0:1788619927.206813:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5025f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571627648:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:3.0:1788619927.207068:0:6703:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 0 -> 1083295 00002000:00010000:3.0:1788619927.207072:0:6703:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619926 -> 1788619927 00002000:00010000:3.0:1788619927.207075:0:6703:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619926 -> 1788619925 00002000:00010000:3.0:1788619927.207077:0:6703:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619926 -> 1788619927 00002000:00010000:3.0:1788619927.207079:0:6703:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 391 -> 391 00000100:00100000:3.0:1788619927.207140:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5025f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571627648:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 329us (545us total) trans 4294967486 rc 0/0 00000100:00100000:3.0:1788619927.207149:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 637 00002000:00010000:3.0:1788619927.207153:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1083295->18446744073709551615) 00002000:00100000:3.0:1788619927.207155:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5025f80 with x1875503571627648 ext(1083295->18446744073709551615) 00002000:00010000:3.0:1788619927.207160:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5025f80. 00000100:00100000:2.0:1788619927.211419:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619927.211654:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571627776 00000100:00100000:0.0:1788619927.211663:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571627776 00002000:00100000:0.0:1788619927.211708:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1073152->1077247) req@ffff8e77c5025c00 x1875503571627776/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:23/0 lens 488/0 e 0 to 0 dl 1788619938 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619927.211720:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5025c00 with x1875503571627776 ext(1073152->1077247) 00002000:00010000:0.0:1788619927.211725:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5025c00 00000100:00100000:0.0:1788619927.211756:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 638 00000100:00100000:0.0:1788619927.211764:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5025c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571627776:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619927.212015:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5025c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571627776:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 250us (606us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619927.212022:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 638 00002000:00100000:0.0:1788619927.212028:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1073152->1077247) req@ffff8e77c5025c00 x1875503571627776/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:23/0 lens 488/4536 e 0 to 0 dl 1788619938 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619927.212037:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5025c00 with x1875503571627776 ext(1073152->1077247) 00002000:00010000:0.0:1788619927.212041:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5025c00 00000100:00100000:2.0:1788619927.215654:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619927.215809:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571627904 00000100:00100000:1.0:1788619927.215820:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571627904 00002000:00100000:1.0:1788619927.215879:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1077248->1081343) req@ffff8e77c5025500 x1875503571627904/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:23/0 lens 488/0 e 0 to 0 dl 1788619938 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619927.215891:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5025500 with x1875503571627904 ext(1077248->1081343) 00002000:00010000:1.0:1788619927.215897:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5025500 00000100:00100000:1.0:1788619927.215932:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 639 00000100:00100000:1.0:1788619927.215942:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5025500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571627904:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619927.216205:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5025500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571627904:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 264us (560us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619927.216210:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 639 00002000:00100000:1.0:1788619927.216216:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1077248->1081343) req@ffff8e77c5025500 x1875503571627904/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:23/0 lens 488/4536 e 0 to 0 dl 1788619938 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619927.216221:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5025500 with x1875503571627904 ext(1077248->1081343) 00002000:00010000:1.0:1788619927.216224:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5025500 00000100:00100000:2.0:1788619927.220673:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619927.220744:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571628032 00000100:00100000:3.0:1788619927.220752:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571628032 00002000:00100000:3.0:1788619927.220793:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1085439) req@ffff8e77c5026680 x1875503571628032/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:23/0 lens 504/0 e 0 to 0 dl 1788619938 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619927.220801:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5026680 with x1875503571628032 ext(1048576->1085439) 00002000:00010000:3.0:1788619927.220805:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5026680 00000100:00100000:3.0:1788619927.220831:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 640 00000100:00100000:3.0:1788619927.220838:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5026680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571628032:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619927.222720:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619927.222795:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571628160 00000100:00100000:1.0:1788619927.222803:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571628160 00002000:00100000:1.0:1788619927.222841:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5026d80 x1875503571628160/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:23/0 lens 488/0 e 0 to 0 dl 1788619938 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619927.222853:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5026d80 with x1875503571628160 00000100:00100000:1.0:1788619927.222897:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 824 00000100:00100000:1.0:1788619927.222905:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5026d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571628160:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619927.224843:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5026d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571628160:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 1928us (2119us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619927.224854:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 824 00002000:00100000:1.0:1788619927.224864:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5026d80 x1875503571628160/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:23/0 lens 488/440 e 0 to 0 dl 1788619938 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619927.225185:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5026d80 with x1875503571628160 00000100:00100000:0.0:1788619927.227101:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5026680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571628032:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 6255us (6431us total) trans 0 rc 28672/28672 00000100:00100000:0.0:1788619927.227110:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 640 00002000:00100000:0.0:1788619927.227119:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1085439) req@ffff8e77c5026680 x1875503571628032/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:23/0 lens 504/440 e 0 to 0 dl 1788619938 ref 1 fl Complete:/600/0 rc 28672/28672 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619927.227127:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5026680 with x1875503571628032 ext(1048576->1085439) 00002000:00010000:0.0:1788619927.227131:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5026680 00000100:00100000:3.0:1788619927.229511:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619927.229753:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571628288 00000100:00100000:0.0:1788619927.229761:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571628288 00002000:00100000:0.0:1788619927.229799:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1073152->1083294) req@ffff8e78fefd6300 x1875503571628288/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:23/0 lens 10632/0 e 0 to 0 dl 1788619938 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619927.229808:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fefd6300 with x1875503571628288 ext(1073152->1083294) 00002000:00010000:0.0:1788619927.229812:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fefd6300 00000100:00100000:0.0:1788619927.229841:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 641 00000100:00100000:0.0:1788619927.229847:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2352:x1875503571628288:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:0.0:1788619927.230273:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+10:2352:x1875503571628288:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 425us (770us total) trans 4294967487 rc 0/0 00000100:00100000:0.0:1788619927.230279:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 641 00002000:00100000:0.0:1788619927.230285:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1073152->1083294) req@ffff8e78fefd6300 x1875503571628288/t4294967487(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:23/0 lens 10632/448 e 0 to 0 dl 1788619938 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619927.230290:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fefd6300 with x1875503571628288 ext(1073152->1083294) 00002000:00010000:0.0:1788619927.230293:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fefd6300 00000100:00100000:1.0:1788619927.232513:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619927.232580:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571628416 00000100:00100000:3.0:1788619927.232588:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571628416 00000100:00100000:3.0:1788619927.232643:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1117 00000100:00100000:3.0:1788619927.232652:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+10:2353:x1875503571628416:12345-192.168.201.5@tcp:16:fsx.0 00000001:00080000:1.0:1788619927.248507:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967487 is committed 00000001:00080000:1.0:1788619927.248528:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967484 is committed 00000001:00080000:1.0:1788619927.248537:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967485 is committed 00000100:00100000:0.0:1788619927.248757:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571628416:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 16101us (16248us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619927.248766:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1117 00000100:00100000:2.0:1788619927.253017:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619927.253130:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571628544 00000100:00100000:3.0:1788619927.253140:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571628544 00000100:00100000:3.0:1788619927.253263:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1365 00000100:00100000:3.0:1788619927.253272:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5027100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:14908:x1875503571628544:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619927.253288:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619927.253306:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71800/0x1cd7fe53687360a7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d249 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619927.253399:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70e00/0x1cd7fe53687360ae lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619927.253416:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70e00/0x1cd7fe53687360ae lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619927.253427:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70e00/0x1cd7fe53687360ae lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788619927.253457:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1083295 00000004:00200000:3.0:1788619927.253461:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788619927.253479:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70e00/0x1cd7fe53687360ae lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619927.253505:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71800/0x1cd7fe53687360a7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d249 expref: 17 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619927.253522:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70e00/0x1cd7fe53687360ae lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d249 expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619927.253531:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619927.253533:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788619927.253537:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f70e00, rc 0) 00000100:00100000:3.0:1788619927.253598:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5027100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:14908:x1875503571628544:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 327us (722us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619927.253617:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1365 00000100:00100000:3.0:1788619927.259406:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619927.259506:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571628672 00000100:00100000:0.0:1788619927.259516:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571628672 00000100:00100000:0.0:1788619927.259575:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1118 00000100:00100000:0.0:1788619927.259585:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:14908:x1875503571628672:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619927.259613:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619927.259639:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c161f442/0x1cd7fe53687360b5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d250 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.259672:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000006901c1d2/0x1cd7fe53687360a0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d242 expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.259681:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(11 13) 00010000:00010000:0.0:1788619927.259686:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(12 13) 00010000:00010000:0.0:1788619927.259691:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c161f442/0x1cd7fe53687360b5 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [2023424->2072575] (req 2023424->2072575) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d250 expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.259727:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000006901c1d2/0x1cd7fe53687360a0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d242 expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.259742:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000006901c1d2/0x1cd7fe53687360a0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d242 expref: 5 pid: 6698 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619927.259756:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589212032:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619927.259902:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590840 going to sleep for 11 seconds 00000100:00100000:1.0:1788619927.266578:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619927.266699:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571628800 00000100:00100000:2.0:1788619927.266709:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571628800 00010000:00010000:2.0:1788619927.266750:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000006901c1d2/0x1cd7fe53687360a0 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d242 expref: 6 pid: 6698 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619927.266789:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 450 00000100:00100000:2.0:1788619927.266799:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571628800:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619927.266814:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:2.0:1788619927.266832:0:5831:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619925 -> 1788619926 00010000:00010000:2.0:1788619927.266839:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000006901c1d2/0x1cd7fe53687360a0 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d242 expref: 6 pid: 6698 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.266854:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000006901c1d2/0x1cd7fe53687360a0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d242 expref: 6 pid: 6698 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.266879:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(12 13) 00010000:00010000:2.0:1788619927.266884:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(12 13) 00010000:00010000:2.0:1788619927.266893:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000c161f442/0x1cd7fe53687360b5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2023424->2072575) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d250 expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.266904:0:5831:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c161f442/0x1cd7fe53687360b5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2023424->2072575) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d250 expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.266942:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000c161f442/0x1cd7fe53687360b5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2023424->2072575) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d250 expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619927.266956:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5024700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5831:1875503589212160:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619927.267041:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164540 going to sleep for 11 seconds 00010000:00010000:0.0:1788619927.267210:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006901c1d2/0x1cd7fe53687360a0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d242 expref: 6 pid: 6698 timeout: 519 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619927.267225:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589212032:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619927.267250:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c161f442/0x1cd7fe53687360b5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2023424->2072575) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d250 expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.267272:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d4400, rc 0) 00000100:00100000:0.0:1788619927.267299:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:14908:x1875503571628672:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7716us (7903us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619927.267306:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1118 00000100:00100000:3.0:1788619927.280211:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5024700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5831:1875503589212160:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619927.280241:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619927.280309:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571628800:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 13510us (13739us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619927.280320:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 450 00000100:00100000:1.0:1788619927.285720:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619927.285824:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571628928 00000100:00100000:2.0:1788619927.285834:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571628928 00000100:00100000:2.0:1788619927.285892:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1119 00000100:00100000:2.0:1788619927.285902:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571628928:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619927.285921:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619927.285937:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002cc077c3/0x1cd7fe53687360bc lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d257 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619927.285996:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5024380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589212288:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619927.286086:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164780 going to sleep for 11 seconds 00002000:00010000:2.0:1788619927.291551:0:6696:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size: 1083295 -> 2072189 00000100:00100000:2.0:1788619927.291583:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5024380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589212288:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619927.291629:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002cc077c3/0x1cd7fe53687360bc lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d257 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.291655:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002cc077c3/0x1cd7fe53687360bc lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d257 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.291667:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f72e00, rc 0) 00000100:00100000:2.0:1788619927.291776:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571628928:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5873us (6063us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619927.291785:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1119 00000100:00100000:2.0:1788619927.306961:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619927.307064:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571629056 00000100:00100000:1.0:1788619927.307073:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571629056 00000100:00100000:1.0:1788619927.307135:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1120 00000100:00100000:1.0:1788619927.307145:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5024380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571629056:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619927.307163:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619927.307180:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003086a8b7/0x1cd7fe53687360c3 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d25e expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619927.307249:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c75f5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589212416:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619927.307403:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc240 going to sleep for 11 seconds 00000100:00100000:1.0:1788619927.310709:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c75f5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589212416:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619927.310748:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003086a8b7/0x1cd7fe53687360c3 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d25e expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.310771:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003086a8b7/0x1cd7fe53687360c3 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d25e expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.310784:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296400, rc 0) 00000100:00100000:1.0:1788619927.310843:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5024380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571629056:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3699us (3892us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619927.310852:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1120 00000100:00100000:2.0:1788619927.316396:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619927.316476:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571629184 00000100:00100000:0.0:1788619927.316486:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571629184 00000100:00100000:0.0:1788619927.316539:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1121 00000100:00100000:0.0:1788619927.316549:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5024a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571629184:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619927.316569:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619927.316588:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dbe89f01/0x1cd7fe53687360ca lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d265 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619927.316671:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589212544:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619927.316759:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590840 going to sleep for 11 seconds 00000100:00100000:0.0:1788619927.323762:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589212544:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619927.323800:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000dbe89f01/0x1cd7fe53687360ca lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d265 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.323821:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dbe89f01/0x1cd7fe53687360ca lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d265 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.323831:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d5a00, rc 0) 00000100:00100000:0.0:1788619927.323890:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5024a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571629184:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7343us (7505us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619927.323896:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1121 00000100:00100000:3.0:1788619927.325972:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619927.326035:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571629312 00000100:00100000:0.0:1788619927.326043:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571629312 00000100:00100000:0.0:1788619927.326085:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1122 00000100:00100000:0.0:1788619927.326092:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571629312:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619927.326104:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619927.326117:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008b39475f/0x1cd7fe53687360d1 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d26c expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619927.326158:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589212672:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619927.326224:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590840 going to sleep for 11 seconds 00000100:00100000:1.0:1788619927.335719:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589212672:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619927.335764:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008b39475f/0x1cd7fe53687360d1 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d26c expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.335789:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008b39475f/0x1cd7fe53687360d1 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d26c expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.335802:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d4e00, rc 0) 00000100:00100000:1.0:1788619927.335878:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571629312:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 9777us (9904us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619927.335888:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1122 00010000:00010000:3.0:1788619927.336023:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788619927.345550:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619927.345699:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571629440 00000100:00100000:0.0:1788619927.345708:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571629440 00000100:00100000:0.0:1788619927.345771:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1123 00000100:00100000:0.0:1788619927.345782:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900987050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:14908:x1875503571629440:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619927.345800:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619927.345820:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dbe89f01/0x1cd7fe53687360d8 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d273 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.345852:0:6696:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000c161f442/0x1cd7fe53687360b5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2023424->2072575) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d250 expref: 5 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.345861:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=1s v=(12 13) 00010000:00010000:0.0:1788619927.345866:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(1 12) 00010000:00010000:0.0:1788619927.345885:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dbe89f01/0x1cd7fe53687360d8 lrc: 4/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1605632->1638399] (req 1605632->1638399) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d273 expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.345920:0:6696:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000c161f442/0x1cd7fe53687360b5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2023424->2072575) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d250 expref: 5 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.345935:0:6696:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000c161f442/0x1cd7fe53687360b5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2023424->2072575) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d250 expref: 5 pid: 6697 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619927.345948:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be6d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589212800:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619927.346023:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590600 going to sleep for 11 seconds 00000100:00100000:0.0:1788619927.350508:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be6d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589212800:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619927.350545:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000dbe89f01/0x1cd7fe53687360d8 lrc: 3/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1605632->1638399] (req 1605632->1638399) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d273 expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.350567:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d5a00, rc 0) 00000100:00100000:0.0:1788619927.350645:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900987050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:14908:x1875503571629440:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4863us (5102us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619927.350654:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1123 00000100:00100000:2.0:1788619927.352170:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619927.352321:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571629568 00000100:00100000:0.0:1788619927.352359:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571629568 00002000:00100000:0.0:1788619927.352406:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (2023424->2072188) req@ffff8e77c5024000 x1875503571629568/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:23/0 lens 488/0 e 0 to 0 dl 1788619938 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619927.352417:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5024000 with x1875503571629568 ext(2023424->2072188) 00010000:00010000:0.0:1788619927.352426:0:6703:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000c161f442/0x1cd7fe53687360b5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2023424->2072575) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d250 expref: 6 pid: 6697 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.352442:0:6703:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000c161f442/0x1cd7fe53687360b5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2023424->2072575) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d250 expref: 6 pid: 6697 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788619927.352450:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff8e77c5024000 00000100:00100000:0.0:1788619927.352473:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 21 00000100:00100000:0.0:1788619927.352482:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5024000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571629568:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:0.0:1788619927.355917:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5024000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+8:2355:x1875503571629568:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 3424us (3741us total) trans 4294967488 rc 0/0 00000100:00100000:0.0:1788619927.355931:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 21 00002000:00100000:0.0:1788619927.355943:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (2023424->2072188) req@ffff8e77c5024000 x1875503571629568/t4294967488(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:23/0 lens 488/448 e 0 to 0 dl 1788619938 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619927.355961:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5024000 with x1875503571629568 ext(2023424->2072188) 00010000:00010000:0.0:1788619927.355972:0:6703:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000c161f442/0x1cd7fe53687360b5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2023424->2072575) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d250 expref: 8 pid: 6697 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.355990:0:6703:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000c161f442/0x1cd7fe53687360b5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2023424->2072575) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d250 expref: 8 pid: 6697 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788619927.355998:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff8e77c5024000 00000100:00100000:2.0:1788619927.360816:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619927.360993:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571629696 00000100:00100000:1.0:1788619927.361003:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571629696 00010000:00010000:1.0:1788619927.361045:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000c161f442/0x1cd7fe53687360b5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2023424->2072575) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d250 expref: 8 pid: 6697 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619927.361079:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 451 00000100:00100000:1.0:1788619927.361088:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5026300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+8:2353:x1875503571629696:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619927.361104:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619927.361125:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000c161f442/0x1cd7fe53687360b5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2023424->2072575) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d250 expref: 8 pid: 6697 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.361138:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000c161f442/0x1cd7fe53687360b5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2023424->2072575) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d250 expref: 8 pid: 6697 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.361162:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c161f442/0x1cd7fe53687360b5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2023424->2072575) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d250 expref: 8 pid: 6697 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.361179:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(1 12) 00010000:00010000:1.0:1788619927.361184:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(1 12) 00010000:00010000:1.0:1788619927.361205:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000dbe89f01/0x1cd7fe53687360d8 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1605632->1638399) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d273 expref: 5 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.361217:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dbe89f01/0x1cd7fe53687360d8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1605632->1638399) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d273 expref: 5 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.361274:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000dbe89f01/0x1cd7fe53687360d8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1605632->1638399) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d273 expref: 5 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619927.361294:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c75f5880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589212928:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619927.361420:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc240 going to sleep for 11 seconds 00000100:00100000:2.0:1788619927.367101:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619927.367255:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571629824 00000100:00100000:3.0:1788619927.367266:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571629824 00002000:00100000:3.0:1788619927.367313:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1605632->1638399) req@ffff8e77c5026a00 x1875503571629824/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:23/0 lens 488/0 e 0 to 0 dl 1788619938 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619927.367361:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5026a00 with x1875503571629824 ext(1605632->1638399) 00002000:00010000:3.0:1788619927.367367:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5026a00 00000100:00100000:3.0:1788619927.367394:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 642 00000100:00100000:3.0:1788619927.367403:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5026a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571629824:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619927.368044:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c75f5880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589212928:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619927.368067:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619927.368105:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5026300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2353:x1875503571629696:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 7017us (7300us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619927.368113:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 451 00000100:00100000:1.0:1788619927.370440:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5026a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571629824:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 3027us (3338us total) trans 0 rc 32768/32768 00000100:00100000:1.0:1788619927.370452:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 642 00002000:00100000:1.0:1788619927.370465:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1605632->1638399) req@ffff8e77c5026a00 x1875503571629824/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:23/0 lens 488/440 e 0 to 0 dl 1788619938 ref 1 fl Complete:/600/0 rc 32768/32768 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619927.370476:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5026a00 with x1875503571629824 ext(1605632->1638399) 00002000:00010000:1.0:1788619927.370481:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5026a00 00000100:00100000:3.0:1788619927.378424:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619927.378530:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571629952 00000100:00100000:2.0:1788619927.378541:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571629952 00000100:00100000:2.0:1788619927.378619:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1366 00000100:00100000:2.0:1788619927.378631:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+14:14908:x1875503571629952:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619927.378652:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619927.378675:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72400/0x1cd7fe53687360df lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d27a expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.378764:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72800/0x1cd7fe53687360e6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.378780:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72800/0x1cd7fe53687360e6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.378791:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72800/0x1cd7fe53687360e6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788619927.378820:0:9417:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1083295 00000004:00200000:2.0:1788619927.378824:0:9417:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788619927.378843:0:9417:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72800/0x1cd7fe53687360e6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.378885:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72400/0x1cd7fe53687360df lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d27a expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.378903:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72800/0x1cd7fe53687360e6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d27a expref: 15 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.378911:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619927.378914:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788619927.378917:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f72800, rc 0) 00000100:00100000:2.0:1788619927.378986:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+15:14908:x1875503571629952:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 358us (573us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619927.378992:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1366 00000100:00100000:3.0:1788619927.382749:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619927.382794:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619927.382904:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571630080 00000100:00100000:0.0:1788619927.382914:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571630080 00000100:00100000:0.0:1788619927.382979:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1367 00000100:00100000:0.0:1788619927.382989:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+15:2355:x1875503571630080:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619927.383012:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619927.383036:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5e00/0x1cd7fe53687360ed lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d281 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.383077:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4a00/0x1cd7fe5368736099 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d23b expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619927.383124:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be4380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589213056:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619927.383215:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590600 going to sleep for 11 seconds 00000100:00100000:1.0:1788619927.383378:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571630208 00000100:00100000:1.0:1788619927.383383:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571630208 00000100:00100000:1.0:1788619927.383426:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1124 00000100:00100000:1.0:1788619927.383433:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571630208:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619927.383442:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619927.383452:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003086a8b7/0x1cd7fe53687360f4 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d288 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.383472:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(1 12) 00010000:00010000:1.0:1788619927.383476:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(1 12) 00010000:00010000:1.0:1788619927.383487:0:6698:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003086a8b7/0x1cd7fe53687360f4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d288 expref: 8 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.383504:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003086a8b7/0x1cd7fe53687360f4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d288 expref: 8 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.383523:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296400, rc 0) 00000100:00100000:1.0:1788619927.383552:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+8:2355:x1875503571630208:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 119us (813us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619927.383559:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1124 00000004:00010000:0.0:1788619927.388858:0:5846:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619926 -> 1788619927 00000100:00100000:0.0:1788619927.388903:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be4380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589213056:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619927.388925:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 541014 00010000:00010000:0.0:1788619927.388946:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5e00/0x1cd7fe53687360ed lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d281 expref: 16 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.388966:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619927.388968:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619927.388973:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5e00/0x1cd7fe53687360ed lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d281 expref: 16 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.388991:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d5e00, rc 0) 00000100:00100000:0.0:1788619927.389052:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+15:2355:x1875503571630080:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6065us (6257us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619927.389061:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1367 00000100:00100000:3.0:1788619927.393776:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619927.394109:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571630336 00000100:00100000:2.0:1788619927.394118:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571630336 00000100:00100000:2.0:1788619927.394169:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1368 00000100:00100000:2.0:1788619927.394175:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+15:2354:x1875503571630336:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619927.394193:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619927.394212:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72400/0x1cd7fe53687360fb lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d28f expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.394251:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4a00/0x1cd7fe5368736099 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d23b expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619927.394294:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5025880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589213184:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619927.394415:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164780 going to sleep for 11 seconds 00000100:00100000:2.0:1788619927.402113:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5025880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589213184:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619927.402141:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 541014 00010000:00010000:2.0:1788619927.402160:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72400/0x1cd7fe53687360fb lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d28f expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.402181:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619927.402191:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619927.402197:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72400/0x1cd7fe53687360fb lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d28f expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.402213:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f72400, rc 0) 00000100:00100000:2.0:1788619927.402289:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+15:2354:x1875503571630336:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8112us (8523us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619927.402298:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1368 00000100:00100000:3.0:1788619927.413261:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619927.413685:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571630464 00000100:00100000:1.0:1788619927.413695:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571630464 00000100:00100000:1.0:1788619927.413756:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1369 00000100:00100000:1.0:1788619927.413766:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+15:2354:x1875503571630464:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619927.413785:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619927.413806:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296c00/0x1cd7fe5368736102 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d296 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.413841:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4a00/0x1cd7fe5368736099 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d23b expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619927.413889:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c75f5f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589213312:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619927.413976:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccd80 going to sleep for 11 seconds 00000100:00100000:1.0:1788619927.416096:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c75f5f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589213312:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619927.416124:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 541014 00010000:00010000:1.0:1788619927.416142:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296c00/0x1cd7fe5368736102 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d296 expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.416164:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619927.416167:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619927.416173:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296c00/0x1cd7fe5368736102 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d296 expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.416188:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296c00, rc 0) 00000100:00100000:1.0:1788619927.416246:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+15:2354:x1875503571630464:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2480us (2996us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619927.416254:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1369 00000100:00100000:3.0:1788619927.421039:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619927.428730:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571630592 00000100:00100000:0.0:1788619927.428745:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571630592 00000100:00100000:0.0:1788619927.428825:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1370 00000100:00100000:0.0:1788619927.428835:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+15:14908:x1875503571630592:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619927.428854:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619927.428876:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5e00/0x1cd7fe5368736109 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d29d expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.428907:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4a00/0x1cd7fe5368736099 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d23b expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.428924:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5e00/0x1cd7fe5368736109 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d29d expref: 16 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.428961:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4a00/0x1cd7fe5368736099 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d23b expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.428974:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4a00/0x1cd7fe5368736099 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d23b expref: 15 pid: 5846 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619927.428988:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be4380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589213440:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619927.429087:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590600 going to sleep for 11 seconds 00000100:00100000:0.0:1788619927.434062:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be4380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589213440:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619927.434098:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5e00/0x1cd7fe5368736109 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d29d expref: 16 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.434117:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619927.434120:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619927.434125:0:5846:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5e00/0x1cd7fe5368736109 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d29d expref: 16 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.434136:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d5e00, rc 0) 00000100:00100000:0.0:1788619927.434214:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+16:14908:x1875503571630592:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5379us (13184us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619927.434223:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1370 00000100:00100000:3.0:1788619927.442258:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619927.442370:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571630720 00000100:00100000:0.0:1788619927.442381:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571630720 00010000:00010000:0.0:1788619927.442424:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4a00/0x1cd7fe5368736099 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d23b expref: 16 pid: 5846 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619927.442472:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 452 00000100:00100000:0.0:1788619927.442482:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcaf450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571630720:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619927.442496:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619927.442521:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4a00/0x1cd7fe5368736099 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d23b expref: 16 pid: 5846 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.442533:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4a00/0x1cd7fe5368736099 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d23b expref: 16 pid: 5846 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.442554:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4a00/0x1cd7fe5368736099 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d23b expref: 16 pid: 5846 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.442569:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619927.442575:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5e00/0x1cd7fe5368736109 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d29d expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.442586:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5e00/0x1cd7fe5368736109 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d29d expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.442596:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5e00/0x1cd7fe5368736109 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d29d expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.442642:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5e00/0x1cd7fe5368736109 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d29d expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619927.442660:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be4380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589213568:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619927.442746:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590600 going to sleep for 11 seconds 00000100:00100000:3.0:1788619927.446815:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619927.446934:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be4380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589213568:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619927.446957:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619927.447018:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcaf450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571630720:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 4537us (4770us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619927.447028:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 452 00000100:00100000:2.0:1788619927.453010:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571630848 00000100:00100000:2.0:1788619927.453027:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571630848 00002000:00100000:3.0:1788619927.453237:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fefd6680 x1875503571630848/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:23/0 lens 488/0 e 0 to 0 dl 1788619938 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619927.453249:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fefd6680 with x1875503571630848 00000100:00100000:3.0:1788619927.453289:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 825 00000100:00100000:3.0:1788619927.453299:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571630848:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619927.453793:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571630848:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 492us (6984us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619927.453802:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 825 00002000:00100000:3.0:1788619927.453809:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fefd6680 x1875503571630848/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:23/0 lens 488/4536 e 0 to 0 dl 1788619938 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619927.453818:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fefd6680 with x1875503571630848 00000100:00100000:1.0:1788619927.468729:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619927.468826:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571630976 00000100:00100000:0.0:1788619927.468835:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571630976 00002000:00100000:0.0:1788619927.468894:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c75f5f80 x1875503571630976/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:23/0 lens 488/0 e 0 to 0 dl 1788619938 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619927.468905:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c75f5f80 with x1875503571630976 00000100:00100000:0.0:1788619927.468945:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 826 00000100:00100000:0.0:1788619927.468955:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+16:2354:x1875503571630976:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619927.469166:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+16:2354:x1875503571630976:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 212us (446us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619927.469173:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 826 00002000:00100000:0.0:1788619927.469185:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c75f5f80 x1875503571630976/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:23/0 lens 488/4536 e 0 to 0 dl 1788619938 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619927.469199:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c75f5f80 with x1875503571630976 00000100:00100000:3.0:1788619927.478104:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619927.478264:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571631104 00000100:00100000:0.0:1788619927.478274:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571631104 00002000:00100000:0.0:1788619927.478321:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fefd7800 x1875503571631104/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:23/0 lens 488/0 e 0 to 0 dl 1788619938 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619927.478355:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fefd7800 with x1875503571631104 00000100:00100000:0.0:1788619927.478568:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 827 00000100:00100000:0.0:1788619927.478579:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571631104:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619927.478818:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571631104:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 240us (721us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619927.478826:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 827 00002000:00100000:0.0:1788619927.478832:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fefd7800 x1875503571631104/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:23/0 lens 488/4536 e 0 to 0 dl 1788619938 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619927.478840:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fefd7800 with x1875503571631104 00000100:00100000:3.0:1788619927.481478:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619927.481556:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571631232 00000100:00100000:0.0:1788619927.481566:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571631232 00002000:00100000:1.0:1788619927.481773:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fefd5500 x1875503571631232/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:23/0 lens 488/0 e 0 to 0 dl 1788619938 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619927.481786:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fefd5500 with x1875503571631232 00000100:00100000:1.0:1788619927.481857:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 828 00000100:00100000:1.0:1788619927.481868:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+16:2352:x1875503571631232:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619927.482102:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+16:2352:x1875503571631232:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 234us (631us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619927.482110:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 828 00002000:00100000:1.0:1788619927.482118:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fefd5500 x1875503571631232/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:23/0 lens 488/4536 e 0 to 0 dl 1788619938 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619927.482127:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fefd5500 with x1875503571631232 00000100:00100000:3.0:1788619927.484668:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619927.484926:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571631360 00000100:00100000:2.0:1788619927.484937:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571631360 00002000:00100000:2.0:1788619927.484986:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fefd5f80 x1875503571631360/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:23/0 lens 488/0 e 0 to 0 dl 1788619938 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619927.484997:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fefd5f80 with x1875503571631360 00000100:00100000:2.0:1788619927.485033:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 829 00000100:00100000:2.0:1788619927.485043:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+16:2354:x1875503571631360:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619927.485293:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+16:2354:x1875503571631360:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 250us (633us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619927.485300:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 829 00002000:00100000:2.0:1788619927.485306:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fefd5f80 x1875503571631360/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:23/0 lens 488/4536 e 0 to 0 dl 1788619938 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619927.485314:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fefd5f80 with x1875503571631360 00000100:00100000:3.0:1788619927.488221:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619927.488369:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571631488 00000100:00100000:1.0:1788619927.488379:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571631488 00002000:00100000:1.0:1788619927.488484:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fefd5180 x1875503571631488/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:23/0 lens 488/0 e 0 to 0 dl 1788619938 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619927.488496:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fefd5180 with x1875503571631488 00000100:00100000:1.0:1788619927.488528:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 830 00000100:00100000:1.0:1788619927.488539:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571631488:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619927.488807:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571631488:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 269us (595us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619927.488815:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 830 00002000:00100000:1.0:1788619927.488822:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fefd5180 x1875503571631488/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:23/0 lens 488/4536 e 0 to 0 dl 1788619938 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619927.488829:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fefd5180 with x1875503571631488 00000100:00100000:3.0:1788619927.498796:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619927.499065:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571631616 00000100:00100000:2.0:1788619927.499076:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571631616 00002000:00100000:2.0:1788619927.499127:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fefd4000 x1875503571631616/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:23/0 lens 488/0 e 0 to 0 dl 1788619938 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619927.499145:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fefd4000 with x1875503571631616 00000100:00100000:2.0:1788619927.499179:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 831 00000100:00100000:2.0:1788619927.499188:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571631616:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619927.499690:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571631616:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 501us (906us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619927.499698:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 831 00002000:00100000:2.0:1788619927.499705:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fefd4000 x1875503571631616/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:23/0 lens 488/4536 e 0 to 0 dl 1788619938 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619927.499714:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fefd4000 with x1875503571631616 00000100:00100000:3.0:1788619927.504171:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619927.504455:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571631744 00000100:00100000:1.0:1788619927.504466:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571631744 00002000:00100000:1.0:1788619927.504517:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69f8000 x1875503571631744/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:23/0 lens 488/0 e 0 to 0 dl 1788619938 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619927.504528:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69f8000 with x1875503571631744 00000100:00100000:1.0:1788619927.504567:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 832 00000100:00100000:1.0:1788619927.504576:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69f8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+16:2354:x1875503571631744:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619927.504801:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69f8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+16:2354:x1875503571631744:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 226us (640us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619927.504809:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 832 00002000:00100000:1.0:1788619927.504816:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69f8000 x1875503571631744/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:23/0 lens 488/4536 e 0 to 0 dl 1788619938 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619927.504825:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69f8000 with x1875503571631744 00000100:00100000:0.0:1788619927.513617:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619927.513712:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571631872 00000100:00100000:2.0:1788619927.513722:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571631872 00002000:00100000:2.0:1788619927.513775:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be4380 x1875503571631872/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:23/0 lens 488/0 e 0 to 0 dl 1788619938 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619927.513786:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be4380 with x1875503571631872 00000100:00100000:2.0:1788619927.513824:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 833 00000100:00100000:2.0:1788619927.513835:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+16:2352:x1875503571631872:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619927.514055:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+16:2352:x1875503571631872:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 222us (458us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619927.514062:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 833 00002000:00100000:2.0:1788619927.514070:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be4380 x1875503571631872/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:23/0 lens 488/4536 e 0 to 0 dl 1788619938 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619927.514077:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be4380 with x1875503571631872 00000100:00100000:3.0:1788619927.518137:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619927.518245:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571632000 00000100:00100000:0.0:1788619927.518255:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571632000 00002000:00100000:0.0:1788619927.518303:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69f9c00 x1875503571632000/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:23/0 lens 488/0 e 0 to 0 dl 1788619938 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619927.518314:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69f9c00 with x1875503571632000 00000100:00100000:0.0:1788619927.518391:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 834 00000100:00100000:1.0:1788619927.518693:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69f9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+16:2352:x1875503571632000:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619927.518927:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69f9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+16:2352:x1875503571632000:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 300us (799us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619927.518937:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 834 00002000:00100000:1.0:1788619927.518944:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69f9c00 x1875503571632000/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:23/0 lens 488/4536 e 0 to 0 dl 1788619938 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619927.518951:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69f9c00 with x1875503571632000 00000100:00100000:3.0:1788619927.528858:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619927.531094:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571632128 00000100:00100000:1.0:1788619927.531110:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571632128 00002000:00100000:1.0:1788619927.531172:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69f8e00 x1875503571632128/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:23/0 lens 488/0 e 0 to 0 dl 1788619938 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619927.531184:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69f8e00 with x1875503571632128 00000100:00100000:1.0:1788619927.531233:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 835 00000100:00100000:1.0:1788619927.531243:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69f8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+16:2354:x1875503571632128:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619927.531741:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69f8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+16:2354:x1875503571632128:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 498us (2892us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619927.531750:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 835 00002000:00100000:1.0:1788619927.531758:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69f8e00 x1875503571632128/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:23/0 lens 488/4536 e 0 to 0 dl 1788619938 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619927.531766:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69f8e00 with x1875503571632128 00000100:00100000:3.0:1788619927.535670:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619927.535766:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571632256 00000100:00100000:2.0:1788619927.535776:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571632256 00002000:00100000:2.0:1788619927.535827:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69f9180 x1875503571632256/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:23/0 lens 488/0 e 0 to 0 dl 1788619938 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619927.535838:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69f9180 with x1875503571632256 00000100:00100000:2.0:1788619927.535880:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 836 00000100:00100000:2.0:1788619927.535890:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69f9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571632256:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619927.536108:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69f9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571632256:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 219us (450us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619927.536115:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 836 00002000:00100000:2.0:1788619927.536123:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69f9180 x1875503571632256/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:23/0 lens 488/4536 e 0 to 0 dl 1788619938 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619927.536131:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69f9180 with x1875503571632256 00000100:00100000:0.0:1788619927.545222:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619927.545318:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571632384 00000100:00100000:1.0:1788619927.545362:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571632384 00002000:00100000:1.0:1788619927.545591:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be6d80 x1875503571632384/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:23/0 lens 488/0 e 0 to 0 dl 1788619938 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619927.545612:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be6d80 with x1875503571632384 00000100:00100000:1.0:1788619927.545651:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 837 00000100:00100000:1.0:1788619927.545661:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+16:2352:x1875503571632384:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619927.546104:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+16:2352:x1875503571632384:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 443us (891us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619927.546117:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 837 00002000:00100000:1.0:1788619927.546125:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be6d80 x1875503571632384/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:23/0 lens 488/4536 e 0 to 0 dl 1788619938 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619927.546133:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be6d80 with x1875503571632384 00000100:00100000:3.0:1788619927.549446:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619927.550522:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571632512 00000100:00100000:1.0:1788619927.550535:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571632512 00002000:00100000:1.0:1788619927.550591:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69f9500 x1875503571632512/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:23/0 lens 488/0 e 0 to 0 dl 1788619938 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619927.550613:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69f9500 with x1875503571632512 00000100:00100000:1.0:1788619927.550655:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 838 00000100:00100000:1.0:1788619927.550664:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69f9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+16:2354:x1875503571632512:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619927.550951:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69f9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+16:2354:x1875503571632512:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 287us (1523us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619927.550959:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 838 00002000:00100000:1.0:1788619927.550966:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69f9500 x1875503571632512/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:23/0 lens 488/4536 e 0 to 0 dl 1788619938 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619927.550974:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69f9500 with x1875503571632512 00000100:00100000:3.0:1788619927.557140:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619927.557413:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571632640 00000100:00100000:2.0:1788619927.557424:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571632640 00002000:00100000:2.0:1788619927.557477:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69fb480 x1875503571632640/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:23/0 lens 488/0 e 0 to 0 dl 1788619938 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619927.557488:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69fb480 with x1875503571632640 00000100:00100000:2.0:1788619927.557531:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 839 00000100:00100000:2.0:1788619927.557541:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571632640:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619927.558111:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571632640:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 570us (982us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619927.558119:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 839 00002000:00100000:2.0:1788619927.558127:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69fb480 x1875503571632640/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:23/0 lens 488/4536 e 0 to 0 dl 1788619938 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619927.558134:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69fb480 with x1875503571632640 00000100:00100000:0.0:1788619927.570681:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619927.570775:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571632768 00000100:00100000:3.0:1788619927.570786:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571632768 00002000:00100000:3.0:1788619927.570837:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be6300 x1875503571632768/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:23/0 lens 488/0 e 0 to 0 dl 1788619938 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619927.570848:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be6300 with x1875503571632768 00000100:00100000:3.0:1788619927.570887:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 840 00000100:00100000:3.0:1788619927.570896:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+16:2354:x1875503571632768:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619927.571170:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+16:2354:x1875503571632768:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 274us (500us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619927.571178:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 840 00002000:00100000:3.0:1788619927.571186:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be6300 x1875503571632768/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:23/0 lens 488/4536 e 0 to 0 dl 1788619938 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619927.571195:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be6300 with x1875503571632768 00000100:00100000:3.0:1788619927.575620:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619927.575885:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571632896 00000100:00100000:2.0:1788619927.575896:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571632896 00002000:00100000:2.0:1788619927.575951:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5dc8700 x1875503571632896/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:23/0 lens 488/0 e 0 to 0 dl 1788619938 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619927.575963:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5dc8700 with x1875503571632896 00000100:00100000:2.0:1788619927.576003:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 841 00000100:00100000:2.0:1788619927.576013:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dc8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571632896:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:0.0:1788619927.581034:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 541014 -> 541014 00000004:00010000:0.0:1788619927.581044:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619927 -> 1788619927 00000004:00010000:0.0:1788619927.581046:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619927 -> 1788619926 00000004:00010000:0.0:1788619927.581049:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619927 -> 1788619927 00000004:00010000:0.0:1788619927.581051:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1800 -> 1800 00000100:00100000:0.0:1788619927.581126:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dc8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+18:2355:x1875503571632896:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 5110us (5521us total) trans 4294967781 rc 0/0 00000100:00100000:0.0:1788619927.581137:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 841 00002000:00100000:0.0:1788619927.581150:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5dc8700 x1875503571632896/t4294967781(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:23/0 lens 488/448 e 0 to 0 dl 1788619938 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619927.581160:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5dc8700 with x1875503571632896 00000100:00100000:3.0:1788619927.585170:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619927.585253:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571633024 00000100:00100000:2.0:1788619927.585262:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571633024 00000100:00100000:2.0:1788619927.585321:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1371 00000100:00100000:2.0:1788619927.585358:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dc9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+18:2354:x1875503571633024:12345-192.168.201.5@tcp:16:fsx.0 00000001:00080000:2.0:1788619927.643148:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967781 is committed 00000001:00080000:2.0:1788619927.643170:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967779 is committed 00000001:00080000:2.0:1788619927.643179:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967780 is committed 00000100:00100000:0.0:1788619927.645068:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dc9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571633024:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 59700us (59896us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619927.645082:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1371 00000100:00100000:3.0:1788619927.654630:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619927.654946:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571633152 00000100:00100000:1.0:1788619927.654963:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571633152 00000100:00100000:1.0:1788619927.655049:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1372 00000100:00100000:1.0:1788619927.655061:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dcaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571633152:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619927.655083:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619927.655106:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297200/0x1cd7fe5368736110 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d2a4 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.655146:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5e00/0x1cd7fe5368736109 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d29d expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.655158:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297200/0x1cd7fe5368736110 lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2a4 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.655197:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5e00/0x1cd7fe5368736109 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d29d expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.655211:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5e00/0x1cd7fe5368736109 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d29d expref: 9 pid: 5846 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619927.655225:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c75f4000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589213696:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619927.655380:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc240 going to sleep for 11 seconds 00000100:00100000:1.0:1788619927.658756:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c75f4000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589213696:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619927.658794:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297200/0x1cd7fe5368736110 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2a4 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.658813:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619927.658817:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619927.658821:0:5845:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297200/0x1cd7fe5368736110 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2a4 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.658831:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297200, rc 0) 00000100:00100000:1.0:1788619927.658915:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dcaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571633152:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3856us (4300us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619927.658924:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1372 00000100:00100000:3.0:1788619927.665322:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619927.666175:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571633280 00000100:00100000:0.0:1788619927.666194:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571633280 00010000:00010000:0.0:1788619927.666253:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5e00/0x1cd7fe5368736109 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d29d expref: 10 pid: 5846 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619927.666303:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 453 00000100:00100000:0.0:1788619927.666313:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dc9500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571633280:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619927.666353:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619927.666382:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5e00/0x1cd7fe5368736109 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d29d expref: 10 pid: 5846 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.666408:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5e00/0x1cd7fe5368736109 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d29d expref: 10 pid: 5846 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.666702:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5e00/0x1cd7fe5368736109 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d29d expref: 10 pid: 5846 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.666721:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619927.666729:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297200/0x1cd7fe5368736110 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2a4 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.666742:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297200/0x1cd7fe5368736110 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2a4 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.666751:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297200/0x1cd7fe5368736110 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2a4 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.666795:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297200/0x1cd7fe5368736110 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2a4 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619927.666810:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be5180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589213824:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619927.667096:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590000 going to sleep for 11 seconds 00000100:00100000:0.0:1788619927.673994:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be5180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589213824:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619927.674023:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619927.674090:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dc9500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571633280:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 7777us (8785us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619927.674101:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 453 00000100:00100000:2.0:1788619927.694449:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619927.694519:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619927.694546:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571633408 00000100:00100000:3.0:1788619927.694558:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571633408 00002000:00100000:3.0:1788619927.694620:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5025880 x1875503571633408/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:23/0 lens 488/0 e 0 to 0 dl 1788619938 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619927.694632:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5025880 with x1875503571633408 00000100:00100000:3.0:1788619927.694674:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 842 00000100:00100000:3.0:1788619927.694686:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5025880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571633408:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619927.695841:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571633536 00000100:00100000:0.0:1788619927.695850:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571633536 00002000:00100000:0.0:1788619927.695899:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1048576->2072575) req@ffff8e77c5027480 x1875503571633536/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:23/0 lens 504/0 e 0 to 0 dl 1788619938 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619927.695910:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5027480 with x1875503571633536 ext(1048576->2072575) 00002000:00010000:0.0:1788619927.695916:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5027480 00000100:00100000:0.0:1788619927.695945:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 643 00000100:00100000:0.0:1788619927.695955:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5027480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571633536:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619927.731115:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5025880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571633408:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 36419us (36664us total) trans 0 rc 544768/544768 00000100:00100000:3.0:1788619927.731130:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 842 00002000:00100000:3.0:1788619927.731142:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5025880 x1875503571633408/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:23/0 lens 488/440 e 0 to 0 dl 1788619938 ref 1 fl Complete:/600/0 rc 544768/544768 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619927.731153:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5025880 with x1875503571633408 00000100:00100000:0.0:1788619927.743995:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5027480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571633536:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 48029us (49463us total) trans 0 rc 991232/991232 00000100:00100000:0.0:1788619927.744011:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 643 00002000:00100000:0.0:1788619927.744025:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1048576->2072575) req@ffff8e77c5027480 x1875503571633536/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:23/0 lens 504/440 e 0 to 0 dl 1788619938 ref 1 fl Complete:/600/0 rc 991232/991232 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619927.744037:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5027480 with x1875503571633536 ext(1048576->2072575) 00002000:00010000:0.0:1788619927.744044:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5027480 00000100:00100000:2.0:1788619927.753799:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619927.756634:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571633792 00000100:00100000:0.0:1788619927.756649:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571633792 00000100:00100000:0.0:1788619927.757548:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1373 00000100:00100000:0.0:1788619927.757559:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5024380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571633792:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:0.0:1788619927.757630:0:5846:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619927.757659:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72800/0x1cd7fe53687360e6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d27a expref: 9 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.758820:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619927.758826:0:5846:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788619927.759103:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5e00/0x1cd7fe5368736117 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.759123:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70e00/0x1cd7fe53687360ae lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d249 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.759131:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5e00/0x1cd7fe5368736117 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.759169:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70e00/0x1cd7fe53687360ae lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d249 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.759183:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70e00/0x1cd7fe53687360ae lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d249 expref: 13 pid: 5844 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619927.759197:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be5180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589213952:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619927.759441:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590000 going to sleep for 11 seconds 00000100:00100000:0.0:1788619927.766786:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be5180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589213952:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619927.766820:0:5846:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5e00/0x1cd7fe5368736117 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619927.773313:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619927.773585:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571633920 00000100:00100000:2.0:1788619927.773605:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571633920 00010000:00010000:2.0:1788619927.773647:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70e00/0x1cd7fe53687360ae lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d249 expref: 14 pid: 5844 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619927.773698:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 454 00000100:00100000:2.0:1788619927.773707:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dc8a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571633920:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619927.773723:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619927.773730:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70e00/0x1cd7fe53687360ae lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d249 expref: 14 pid: 5844 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.773744:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70e00/0x1cd7fe53687360ae lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d249 expref: 14 pid: 5844 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.773766:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70e00/0x1cd7fe53687360ae lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d249 expref: 14 pid: 5844 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.773782:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619927.773788:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5e00/0x1cd7fe5368736117 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.773797:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5e00/0x1cd7fe5368736117 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.773806:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5e00/0x1cd7fe5368736117 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.773841:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619927.773897:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dc8a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571633920:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 191us (594us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619927.773904:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 454 00010000:00010000:1.0:1788619927.773979:0:5846:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5e00/0x1cd7fe5368736117 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.773988:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5e00/0x1cd7fe5368736117 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788619927.774100:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967782 00000004:00010000:1.0:1788619927.774147:0:5846:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 541014 -> 541014 00000004:00010000:1.0:1788619927.774150:0:5846:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619927 -> 1788619927 00000004:00010000:1.0:1788619927.774152:0:5846:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619926 -> 1788619926 00000004:00010000:1.0:1788619927.774154:0:5846:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619927 -> 1788619927 00000004:00010000:1.0:1788619927.774156:0:5846:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1800 -> 902 00000004:00010000:1.0:1788619927.774163:0:5846:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000096d2e2f2 reply state 00000000a1797754 transno 4294967783 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5e00/0x1cd7fe5368736117 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619927.774172:0:5846:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788619927.774238:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5024380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571633792:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 16681us (20449us total) trans 4294967783 rc 0/0 00000100:00100000:1.0:1788619927.774246:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1373 00000020:00080000:1.0:1788619927.774249:0:5846:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1788619927.777784:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619927.777940:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571634048 00000100:00100000:1.0:1788619927.777951:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571634048 00000100:00100000:1.0:1788619927.778006:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1374 00000100:00100000:1.0:1788619927.778016:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5024a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571634048:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619927.778043:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619927.778067:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe536873611e lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d2ab expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.778099:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297200/0x1cd7fe5368736110 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2a4 expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.778110:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe536873611e lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2ab expref: 12 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.778143:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297200/0x1cd7fe5368736110 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2a4 expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.778157:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297200/0x1cd7fe5368736110 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2a4 expref: 12 pid: 5845 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619927.778170:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c75f4000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589214080:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619927.778248:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc240 going to sleep for 11 seconds 00010000:00010000:0.0:1788619927.778367:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5e00/0x1cd7fe5368736117 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.778376:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5e00/0x1cd7fe5368736117 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.778384:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5e00/0x1cd7fe5368736117 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.778389:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c64d5e00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788619927.778403:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5e00/0x1cd7fe5368736117 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.778412:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5e00/0x1cd7fe5368736117 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.778428:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619927.778432:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788619927.778436:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5e00/0x1cd7fe5368736117 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.778443:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5e00/0x1cd7fe5368736117 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619927.782379:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c75f4000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589214080:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619927.782424:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe536873611e lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2ab expref: 11 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.782444:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619927.782447:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619927.782451:0:5844:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe536873611e lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2ab expref: 11 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.782461:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297600, rc 0) 00000100:00100000:1.0:1788619927.782511:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5024a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571634048:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4496us (4735us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619927.782521:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1374 00000100:00100000:3.0:1788619927.784198:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619927.784772:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571634176 00000100:00100000:1.0:1788619927.784782:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571634176 00010000:00010000:1.0:1788619927.784820:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297200/0x1cd7fe5368736110 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2a4 expref: 13 pid: 5845 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619927.785060:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 455 00000100:00100000:1.0:1788619927.785073:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dc9c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571634176:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619927.785088:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619927.785112:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297200/0x1cd7fe5368736110 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2a4 expref: 13 pid: 5845 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.785126:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297200/0x1cd7fe5368736110 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2a4 expref: 13 pid: 5845 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.785147:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297200/0x1cd7fe5368736110 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2a4 expref: 13 pid: 5845 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.785160:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619927.785166:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe536873611e lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2ab expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.785178:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe536873611e lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2ab expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.785188:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe536873611e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2ab expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.785223:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe536873611e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2ab expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619927.785236:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c75f4000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589214208:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619927.785365:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc240 going to sleep for 11 seconds 00000100:00100000:1.0:1788619927.788111:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c75f4000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589214208:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619927.788138:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619927.788196:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dc9c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2355:x1875503571634176:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3124us (4006us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619927.788208:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 455 00000100:00100000:3.0:1788619927.788472:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619927.788549:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571634304 00000100:00100000:3.0:1788619927.788556:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571634304 00000100:00100000:3.0:1788619927.788905:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1125 00000100:00100000:3.0:1788619927.788915:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dcad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+8:14908:x1875503571634304:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619927.788943:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619927.788961:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7d29fbb/0x1cd7fe5368736125 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d2b2 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619927.788993:0:7654:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000dbe89f01/0x1cd7fe53687360d8 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1605632->1638399) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d273 expref: 5 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619927.789002:0:7654:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000003086a8b7/0x1cd7fe53687360f4 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d288 expref: 9 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619927.789009:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(1 12) 00010000:00010000:3.0:1788619927.789013:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(1 12) 00010000:00010000:3.0:1788619927.789018:0:7654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7d29fbb/0x1cd7fe5368736125 lrc: 5/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2b2 expref: 9 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619927.789051:0:7654:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000003086a8b7/0x1cd7fe53687360f4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d288 expref: 9 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619927.789064:0:7654:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000003086a8b7/0x1cd7fe53687360f4 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d288 expref: 9 pid: 6698 timeout: 520 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619927.789077:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5dc8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589214336:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619927.789157:0:7654:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000dbe89f01/0x1cd7fe53687360d8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1605632->1638399) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d273 expref: 5 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619927.789167:0:7654:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000dbe89f01/0x1cd7fe53687360d8 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1605632->1638399) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d273 expref: 5 pid: 6696 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619927.789177:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63fb100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589214464:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619927.789200:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619927.791949:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5dc8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589214336:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619927.791965:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63fb100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589214464:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619927.791995:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c7d29fbb/0x1cd7fe5368736125 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2b2 expref: 9 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.792015:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f71400, rc 0) 00000100:00100000:0.0:1788619927.792073:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dcad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571634304:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3160us (3602us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619927.792082:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1125 00000100:00100000:2.0:1788619927.796751:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619927.796903:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571634432 00000100:00100000:3.0:1788619927.796914:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571634432 00010000:00010000:3.0:1788619927.796960:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000003086a8b7/0x1cd7fe53687360f4 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d288 expref: 9 pid: 6698 timeout: 520 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619927.797005:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 456 00000100:00100000:3.0:1788619927.797016:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5024700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571634432:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619927.797030:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619927.797052:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000003086a8b7/0x1cd7fe53687360f4 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d288 expref: 9 pid: 6698 timeout: 520 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619927.797066:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000003086a8b7/0x1cd7fe53687360f4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d288 expref: 9 pid: 6698 timeout: 520 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619927.797088:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003086a8b7/0x1cd7fe53687360f4 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d288 expref: 9 pid: 6698 timeout: 520 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619927.797112:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619927.797169:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5024700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+8:2354:x1875503571634432:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 154us (428us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619927.797176:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 456 00000100:00100000:3.0:1788619927.801995:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619927.802087:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571634560 00000100:00100000:0.0:1788619927.802096:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571634560 00010000:00010000:0.0:1788619927.802133:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000dbe89f01/0x1cd7fe53687360d8 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1605632->1638399) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d273 expref: 6 pid: 6696 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619927.802177:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 457 00000100:00100000:0.0:1788619927.802186:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63f8700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571634560:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619927.802201:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619927.802223:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000dbe89f01/0x1cd7fe53687360d8 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1605632->1638399) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d273 expref: 6 pid: 6696 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.802243:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000dbe89f01/0x1cd7fe53687360d8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1605632->1638399) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d273 expref: 6 pid: 6696 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.802276:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dbe89f01/0x1cd7fe53687360d8 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1605632->1638399) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d273 expref: 6 pid: 6696 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.802293:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(1 12) 00010000:00010000:0.0:1788619927.802297:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(1 12) 00010000:00010000:0.0:1788619927.802304:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000c7d29fbb/0x1cd7fe5368736125 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2b2 expref: 7 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.802315:0:5829:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7d29fbb/0x1cd7fe5368736125 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2b2 expref: 7 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.802390:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000c7d29fbb/0x1cd7fe5368736125 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2b2 expref: 7 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619927.802405:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be5180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1875503589214592:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619927.802491:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590000 going to sleep for 11 seconds 00000100:00100000:1.0:1788619927.805942:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619927.805981:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619927.806082:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be5180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1875503589214592:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619927.806109:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619927.806170:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63f8700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571634560:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3984us (4184us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619927.806180:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 457 00000100:00100000:2.0:1788619927.806274:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571634688 00000100:00100000:2.0:1788619927.806281:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571634688 00002000:00010000:2.0:1788619927.806318:0:5859:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:2.0:1788619927.806323:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c75f7800 with x1875503571634688 00000100:00100000:2.0:1788619927.806377:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 843 00000100:00100000:2.0:1788619927.806385:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571634688:12345-192.168.201.5@tcp:10:fsx.0 00000004:00010000:2.0:1788619927.806568:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 541014 -> 554994 00000004:00010000:2.0:1788619927.806572:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619927 -> 1788619927 00000004:00010000:2.0:1788619927.806574:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619926 -> 1788619926 00000004:00010000:2.0:1788619927.806576:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619927 -> 1788619927 00000004:00010000:2.0:1788619927.806579:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 902 -> 902 00000100:00100000:2.0:1788619927.806622:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571634688:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 238us (640us total) trans 4294967784 rc 0/0 00000100:00100000:2.0:1788619927.806629:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 843 00002000:00010000:2.0:1788619927.806632:0:5859:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:2.0:1788619927.806634:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c75f7800 with x1875503571634688 00000100:00100000:3.0:1788619927.806874:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571634816 00000100:00100000:3.0:1788619927.806879:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571634816 00002000:00010000:3.0:1788619927.806900:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:3.0:1788619927.806904:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c75f4000 with x1875503571634816 ext(0->18446744073709551615) 00002000:00010000:3.0:1788619927.806912:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c75f4000. 00000100:00100000:3.0:1788619927.806938:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 644 00000100:00100000:3.0:1788619927.806945:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+8:2355:x1875503571634816:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:3.0:1788619927.807163:0:6702:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 2072189 -> 0 00002000:00010000:3.0:1788619927.807167:0:6702:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619927 -> 1788619927 00002000:00010000:3.0:1788619927.807169:0:6702:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619926 -> 1788619926 00002000:00010000:3.0:1788619927.807171:0:6702:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619927 -> 1788619927 00002000:00010000:3.0:1788619927.807173:0:6702:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 391 -> 135 00000100:00100000:3.0:1788619927.807203:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571634816:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 257us (1270us total) trans 4294967489 rc 0/0 00000100:00100000:3.0:1788619927.807210:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 644 00002000:00010000:3.0:1788619927.807213:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:3.0:1788619927.807216:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c75f4000 with x1875503571634816 ext(0->18446744073709551615) 00002000:00010000:3.0:1788619927.807220:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c75f4000. 00000100:00100000:3.0:1788619927.812250:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619927.812395:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571634944 00000100:00100000:1.0:1788619927.812405:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571634944 00000100:00100000:1.0:1788619927.812469:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1375 00000100:00100000:1.0:1788619927.812480:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63f9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+12:14908:x1875503571634944:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619927.812500:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619927.812521:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296600/0x1cd7fe536873612c lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d2b9 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.812620:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296200/0x1cd7fe5368736133 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.812635:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296200/0x1cd7fe5368736133 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.812651:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296200/0x1cd7fe5368736133 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788619927.812691:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 554994 00000004:00200000:1.0:1788619927.812696:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788619927.812715:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296200/0x1cd7fe5368736133 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.812741:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296600/0x1cd7fe536873612c lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2b9 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.812760:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296200/0x1cd7fe5368736133 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2b9 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.812768:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619927.812771:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788619927.812774:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296200, rc 0) 00000100:00100000:1.0:1788619927.812833:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63f9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571634944:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 355us (596us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619927.812842:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1375 00000100:00100000:1.0:1788619927.815864:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619927.815898:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619927.815985:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571635072 00000100:00100000:2.0:1788619927.815995:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571635072 00000100:00100000:2.0:1788619927.816055:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1376 00000100:00100000:2.0:1788619927.816065:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571635072:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619927.816084:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619927.816103:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7800/0x1cd7fe536873613a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d2c0 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.816141:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe536873611e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2ab expref: 11 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619927.816182:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5025180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589214720:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619927.816269:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81646c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619927.816513:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571635200 00000100:00100000:3.0:1788619927.816519:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571635200 00000100:00100000:3.0:1788619927.816561:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1126 00000100:00100000:3.0:1788619927.816568:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571635200:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619927.816578:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619927.816588:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fe91493a/0x1cd7fe5368736141 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d2c7 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619927.816636:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63fbb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589214848:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619927.816687:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebcc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619927.822132:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63fbb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589214848:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619927.822173:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000fe91493a/0x1cd7fe5368736141 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2c7 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619927.822199:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fe91493a/0x1cd7fe5368736141 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2c7 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619927.822213:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f71800, rc 0) 00000100:00100000:3.0:1788619927.822649:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571635200:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6078us (6746us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619927.822658:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1126 00000100:00100000:2.0:1788619927.822920:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5025180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589214720:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619927.822937:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 554994 00010000:00010000:2.0:1788619927.822952:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7800/0x1cd7fe536873613a lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2c0 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.822966:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619927.822970:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619927.822975:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7800/0x1cd7fe536873613a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2c0 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.822986:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d7800, rc 0) 00000100:00100000:2.0:1788619927.823014:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571635072:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6952us (7160us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619927.823021:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1376 00000100:00100000:0.0:1788619927.833183:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619927.833224:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619927.833420:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571635328 00000100:00100000:1.0:1788619927.833430:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571635328 00000100:00100000:1.0:1788619927.833494:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1377 00000100:00100000:1.0:1788619927.833504:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571635328:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619927.833525:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619927.833545:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296600/0x1cd7fe5368736148 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d2ce expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.833584:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe536873611e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2ab expref: 11 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619927.833639:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c75f4a80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589214976:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619927.833722:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc240 going to sleep for 11 seconds 00000100:00100000:0.0:1788619927.836146:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c75f4a80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589214976:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619927.836173:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 554994 00010000:00010000:0.0:1788619927.836192:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296600/0x1cd7fe5368736148 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2ce expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.836214:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619927.836217:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619927.836222:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296600/0x1cd7fe5368736148 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2ce expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.836238:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0296600, rc 0) 00000100:00100000:0.0:1788619927.836314:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571635328:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2810us (3140us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619927.836322:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1377 00000100:00100000:2.0:1788619927.837020:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571635456 00000100:00100000:2.0:1788619927.837030:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571635456 00000100:00100000:2.0:1788619927.837083:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1127 00000100:00100000:2.0:1788619927.837093:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571635456:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619927.837107:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619927.837123:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8dcc7dd/0x1cd7fe536873614f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d2d5 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619927.837175:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5025180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589215104:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619927.837373:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81646c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619927.847375:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5025180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589215104:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619927.847417:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b8dcc7dd/0x1cd7fe536873614f lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2d5 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.847443:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8dcc7dd/0x1cd7fe536873614f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2d5 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.847457:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d7800, rc 0) 00000100:00100000:2.0:1788619927.847529:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571635456:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 10436us (14302us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619927.847538:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1127 00000100:00100000:2.0:1788619927.852668:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619927.852759:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571635712 00000100:00100000:1.0:1788619927.852770:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571635712 00000100:00100000:1.0:1788619927.852832:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1378 00000100:00100000:1.0:1788619927.852843:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5025180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+12:14908:x1875503571635712:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:1.0:1788619927.853125:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296e00/0x1cd7fe5368736156 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.853143:0:11397:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296200/0x1cd7fe5368736133 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2b9 expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.853152:0:11397:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296e00/0x1cd7fe5368736156 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.853185:0:11397:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296200/0x1cd7fe5368736133 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2b9 expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.853200:0:11397:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296200/0x1cd7fe5368736133 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2b9 expref: 12 pid: 5845 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619927.853215:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c75f4700 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589215232:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619927.854604:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccd80 going to sleep for 11 seconds 00000100:00100000:1.0:1788619927.862882:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619927.863005:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c75f4700 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589215232:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619927.863032:0:11397:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296e00/0x1cd7fe5368736156 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619927.863979:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571635840 00000100:00100000:3.0:1788619927.863988:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571635840 00010000:00010000:3.0:1788619927.864032:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296200/0x1cd7fe5368736133 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2b9 expref: 13 pid: 5845 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619927.864141:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 458 00000100:00100000:3.0:1788619927.864152:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f4e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571635840:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619927.864169:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619927.864178:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296200/0x1cd7fe5368736133 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2b9 expref: 13 pid: 5845 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619927.864189:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296200/0x1cd7fe5368736133 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2b9 expref: 13 pid: 5845 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619927.864209:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296200/0x1cd7fe5368736133 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2b9 expref: 13 pid: 5845 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619927.864224:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619927.864229:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296e00/0x1cd7fe5368736156 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619927.864238:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296e00/0x1cd7fe5368736156 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619927.864245:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296e00/0x1cd7fe5368736156 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619927.866037:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619927.866111:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f4e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571635840:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 1955us (3233us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619927.866122:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 458 00010000:00010000:1.0:1788619927.868151:0:11397:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296e00/0x1cd7fe5368736156 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.868167:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296e00/0x1cd7fe5368736156 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788619927.868295:0:11397:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967785 00000004:00010000:1.0:1788619927.868375:0:11397:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 554994 -> 554994 00000004:00010000:1.0:1788619927.868379:0:11397:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619927 -> 1788619927 00000004:00010000:1.0:1788619927.868381:0:11397:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619926 -> 1788619926 00000004:00010000:1.0:1788619927.868384:0:11397:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619927 -> 1788619927 00000004:00010000:1.0:1788619927.868386:0:11397:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 902 -> 902 00000004:00010000:1.0:1788619927.868392:0:11397:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000005d292469 reply state 00000000089c6b44 transno 4294967786 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296e00/0x1cd7fe5368736156 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619927.868400:0:11397:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788619927.868498:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5025180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+15:14908:x1875503571635712:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 15654us (15837us total) trans 4294967786 rc 0/0 00000100:00100000:1.0:1788619927.868507:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1378 00000020:00080000:1.0:1788619927.868511:0:11397:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788619927.874670:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296e00/0x1cd7fe5368736156 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.874689:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296e00/0x1cd7fe5368736156 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.874696:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296e00/0x1cd7fe5368736156 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.874702:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78e0296e00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788619927.874708:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296e00/0x1cd7fe5368736156 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.874715:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296e00/0x1cd7fe5368736156 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.874731:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619927.874734:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788619927.874738:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296e00/0x1cd7fe5368736156 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.874744:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296e00/0x1cd7fe5368736156 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619927.876025:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619927.876107:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619927.876177:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571635968 00000100:00100000:3.0:1788619927.876187:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571635968 00002000:00010000:3.0:1788619927.876232:0:5857:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:3.0:1788619927.876237:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5024e00 with x1875503571635968 00000100:00100000:3.0:1788619927.876282:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 844 00000100:00100000:3.0:1788619927.876293:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5024e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+14:2355:x1875503571635968:12345-192.168.201.5@tcp:10:fsx.0 00000004:00010000:3.0:1788619927.876609:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 554994 -> 466346 00000004:00010000:3.0:1788619927.876613:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619927 -> 1788619927 00000004:00010000:3.0:1788619927.876616:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619926 -> 1788619926 00000004:00010000:3.0:1788619927.876618:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619927 -> 1788619927 00000004:00010000:3.0:1788619927.876621:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 902 -> 902 00000100:00100000:3.0:1788619927.876687:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5024e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+15:2355:x1875503571635968:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 395us (671us total) trans 4294967787 rc 0/0 00000100:00100000:3.0:1788619927.876696:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 844 00002000:00010000:3.0:1788619927.876700:0:5857:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:3.0:1788619927.876702:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5024e00 with x1875503571635968 00000100:00100000:0.0:1788619927.877109:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571636096 00000100:00100000:0.0:1788619927.877115:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571636096 00002000:00010000:0.0:1788619927.877137:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:0.0:1788619927.877141:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5026300 with x1875503571636096 ext(0->18446744073709551615) 00002000:00010000:0.0:1788619927.877149:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5026300. 00000100:00100000:0.0:1788619927.877174:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 645 00000100:00100000:0.0:1788619927.877182:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5026300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571636096:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:0.0:1788619927.877356:0:6703:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:0.0:1788619927.877360:0:6703:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619927 -> 1788619927 00002000:00010000:0.0:1788619927.877362:0:6703:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619926 -> 1788619926 00002000:00010000:0.0:1788619927.877364:0:6703:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619927 -> 1788619927 00002000:00010000:0.0:1788619927.877366:0:6703:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 135 -> 135 00000100:00100000:0.0:1788619927.877393:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5026300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571636096:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 213us (1286us total) trans 4294967490 rc 0/0 00000100:00100000:0.0:1788619927.877400:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 645 00002000:00010000:0.0:1788619927.877403:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:0.0:1788619927.877406:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5026300 with x1875503571636096 ext(0->18446744073709551615) 00002000:00010000:0.0:1788619927.877410:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5026300. 00000100:00100000:2.0:1788619927.882000:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619927.886835:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571636224 00000100:00100000:1.0:1788619927.886854:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571636224 00000100:00100000:1.0:1788619927.886928:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1379 00000100:00100000:1.0:1788619927.886938:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e647b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+15:14908:x1875503571636224:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619927.886962:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619927.886983:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296a00/0x1cd7fe536873615d lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d2dc expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.887074:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe5368736164 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.887089:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe5368736164 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.887100:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe5368736164 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788619927.887128:0:5846:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 466346 00000004:00200000:1.0:1788619927.887132:0:5846:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788619927.887151:0:5846:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe5368736164 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.887177:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296a00/0x1cd7fe536873615d lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2dc expref: 17 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.887197:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe5368736164 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2dc expref: 16 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619927.887206:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619927.887209:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788619927.887213:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e0297000, rc 0) 00000100:00100000:1.0:1788619927.887310:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e647b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+16:14908:x1875503571636224:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 373us (5325us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619927.887317:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1379 00000100:00100000:1.0:1788619927.904480:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619927.904634:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571636352 00000100:00100000:0.0:1788619927.904644:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571636352 00002000:00100000:0.0:1788619927.904700:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c75f5180 x1875503571636352/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:23/0 lens 488/0 e 0 to 0 dl 1788619938 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619927.904713:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c75f5180 with x1875503571636352 00000100:00100000:0.0:1788619927.904743:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 845 00000100:00100000:0.0:1788619927.904752:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+16:2353:x1875503571636352:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619927.919050:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+16:2353:x1875503571636352:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 14284us (14566us total) trans 0 rc 466944/466944 00000100:00100000:0.0:1788619927.919066:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 845 00002000:00100000:0.0:1788619927.919079:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c75f5180 x1875503571636352/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:23/0 lens 488/440 e 0 to 0 dl 1788619938 ref 1 fl Complete:/600/0 rc 466944/466944 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619927.919090:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c75f5180 with x1875503571636352 00000100:00100000:2.0:1788619927.924962:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619927.925071:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571636480 00000100:00100000:0.0:1788619927.925081:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571636480 00000100:00100000:0.0:1788619927.925143:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1380 00000100:00100000:0.0:1788619927.925154:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e647b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:14908:x1875503571636480:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619927.925174:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619927.925193:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5e00/0x1cd7fe536873616b lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d2e3 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.925637:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5400/0x1cd7fe5368736172 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.925655:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5400/0x1cd7fe5368736172 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.925795:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5400/0x1cd7fe5368736172 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619927.925826:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 466346 00000004:00200000:0.0:1788619927.925831:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619927.925851:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5400/0x1cd7fe5368736172 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.925882:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5e00/0x1cd7fe536873616b lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2e3 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.925901:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5400/0x1cd7fe5368736172 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2e3 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.925910:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619927.925913:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619927.925917:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d5400, rc 0) 00000100:00100000:0.0:1788619927.925995:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e647b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571636480:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 842us (1043us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619927.926003:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1380 00000100:00100000:1.0:1788619927.935697:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619927.935752:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619927.935849:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571636608 00000100:00100000:3.0:1788619927.935865:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571636608 00000100:00100000:2.0:1788619927.935872:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571636736 00000100:00100000:2.0:1788619927.935884:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571636736 00000100:00100000:2.0:1788619927.935957:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1128 00000100:00100000:2.0:1788619927.935972:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571636736:12345-192.168.201.5@tcp:101:fsx.0 00000100:00100000:3.0:1788619927.935978:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1381 00000100:00100000:3.0:1788619927.935999:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571636608:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619927.936002:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619927.936021:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e4266ce/0x1cd7fe5368736179 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d2f1 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619927.936023:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619927.936050:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71800/0x1cd7fe5368736180 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d2ea expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619927.936088:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e6478700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589215360:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619927.936099:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe536873611e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2ab expref: 15 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619927.936160:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63fbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589215488:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619927.936188:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164240 going to sleep for 11 seconds 00000100:00100000:3.0:1788619927.936230:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebcc0 going to sleep for 11 seconds 00000004:00010000:3.0:1788619927.938887:0:11397:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619926 -> 1788619927 00000100:00100000:3.0:1788619927.938934:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63fbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589215488:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619927.938964:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 466346 00010000:00010000:3.0:1788619927.938990:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71800/0x1cd7fe5368736180 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2ea expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619927.939015:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619927.939019:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619927.939026:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71800/0x1cd7fe5368736180 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2ea expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619927.939043:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f71800, rc 0) 00002000:00010000:2.0:1788619927.939306:0:6697:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619926 -> 1788619927 00000100:00100000:2.0:1788619927.939360:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e6478700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589215360:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619927.939397:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003e4266ce/0x1cd7fe5368736179 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2f1 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.939423:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e4266ce/0x1cd7fe5368736179 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2f1 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.939436:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d6200, rc 0) 00000100:00100000:3.0:1788619927.939575:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571636608:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3578us (3818us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619927.939587:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1381 00000100:00100000:2.0:1788619927.944557:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571636736:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8575us (8854us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619927.944573:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1128 00000100:00100000:0.0:1788619927.954436:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619927.954540:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571636864 00000100:00100000:3.0:1788619927.954553:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571636864 00000100:00100000:3.0:1788619927.954657:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1382 00000100:00100000:3.0:1788619927.954671:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571636864:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619927.954697:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619927.954725:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71800/0x1cd7fe5368736187 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d2f8 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619927.954775:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe536873611e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2ab expref: 15 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619927.954835:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63fbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589215616:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619927.954955:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebcc0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619927.959488:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619927.959603:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571636992 00000100:00100000:0.0:1788619927.959617:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571636992 00000100:00100000:0.0:1788619927.959697:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1129 00000100:00100000:0.0:1788619927.959711:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571636992:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619927.959736:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619927.959761:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000010a73d3/0x1cd7fe536873618e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d2ff expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619927.959845:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589215744:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619927.959982:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590000 going to sleep for 11 seconds 00000100:00100000:3.0:1788619927.962160:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63fbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589215616:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619927.962195:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 466346 00000100:00100000:0.0:1788619927.962221:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589215744:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619927.962222:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71800/0x1cd7fe5368736187 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2f8 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619927.962270:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619927.962274:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000010a73d3/0x1cd7fe536873618e lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2ff expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619927.962277:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619927.962293:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71800/0x1cd7fe5368736187 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2f8 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619927.962309:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000010a73d3/0x1cd7fe536873618e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2ff expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619927.962312:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f71800, rc 0) 00000100:00100000:3.0:1788619927.962494:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571636864:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7823us (8078us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619927.962505:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1382 00010000:00010000:0.0:1788619927.963018:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d4000, rc 0) 00000100:00100000:0.0:1788619927.963095:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571636992:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3386us (3623us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619927.963106:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1129 00000100:00100000:1.0:1788619927.971139:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619927.971262:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571637120 00000100:00100000:2.0:1788619927.971275:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571637120 00000100:00100000:2.0:1788619927.971410:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 846 00000100:00100000:2.0:1788619927.971423:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+16:2355:x1875503571637120:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:2.0:1788619927.971790:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 466346 -> 466346 00000004:00010000:2.0:1788619927.971795:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619927 -> 1788619927 00000004:00010000:2.0:1788619927.971797:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619927 -> 1788619927 00000004:00010000:2.0:1788619927.971800:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619927 -> 1788619927 00000004:00010000:2.0:1788619927.971803:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 902 -> 902 00000100:00100000:2.0:1788619927.971887:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+17:2355:x1875503571637120:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 466us (762us total) trans 4294967788 rc 0/0 00000100:00100000:2.0:1788619927.971896:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 846 00000100:00100000:1.0:1788619927.979157:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619927.979208:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619927.979302:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571637376 00000100:00100000:3.0:1788619927.979315:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571637376 00000100:00100000:3.0:1788619927.979425:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1130 00000100:00100000:3.0:1788619927.979435:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571637376:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619927.979458:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619927.979480:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fe91493a/0x1cd7fe5368736195 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d30d expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619927.979559:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63fbb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589215872:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619927.979678:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebcc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619927.980115:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571637248 00000100:00100000:2.0:1788619927.980122:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571637248 00000100:00100000:2.0:1788619927.980172:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1383 00000100:00100000:2.0:1788619927.980181:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571637248:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619927.980195:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619927.980213:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7e00/0x1cd7fe536873619c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d306 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.980247:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe536873611e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2ab expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619927.980276:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e6478700 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589216000:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619927.980323:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164240 going to sleep for 11 seconds 00000100:00100000:3.0:1788619927.983115:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63fbb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589215872:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619927.983169:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000fe91493a/0x1cd7fe5368736195 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d30d expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619927.983203:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fe91493a/0x1cd7fe5368736195 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d30d expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619927.983220:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f71800, rc 0) 00000100:00100000:3.0:1788619927.983305:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571637376:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3870us (4162us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619927.983315:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1130 00000100:00100000:2.0:1788619927.985197:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e6478700 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589216000:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619927.985235:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 466346 00010000:00010000:2.0:1788619927.985262:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7e00/0x1cd7fe536873619c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d306 expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.985290:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619927.985294:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619927.985301:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7e00/0x1cd7fe536873619c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d306 expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.985319:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d7e00, rc 0) 00000100:00100000:2.0:1788619927.985467:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571637248:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5285us (6257us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619927.985477:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1383 00000100:00100000:1.0:1788619927.994517:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619927.994578:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619927.994726:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571637632 00000100:00100000:2.0:1788619927.994739:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571637632 00000100:00100000:2.0:1788619927.994817:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1131 00000100:00100000:2.0:1788619927.994829:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571637632:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619927.994857:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619927.994885:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002a635b6d/0x1cd7fe53687361a3 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d31b expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619927.994979:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e6478700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589216128:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619927.995097:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164240 going to sleep for 11 seconds 00000100:00100000:3.0:1788619927.995219:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571637504 00000100:00100000:3.0:1788619927.995224:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571637504 00000100:00100000:3.0:1788619927.995264:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1384 00000100:00100000:3.0:1788619927.995271:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571637504:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619927.995283:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619927.995299:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71800/0x1cd7fe53687361aa lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d314 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619927.995471:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe536873611e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2ab expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619927.995502:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63fbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589216256:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619927.995538:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebcc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619927.998895:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e6478700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589216128:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619927.998946:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002a635b6d/0x1cd7fe53687361a3 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d31b expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.998979:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002a635b6d/0x1cd7fe53687361a3 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d31b expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619927.998997:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d7e00, rc 0) 00000100:00100000:2.0:1788619927.999080:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571637632:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4252us (4580us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619927.999092:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1131 00000100:00100000:3.0:1788619927.999552:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63fbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589216256:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619927.999580:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 466346 00010000:00010000:3.0:1788619927.999611:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71800/0x1cd7fe53687361aa lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d314 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619927.999634:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619927.999637:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619927.999644:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71800/0x1cd7fe53687361aa lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d314 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619927.999659:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f71800, rc 0) 00000100:00100000:2.0:1788619927.999792:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571637504:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4520us (5211us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619927.999801:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1384 00000100:00100000:1.0:1788619928.004700:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.004895:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571637888 00000100:00100000:3.0:1788619928.004909:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571637888 00000100:00100000:3.0:1788619928.005007:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1385 00000100:00100000:3.0:1788619928.005021:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+17:14908:x1875503571637888:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:3.0:1788619928.005101:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619928.005136:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297000/0x1cd7fe5368736164 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2dc expref: 17 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.005162:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619928.005169:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788619928.005640:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70000/0x1cd7fe53687361b1 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.005662:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5400/0x1cd7fe5368736172 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2e3 expref: 12 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.005673:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70000/0x1cd7fe53687361b1 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.005718:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5400/0x1cd7fe5368736172 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2e3 expref: 12 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.005739:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5400/0x1cd7fe5368736172 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2e3 expref: 12 pid: 5844 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619928.005757:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63fbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589216384:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619928.005996:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebc00 going to sleep for 11 seconds 00000100:00080000:2.0:1788619928.006080:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1788619928.010109:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619928.010199:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571638016 00000100:00100000:0.0:1788619928.010212:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571638016 00010000:00010000:0.0:1788619928.010287:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5400/0x1cd7fe5368736172 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2e3 expref: 13 pid: 5844 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619928.010371:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 459 00000100:00100000:0.0:1788619928.010393:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcae450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571638016:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619928.010412:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619928.010419:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5400/0x1cd7fe5368736172 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2e3 expref: 13 pid: 5844 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.010440:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5400/0x1cd7fe5368736172 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2e3 expref: 13 pid: 5844 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.010469:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619928.010476:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70000/0x1cd7fe53687361b1 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.010489:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70000/0x1cd7fe53687361b1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.010498:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70000/0x1cd7fe53687361b1 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.010523:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619928.010586:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcae450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571638016:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 196us (494us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619928.010606:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 459 00000100:00080000:2.0:1788619928.010733:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:3.0:1788619928.010840:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5400/0x1cd7fe5368736172 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2e3 expref: 12 pid: 5844 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619928.010861:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63fbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589216384:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619928.010886:0:5845:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70000/0x1cd7fe53687361b1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.010897:0:5845:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70000/0x1cd7fe53687361b1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.010905:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70000/0x1cd7fe53687361b1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788619928.011062:0:5845:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967789 00000004:00010000:3.0:1788619928.011125:0:5845:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 466346 -> 466346 00000004:00010000:3.0:1788619928.011129:0:5845:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619927 -> 1788619927 00000004:00010000:3.0:1788619928.011132:0:5845:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619927 -> 1788619927 00000004:00010000:3.0:1788619928.011135:0:5845:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619927 -> 1788619927 00000004:00010000:3.0:1788619928.011137:0:5845:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 902 -> 902 00000004:00010000:3.0:1788619928.011147:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000ea80c901 reply state 00000000c95a1e38 transno 4294967790 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70000/0x1cd7fe53687361b1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619928.011157:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788619928.011249:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+19:14908:x1875503571637888:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 6231us (6574us total) trans 4294967790 rc 0/0 00000100:00100000:3.0:1788619928.011257:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1385 00000020:00080000:3.0:1788619928.011262:0:5845:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:2.0:1788619928.013444:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70000/0x1cd7fe53687361b1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.013473:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70000/0x1cd7fe53687361b1 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.013482:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70000/0x1cd7fe53687361b1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.013489:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c5f70000 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:2.0:1788619928.013495:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70000/0x1cd7fe53687361b1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.013504:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70000/0x1cd7fe53687361b1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.013525:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619928.013529:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788619928.013534:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70000/0x1cd7fe53687361b1 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.013541:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70000/0x1cd7fe53687361b1 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619928.015747:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.015888:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571638144 00000100:00100000:3.0:1788619928.015957:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571638144 00002000:00010000:3.0:1788619928.016017:0:6701:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (2013748->18446744073709551615) 00002000:00100000:3.0:1788619928.016023:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c75f4e00 with x1875503571638144 ext(2013748->18446744073709551615) 00002000:00010000:3.0:1788619928.016036:0:6701:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c75f4e00. 00000100:00100000:3.0:1788619928.016070:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 646 00000100:00100000:3.0:1788619928.016081:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571638144:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:3.0:1788619928.016305:0:6701:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 0 -> 2013748 00002000:00010000:3.0:1788619928.016309:0:6701:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619927 -> 1788619927 00002000:00010000:3.0:1788619928.016312:0:6701:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619927 -> 1788619926 00002000:00010000:3.0:1788619928.016314:0:6701:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619927 -> 1788619927 00002000:00010000:3.0:1788619928.016316:0:6701:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 135 -> 135 00000100:00080000:2.0:1788619928.016713:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1788619928.016883:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571638144:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 802us (1148us total) trans 4294967491 rc 0/0 00000100:00100000:3.0:1788619928.016893:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 646 00002000:00010000:3.0:1788619928.016898:0:6701:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (2013748->18446744073709551615) 00002000:00100000:3.0:1788619928.016901:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c75f4e00 with x1875503571638144 ext(2013748->18446744073709551615) 00002000:00010000:3.0:1788619928.016906:0:6701:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c75f4e00. 00000100:00100000:0.0:1788619928.020822:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619928.020953:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571638272 00000100:00100000:2.0:1788619928.020962:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571638272 00000100:00100000:2.0:1788619928.021042:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1386 00000100:00100000:2.0:1788619928.021051:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+18:14908:x1875503571638272:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619928.021072:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619928.021090:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7e00/0x1cd7fe53687361b8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d322 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.021173:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7c00/0x1cd7fe53687361bf lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.021187:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7c00/0x1cd7fe53687361bf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.021198:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7c00/0x1cd7fe53687361bf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788619928.021222:0:9417:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 2013748 00000004:00200000:2.0:1788619928.021226:0:9417:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788619928.021241:0:9417:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7c00/0x1cd7fe53687361bf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.021265:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7e00/0x1cd7fe53687361b8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d322 expref: 20 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.021282:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7c00/0x1cd7fe53687361bf lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d322 expref: 19 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.021291:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619928.021294:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788619928.021297:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d7c00, rc 0) 00000100:00100000:2.0:1788619928.021848:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+19:14908:x1875503571638272:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 794us (1030us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619928.021927:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1386 00000100:00100000:1.0:1788619928.032958:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619928.032997:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619928.033086:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571638528 00000100:00100000:0.0:1788619928.033095:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571638528 00002000:00100000:0.0:1788619928.033142:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fe002d80 x1875503571638528/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:24/0 lens 504/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619928.033153:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fe002d80 with x1875503571638528 00000100:00100000:0.0:1788619928.033192:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 847 00000100:00100000:0.0:1788619928.033200:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe002d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+19:2355:x1875503571638528:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619928.033938:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571638400 00000100:00100000:3.0:1788619928.033945:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571638400 00002000:00100000:3.0:1788619928.033975:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1048576->2015231) req@ffff8e78fe003b80 x1875503571638400/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619928.033985:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fe003b80 with x1875503571638400 ext(1048576->2015231) 00002000:00010000:3.0:1788619928.033990:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fe003b80 00000100:00100000:3.0:1788619928.034017:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 647 00000100:00100000:3.0:1788619928.034024:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe003b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571638400:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619928.045189:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe002d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+19:2355:x1875503571638528:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 11977us (12178us total) trans 0 rc 36864/36864 00000100:00100000:0.0:1788619928.045204:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 847 00002000:00100000:0.0:1788619928.045216:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fe002d80 x1875503571638528/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:24/0 lens 504/440 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 36864/36864 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619928.045227:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fe002d80 with x1875503571638528 00000100:00100000:3.0:1788619928.048868:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe003b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571638400:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 14825us (15901us total) trans 0 rc 966656/966656 00000100:00100000:3.0:1788619928.048884:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 647 00002000:00100000:3.0:1788619928.048898:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1048576->2015231) req@ffff8e78fe003b80 x1875503571638400/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:24/0 lens 488/440 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 966656/966656 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619928.048908:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fe003b80 with x1875503571638400 ext(1048576->2015231) 00002000:00010000:3.0:1788619928.048914:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fe003b80 00000100:00100000:3.0:1788619928.055263:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619928.055369:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571638656 00000100:00100000:2.0:1788619928.055380:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571638656 00000100:00100000:2.0:1788619928.055453:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1387 00000100:00100000:2.0:1788619928.055465:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63fbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571638656:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619928.055487:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619928.055510:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7e00/0x1cd7fe53687361c6 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d329 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.055551:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe536873611e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2ab expref: 18 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619928.055621:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e6478700 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589216512:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619928.055714:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81646c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619928.058563:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619928.058727:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571638784 00000100:00100000:0.0:1788619928.058738:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571638784 00000100:00100000:0.0:1788619928.058815:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1132 00000100:00100000:0.0:1788619928.058825:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe000000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571638784:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619928.058843:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619928.058877:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025706efa/0x1cd7fe53687361cd lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d330 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619928.058936:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589216640:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619928.059007:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590b40 going to sleep for 11 seconds 00000004:00010000:2.0:1788619928.059166:0:11397:0:(mdt_lvb.c:177:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size: 466346 -> 794416 00000100:00100000:2.0:1788619928.059191:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e6478700 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589216512:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619928.059209:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 794416 00010000:00010000:2.0:1788619928.059225:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7e00/0x1cd7fe53687361c6 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d329 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.059245:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619928.059248:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619928.059254:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7e00/0x1cd7fe53687361c6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d329 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.059270:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d7e00, rc 0) 00000100:00100000:2.0:1788619928.059299:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63fbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571638656:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3836us (4046us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619928.059307:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1387 00000100:00080000:3.0:1788619928.059412:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:0.0:1788619928.062223:0:6697:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619926 -> 1788619927 00000100:00100000:0.0:1788619928.062256:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589216640:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619928.062301:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000025706efa/0x1cd7fe53687361cd lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d330 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.062368:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025706efa/0x1cd7fe53687361cd lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d330 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.062382:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d5400, rc 0) 00000100:00100000:0.0:1788619928.062442:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe000000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571638784:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3619us (3888us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619928.062451:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1132 00000100:00100000:3.0:1788619928.065694:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619928.065796:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571638912 00000100:00100000:1.0:1788619928.065806:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571638912 00000100:00100000:1.0:1788619928.065870:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1388 00000100:00100000:1.0:1788619928.065881:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63f9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+12:14908:x1875503571638912:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619928.065915:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619928.065937:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe53687361d4 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d337 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619928.065969:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe536873611e lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2ab expref: 18 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619928.065978:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe53687361d4 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d337 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619928.066014:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe536873611e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2ab expref: 18 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619928.066028:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe536873611e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2ab expref: 18 pid: 5844 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619928.066042:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fe001c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589216768:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619928.066133:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccd80 going to sleep for 11 seconds 00000100:00100000:1.0:1788619928.069103:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.069218:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe001c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589216768:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619928.069254:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe53687361d4 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d337 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.069275:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619928.069279:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619928.069283:0:5845:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe53687361d4 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d337 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.069300:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079c00, rc 0) 00000100:00100000:3.0:1788619928.069384:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63f9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571638912:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3504us (3698us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619928.069393:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1388 00000100:00100000:2.0:1788619928.069728:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571639040 00000100:00100000:2.0:1788619928.069735:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571639040 00002000:00100000:2.0:1788619928.069775:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fe003800 x1875503571639040/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619928.069784:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fe003800 with x1875503571639040 00010000:00010000:2.0:1788619928.069790:0:5857:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe536873611e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2ab expref: 19 pid: 5844 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.069801:0:5857:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe536873611e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2ab expref: 19 pid: 5844 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788619928.069808:0:5857:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e78fe003800 00000100:00100000:2.0:1788619928.069837:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 35 00000100:00100000:2.0:1788619928.069846:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe003800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+19:2354:x1875503571639040:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:2.0:1788619928.073729:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 794416 -> 794416 00000004:00010000:2.0:1788619928.073738:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619927 -> 1788619927 00000004:00010000:2.0:1788619928.073740:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619927 -> 1788619927 00000004:00010000:2.0:1788619928.073742:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619927 -> 1788619927 00000004:00010000:2.0:1788619928.073744:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 902 -> 902 00000100:00100000:2.0:1788619928.073883:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe003800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+21:2354:x1875503571639040:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 4032us (4785us total) trans 4294967791 rc 0/0 00000100:00100000:2.0:1788619928.073895:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 35 00002000:00100000:2.0:1788619928.073906:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fe003800 x1875503571639040/t4294967791(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:24/0 lens 488/448 e 0 to 0 dl 1788619939 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619928.073915:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fe003800 with x1875503571639040 00010000:00010000:2.0:1788619928.073925:0:5857:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe536873611e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2ab expref: 21 pid: 5844 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.073946:0:5857:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe536873611e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2ab expref: 21 pid: 5844 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788619928.073954:0:5857:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e78fe003800 00000100:00100000:1.0:1788619928.079148:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619928.079257:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571639168 00000100:00100000:0.0:1788619928.079267:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571639168 00010000:00010000:0.0:1788619928.079314:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe536873611e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2ab expref: 21 pid: 5844 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619928.079431:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 460 00000100:00100000:0.0:1788619928.079444:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe001180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+21:2352:x1875503571639168:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619928.079466:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619928.079501:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe536873611e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2ab expref: 21 pid: 5844 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.079517:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe536873611e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2ab expref: 21 pid: 5844 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.079545:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0297600/0x1cd7fe536873611e lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2ab expref: 21 pid: 5844 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.079565:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619928.079572:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe53687361d4 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d337 expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.079586:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe53687361d4 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d337 expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.079608:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe53687361d4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d337 expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.079648:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe53687361d4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d337 expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619928.079662:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be7480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589216896:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619928.079783:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590b40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619928.083468:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619928.083905:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571639296 00000100:00100000:1.0:1788619928.083916:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571639296 00002000:00100000:1.0:1788619928.083973:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be6680 x1875503571639296/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619928.083985:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be6680 with x1875503571639296 00000100:00100000:1.0:1788619928.084023:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 848 00000100:00100000:1.0:1788619928.084033:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571639296:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619928.084385:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571639296:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 352us (926us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619928.084395:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 848 00002000:00100000:1.0:1788619928.084402:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be6680 x1875503571639296/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/4536 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619928.084411:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be6680 with x1875503571639296 00000100:00100000:3.0:1788619928.084664:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be7480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589216896:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619928.084686:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619928.084773:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe001180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+20:2352:x1875503571639168:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 5332us (5635us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619928.084782:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 460 00000100:00100000:0.0:1788619928.088106:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619928.088317:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571639424 00000100:00100000:2.0:1788619928.088358:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571639424 00002000:00100000:2.0:1788619928.088410:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be6a00 x1875503571639424/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619928.088432:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be6a00 with x1875503571639424 00000100:00100000:2.0:1788619928.088459:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 849 00000100:00100000:2.0:1788619928.088469:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571639424:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619928.088690:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571639424:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 220us (593us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619928.088697:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 849 00002000:00100000:2.0:1788619928.088704:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be6a00 x1875503571639424/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/4536 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619928.088712:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be6a00 with x1875503571639424 00000100:00100000:2.0:1788619928.092752:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619928.093091:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571639552 00000100:00100000:1.0:1788619928.093102:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571639552 00002000:00100000:1.0:1788619928.093152:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e6478700 x1875503571639552/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619928.093170:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e6478700 with x1875503571639552 00000100:00100000:1.0:1788619928.093210:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 850 00000100:00100000:1.0:1788619928.093221:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e6478700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571639552:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619928.093491:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e6478700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571639552:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 272us (747us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619928.093501:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 850 00002000:00100000:1.0:1788619928.093507:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e6478700 x1875503571639552/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/4536 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619928.093515:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e6478700 with x1875503571639552 00000100:00100000:1.0:1788619928.096887:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619928.096986:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571639680 00000100:00100000:0.0:1788619928.096996:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571639680 00002000:00100000:0.0:1788619928.097049:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fe000380 x1875503571639680/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619928.097059:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fe000380 with x1875503571639680 00000100:00100000:0.0:1788619928.097095:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 851 00000100:00100000:0.0:1788619928.097104:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe000380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571639680:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619928.097304:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe000380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571639680:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 200us (425us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619928.097312:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 851 00002000:00100000:0.0:1788619928.097320:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fe000380 x1875503571639680/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/4536 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619928.097403:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fe000380 with x1875503571639680 00000100:00100000:0.0:1788619928.101190:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619928.101295:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571639808 00000100:00100000:2.0:1788619928.101307:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571639808 00002000:00100000:2.0:1788619928.101399:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be4000 x1875503571639808/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619928.101411:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be4000 with x1875503571639808 00000100:00100000:2.0:1788619928.101546:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 852 00000100:00100000:2.0:1788619928.101556:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571639808:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619928.101790:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571639808:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 233us (610us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619928.101798:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 852 00002000:00100000:2.0:1788619928.101804:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be4000 x1875503571639808/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/4536 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619928.101812:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be4000 with x1875503571639808 00000100:00100000:2.0:1788619928.105367:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619928.105558:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571639936 00000100:00100000:1.0:1788619928.105569:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571639936 00002000:00100000:1.0:1788619928.105633:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e6478000 x1875503571639936/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619928.105645:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e6478000 with x1875503571639936 00000100:00100000:1.0:1788619928.105681:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 853 00000100:00100000:1.0:1788619928.105690:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e6478000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571639936:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:0.0:1788619928.109079:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 794416 -> 794416 00000004:00010000:0.0:1788619928.109090:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619927 -> 1788619927 00000004:00010000:0.0:1788619928.109093:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619927 -> 1788619927 00000004:00010000:0.0:1788619928.109096:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619927 -> 1788619927 00000004:00010000:0.0:1788619928.109099:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 902 -> 902 00000100:00100000:0.0:1788619928.109184:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e6478000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2352:x1875503571639936:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 3490us (3862us total) trans 4294967792 rc 0/0 00000100:00100000:0.0:1788619928.109196:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 853 00002000:00100000:0.0:1788619928.109209:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e6478000 x1875503571639936/t4294967792(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/448 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619928.109219:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e6478000 with x1875503571639936 00000100:00100000:1.0:1788619928.112790:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619928.112890:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571640064 00000100:00100000:2.0:1788619928.112898:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571640064 00000100:00100000:2.0:1788619928.112959:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1389 00000100:00100000:2.0:1788619928.112969:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe001c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2353:x1875503571640064:12345-192.168.201.5@tcp:16:fsx.0 00000100:00100000:2.0:1788619928.136000:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe001c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2353:x1875503571640064:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 23020us (23207us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619928.136017:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1389 00000001:00080000:2.0:1788619928.136055:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967792 is committed 00000001:00080000:2.0:1788619928.136066:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967782 is committed 00000001:00080000:2.0:1788619928.136072:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967786 is committed 00000001:00080000:2.0:1788619928.136083:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967791 is committed 00000100:00100000:3.0:1788619928.138822:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619928.139038:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571640192 00000100:00100000:2.0:1788619928.139049:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571640192 00000100:00100000:2.0:1788619928.139111:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1390 00000100:00100000:2.0:1788619928.139122:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63f9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571640192:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619928.139144:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619928.139168:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6e00/0x1cd7fe53687361db lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d33e expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.139255:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6c00/0x1cd7fe53687361e2 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.139271:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6c00/0x1cd7fe53687361e2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.139284:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6c00/0x1cd7fe53687361e2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788619928.139313:0:11397:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 2013748 00000004:00200000:2.0:1788619928.139317:0:11397:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788619928.139366:0:11397:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6c00/0x1cd7fe53687361e2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.139392:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6e00/0x1cd7fe53687361db lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d33e expref: 15 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.139410:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6c00/0x1cd7fe53687361e2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d33e expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.139431:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619928.139433:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788619928.139437:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d6c00, rc 0) 00000100:00100000:2.0:1788619928.139495:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63f9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571640192:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 375us (683us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619928.139502:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1390 00000100:00100000:2.0:1788619928.142743:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.142819:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571640320 00000100:00100000:3.0:1788619928.142829:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571640320 00000100:00100000:3.0:1788619928.142888:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1133 00000100:00100000:3.0:1788619928.142899:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e6478380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571640320:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619928.142915:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619928.142932:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fe91493a/0x1cd7fe53687361e9 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d345 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619928.142987:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63fad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589217024:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619928.143062:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c300 going to sleep for 11 seconds 00000100:00100000:3.0:1788619928.146056:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63fad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589217024:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619928.146096:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000fe91493a/0x1cd7fe53687361e9 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d345 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.146127:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fe91493a/0x1cd7fe53687361e9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d345 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.146139:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f71800, rc 0) 00000100:00100000:3.0:1788619928.146193:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e6478380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571640320:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3295us (3460us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619928.146201:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1133 00000100:00100000:1.0:1788619928.149242:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.149449:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571640448 00000100:00100000:3.0:1788619928.149459:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571640448 00000100:00100000:3.0:1788619928.149511:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1134 00000100:00100000:3.0:1788619928.149522:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8595880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571640448:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619928.149540:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619928.149557:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002257c85c/0x1cd7fe53687361f0 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d34c expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619928.149627:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63fad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589217152:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619928.149710:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c300 going to sleep for 11 seconds 00000100:00100000:3.0:1788619928.152613:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63fad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589217152:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619928.152650:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002257c85c/0x1cd7fe53687361f0 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d34c expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.152677:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002257c85c/0x1cd7fe53687361f0 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d34c expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.152696:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f71e00, rc 0) 00000100:00100000:3.0:1788619928.152756:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8595880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571640448:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3236us (3524us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619928.152764:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1134 00000100:00100000:1.0:1788619928.155519:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.155607:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571640704 00000100:00100000:3.0:1788619928.155617:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571640704 00000100:00100000:3.0:1788619928.155672:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1391 00000100:00100000:3.0:1788619928.155682:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8594e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571640704:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:3.0:1788619928.155738:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619928.155761:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7c00/0x1cd7fe53687361bf lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d322 expref: 9 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.155778:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619928.155784:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788619928.156064:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71800/0x1cd7fe53687361f7 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.156079:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6c00/0x1cd7fe53687361e2 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d33e expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.156087:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71800/0x1cd7fe53687361f7 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.156118:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6c00/0x1cd7fe53687361e2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d33e expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.156136:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6c00/0x1cd7fe53687361e2 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d33e expref: 13 pid: 11397 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619928.156148:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63fad80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589217280:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619928.156225:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c300 going to sleep for 11 seconds 00000100:00100000:3.0:1788619928.159799:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63fad80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589217280:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619928.159828:0:5845:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71800/0x1cd7fe53687361f7 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619928.161921:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619928.162077:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571640832 00000100:00100000:0.0:1788619928.162086:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571640832 00010000:00010000:0.0:1788619928.162125:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6c00/0x1cd7fe53687361e2 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d33e expref: 14 pid: 11397 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619928.162164:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 461 00000100:00100000:0.0:1788619928.162172:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8594700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571640832:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619928.162186:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619928.162192:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6c00/0x1cd7fe53687361e2 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d33e expref: 14 pid: 11397 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.162204:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6c00/0x1cd7fe53687361e2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d33e expref: 14 pid: 11397 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.162225:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6c00/0x1cd7fe53687361e2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d33e expref: 14 pid: 11397 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.162240:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619928.162246:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71800/0x1cd7fe53687361f7 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.162255:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71800/0x1cd7fe53687361f7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.162263:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71800/0x1cd7fe53687361f7 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.162293:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619928.162379:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8594700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571640832:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 206us (478us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619928.162386:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 461 00010000:00010000:3.0:1788619928.162495:0:5845:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71800/0x1cd7fe53687361f7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.162504:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71800/0x1cd7fe53687361f7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788619928.162636:0:5845:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967793 00000004:00010000:3.0:1788619928.162684:0:5845:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 794416 -> 794416 00000004:00010000:3.0:1788619928.162688:0:5845:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619927 -> 1788619928 00000004:00010000:3.0:1788619928.162691:0:5845:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619927 -> 1788619927 00000004:00010000:3.0:1788619928.162693:0:5845:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619927 -> 1788619928 00000004:00010000:3.0:1788619928.162695:0:5845:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 902 -> 1158 00000004:00010000:3.0:1788619928.162703:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000011c09927 reply state 0000000020ba5be8 transno 4294967794 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71800/0x1cd7fe53687361f7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619928.162712:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788619928.162781:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8594e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571640704:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 7101us (7273us total) trans 4294967794 rc 0/0 00000100:00100000:3.0:1788619928.162791:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1391 00000020:00080000:3.0:1788619928.162795:0:5845:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788619928.170984:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71800/0x1cd7fe53687361f7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.171002:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71800/0x1cd7fe53687361f7 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.171009:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71800/0x1cd7fe53687361f7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.171015:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c5f71800 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788619928.171020:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71800/0x1cd7fe53687361f7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.171028:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71800/0x1cd7fe53687361f7 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.171046:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619928.171049:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788619928.171054:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71800/0x1cd7fe53687361f7 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.171062:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71800/0x1cd7fe53687361f7 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619928.171166:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619928.171366:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571640960 00000100:00100000:2.0:1788619928.171377:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571640960 00000100:00100000:2.0:1788619928.171441:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1392 00000100:00100000:2.0:1788619928.171451:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8597b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:14908:x1875503571640960:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619928.171474:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619928.171491:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6a00/0x1cd7fe53687361fe lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d353 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.171517:0:9417:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe53687361d4 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d337 expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.171527:0:9417:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6a00/0x1cd7fe53687361fe lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d353 expref: 11 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.171562:0:9417:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe53687361d4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d337 expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.171576:0:9417:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe53687361d4 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d337 expref: 12 pid: 5845 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619928.171754:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e647bb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589217408:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619928.171832:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164300 going to sleep for 11 seconds 00000100:00100000:2.0:1788619928.177410:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619928.177658:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e647bb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589217408:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619928.177697:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6a00/0x1cd7fe53687361fe lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d353 expref: 11 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.177717:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619928.177720:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619928.177725:0:9417:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6a00/0x1cd7fe53687361fe lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d353 expref: 11 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.177735:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d6a00, rc 0) 00000100:00100000:0.0:1788619928.177801:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8597b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571640960:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6350us (6640us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619928.177810:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1392 00000100:00100000:3.0:1788619928.178306:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571641088 00000100:00100000:3.0:1788619928.178314:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571641088 00010000:00010000:3.0:1788619928.178380:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe53687361d4 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d337 expref: 13 pid: 5845 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619928.178518:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 462 00000100:00100000:3.0:1788619928.178528:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e647aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571641088:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619928.178541:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619928.178564:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe53687361d4 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d337 expref: 13 pid: 5845 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.178576:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe53687361d4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d337 expref: 13 pid: 5845 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.178664:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe53687361d4 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d337 expref: 13 pid: 5845 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.178682:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619928.178687:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6a00/0x1cd7fe53687361fe lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d353 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.178697:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6a00/0x1cd7fe53687361fe lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d353 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.178707:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6a00/0x1cd7fe53687361fe lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d353 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.178741:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6a00/0x1cd7fe53687361fe lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d353 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619928.178753:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63fad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589217536:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619928.178807:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c300 going to sleep for 11 seconds 00000100:00100000:0.0:1788619928.186126:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619928.186437:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571641216 00000100:00100000:2.0:1788619928.186446:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571641216 00002000:00010000:2.0:1788619928.186482:0:6701:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1845232->18446744073709551615) 00002000:00100000:2.0:1788619928.186492:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4be7800 with x1875503571641216 ext(1845232->18446744073709551615) 00002000:00010000:2.0:1788619928.186505:0:6701:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4be7800. 00000100:00100000:2.0:1788619928.186539:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 648 00000100:00100000:2.0:1788619928.186549:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571641216:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:2.0:1788619928.187648:0:6701:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 2013748 -> 1845232 00002000:00010000:2.0:1788619928.187655:0:6701:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619927 -> 1788619928 00002000:00010000:2.0:1788619928.187658:0:6701:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619927 -> 1788619926 00002000:00010000:2.0:1788619928.187660:0:6701:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619927 -> 1788619928 00002000:00010000:2.0:1788619928.187663:0:6701:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 135 -> 135 00000100:00100000:2.0:1788619928.187732:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571641216:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 1181us (1613us total) trans 4294967492 rc 0/0 00000100:00100000:2.0:1788619928.187741:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 648 00002000:00010000:2.0:1788619928.187745:0:6701:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1845232->18446744073709551615) 00002000:00100000:2.0:1788619928.187748:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4be7800 with x1875503571641216 ext(1845232->18446744073709551615) 00002000:00010000:2.0:1788619928.187754:0:6701:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4be7800. 00000100:00100000:3.0:1788619928.188146:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63fad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589217536:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619928.188168:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619928.188210:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e647aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571641088:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 9683us (10810us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619928.188219:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 462 00000100:00100000:1.0:1788619928.193038:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619928.193143:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571641344 00000100:00100000:0.0:1788619928.193154:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571641344 00000100:00100000:0.0:1788619928.193248:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1393 00000100:00100000:0.0:1788619928.193259:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8596680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:14908:x1875503571641344:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619928.193280:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619928.193303:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5e00/0x1cd7fe5368736205 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d35a expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.193499:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5a00/0x1cd7fe536873620c lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.193517:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5a00/0x1cd7fe536873620c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.193529:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5a00/0x1cd7fe536873620c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619928.193561:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1845232 00000004:00200000:0.0:1788619928.193566:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619928.193586:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5a00/0x1cd7fe536873620c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.193626:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5e00/0x1cd7fe5368736205 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d35a expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.193645:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5a00/0x1cd7fe536873620c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d35a expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.193653:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619928.193655:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619928.193660:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d5a00, rc 0) 00000100:00100000:0.0:1788619928.193728:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8596680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571641344:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 471us (700us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619928.193734:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1393 00000100:00100000:1.0:1788619928.197045:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619928.197092:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.197164:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571641472 00000100:00100000:3.0:1788619928.197173:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571641472 00000100:00100000:3.0:1788619928.197227:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1394 00000100:00100000:3.0:1788619928.197236:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8597100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571641472:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619928.197254:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619928.197273:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71e00/0x1cd7fe5368736213 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d361 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.197309:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6a00/0x1cd7fe53687361fe lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d353 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619928.197372:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63fad80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589217664:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619928.197457:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c300 going to sleep for 11 seconds 00000100:00100000:2.0:1788619928.197664:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571641600 00000100:00100000:2.0:1788619928.197668:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571641600 00000100:00100000:2.0:1788619928.197707:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1135 00000100:00100000:2.0:1788619928.197715:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8597800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571641600:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619928.197724:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619928.197734:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e3d6efb/0x1cd7fe536873621a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d368 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619928.197767:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e647b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589217792:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619928.197810:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164a80 going to sleep for 11 seconds 00000100:00100000:3.0:1788619928.199921:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63fad80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589217664:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619928.199949:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 794416 00010000:00010000:3.0:1788619928.199968:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71e00/0x1cd7fe5368736213 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d361 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.199990:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619928.199994:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619928.200001:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71e00/0x1cd7fe5368736213 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d361 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.200017:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f71e00, rc 0) 00000100:00100000:3.0:1788619928.200119:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8597100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571641472:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2882us (3085us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619928.200128:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1394 00002000:00010000:0.0:1788619928.203648:0:6696:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619926 -> 1788619927 00000100:00100000:0.0:1788619928.203680:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e647b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589217792:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619928.203718:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009e3d6efb/0x1cd7fe536873621a lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d368 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.203749:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e3d6efb/0x1cd7fe536873621a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d368 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.203762:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d7600, rc 0) 00000100:00100000:0.0:1788619928.203822:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8597800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571641600:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6107us (6729us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619928.203831:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1135 00000100:00100000:0.0:1788619928.207005:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619928.207090:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619928.207168:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571641728 00000100:00100000:1.0:1788619928.207179:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571641728 00000100:00100000:1.0:1788619928.207236:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1395 00000100:00100000:1.0:1788619928.207245:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571641728:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619928.207264:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619928.207285:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368736221 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d36f expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619928.207317:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6a00/0x1cd7fe53687361fe lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d353 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619928.207379:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c8595f80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589217920:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619928.207553:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc600 going to sleep for 11 seconds 00000100:00100000:2.0:1788619928.207678:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571641856 00000100:00100000:2.0:1788619928.207684:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571641856 00000100:00100000:2.0:1788619928.207729:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1136 00000100:00100000:2.0:1788619928.207737:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571641856:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619928.207747:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619928.207758:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029d0975e/0x1cd7fe5368736228 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d376 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619928.207796:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e6478e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589218048:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619928.207824:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164480 going to sleep for 11 seconds 00000100:00100000:2.0:1788619928.210572:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c8595f80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589217920:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619928.210609:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 794416 00010000:00010000:2.0:1788619928.210627:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368736221 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d36f expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.210647:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619928.210650:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619928.210657:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368736221 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d36f expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.210670:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079400, rc 0) 00000100:00100000:3.0:1788619928.210772:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e6478e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589218048:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619928.210792:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000029d0975e/0x1cd7fe5368736228 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d376 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.210808:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029d0975e/0x1cd7fe5368736228 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d376 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.210817:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d7400, rc 0) 00000100:00100000:2.0:1788619928.211081:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571641728:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3836us (3990us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619928.211090:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1395 00000100:00100000:3.0:1788619928.211148:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571641856:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3411us (4152us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619928.211154:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1136 00000100:00100000:0.0:1788619928.213664:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619928.213759:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571641984 00000100:00100000:1.0:1788619928.213768:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571641984 00000100:00100000:1.0:1788619928.213826:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1396 00000100:00100000:1.0:1788619928.213836:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571641984:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619928.213866:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619928.213888:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079800/0x1cd7fe536873622f lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d37d expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619928.214304:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079600/0x1cd7fe5368736236 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619928.214320:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079600/0x1cd7fe5368736236 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619928.214364:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079600/0x1cd7fe5368736236 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788619928.214392:0:11397:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1845232 00000004:00200000:1.0:1788619928.214396:0:11397:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788619928.214556:0:11397:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079600/0x1cd7fe5368736236 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619928.214596:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079800/0x1cd7fe536873622f lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d37d expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619928.214613:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079600/0x1cd7fe5368736236 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d37d expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619928.214621:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619928.214624:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788619928.214628:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079600, rc 0) 00000100:00100000:1.0:1788619928.214680:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+12:14908:x1875503571641984:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 845us (1026us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619928.214687:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1396 00000100:00100000:1.0:1788619928.221130:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619928.221173:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.221242:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571642240 00000100:00100000:3.0:1788619928.221252:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571642240 00000100:00100000:3.0:1788619928.221313:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1137 00000100:00100000:3.0:1788619928.221322:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8594380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571642240:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619928.221360:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619928.221377:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002257c85c/0x1cd7fe536873623d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d38b expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619928.221445:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63fad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589218176:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619928.221524:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c300 going to sleep for 11 seconds 00000100:00100000:2.0:1788619928.221670:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571642112 00000100:00100000:2.0:1788619928.221675:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571642112 00000100:00100000:2.0:1788619928.221712:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1397 00000100:00100000:2.0:1788619928.221720:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8594000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571642112:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619928.221729:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619928.221741:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6800/0x1cd7fe5368736244 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d384 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.221771:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6a00/0x1cd7fe53687361fe lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d353 expref: 11 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619928.221795:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e6479f80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589218304:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619928.221821:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164f00 going to sleep for 11 seconds 00000004:00010000:0.0:1788619928.224242:0:5844:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619927 -> 1788619928 00000100:00100000:0.0:1788619928.224273:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e6479f80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589218304:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619928.224295:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 794416 00010000:00010000:0.0:1788619928.224313:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6800/0x1cd7fe5368736244 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d384 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.224351:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619928.224355:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619928.224361:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6800/0x1cd7fe5368736244 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d384 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.224376:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d6800, rc 0) 00000100:00100000:0.0:1788619928.224449:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8594000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571642112:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2729us (3274us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619928.224458:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1397 00002000:00010000:3.0:1788619928.225665:0:7654:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619927 -> 1788619928 00000100:00100000:3.0:1788619928.225691:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63fad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589218176:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619928.225717:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002257c85c/0x1cd7fe536873623d lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d38b expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.225738:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002257c85c/0x1cd7fe536873623d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d38b expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.225749:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f71e00, rc 0) 00000100:00100000:3.0:1788619928.225799:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8594380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571642240:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4478us (4679us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619928.225806:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1137 00000100:00100000:2.0:1788619928.228320:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619928.228385:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619928.228423:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571642368 00000100:00100000:2.0:1788619928.228434:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571642368 00000100:00100000:2.0:1788619928.228489:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1398 00000100:00100000:2.0:1788619928.228500:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e647a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571642368:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619928.228521:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619928.228540:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7800/0x1cd7fe536873624b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d392 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.228574:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6a00/0x1cd7fe53687361fe lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d353 expref: 11 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619928.228624:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e6479c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589218432:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619928.228695:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81649c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619928.228886:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571642496 00000100:00100000:0.0:1788619928.228891:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571642496 00000100:00100000:0.0:1788619928.228930:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1138 00000100:00100000:0.0:1788619928.228938:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e6479880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571642496:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619928.228947:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619928.228960:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045d2447d/0x1cd7fe5368736252 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d399 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619928.228996:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5024700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589218560:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619928.229031:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590900 going to sleep for 11 seconds 00000100:00100000:3.0:1788619928.232575:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e6479c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589218432:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619928.232614:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 794416 00010000:00010000:3.0:1788619928.232633:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7800/0x1cd7fe536873624b lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d392 expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.232659:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619928.232662:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619928.232668:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7800/0x1cd7fe536873624b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d392 expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.232683:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d7800, rc 0) 00000100:00100000:3.0:1788619928.232736:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e647a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571642368:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4237us (4349us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619928.232745:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1398 00000100:00100000:3.0:1788619928.232999:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5024700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589218560:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619928.233021:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000045d2447d/0x1cd7fe5368736252 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d399 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.233038:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045d2447d/0x1cd7fe5368736252 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d399 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.233049:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d4800, rc 0) 00000100:00100000:3.0:1788619928.233084:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e6479880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571642496:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4146us (4771us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619928.233091:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1138 00000100:00100000:2.0:1788619928.237385:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619928.237497:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571642624 00000100:00100000:1.0:1788619928.237509:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571642624 00000100:00100000:1.0:1788619928.237567:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1399 00000100:00100000:1.0:1788619928.237579:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e647ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571642624:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619928.237608:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619928.237632:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe5368736259 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d3a0 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619928.237674:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6a00/0x1cd7fe53687361fe lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d353 expref: 11 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619928.237715:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c8595500 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589218688:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619928.237794:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc600 going to sleep for 11 seconds 00000100:00100000:0.0:1788619928.239758:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619928.239825:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571642752 00000100:00100000:1.0:1788619928.239832:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571642752 00000100:00100000:1.0:1788619928.239895:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1139 00000100:00100000:1.0:1788619928.239913:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5024700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571642752:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619928.239934:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619928.239952:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008447b954/0x1cd7fe5368736260 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d3a7 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619928.240004:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c8594a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589218816:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619928.240068:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619928.242702:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c8595500 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589218688:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619928.242727:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 794416 00010000:00010000:1.0:1788619928.242745:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe5368736259 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d3a0 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619928.242766:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619928.242768:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619928.242774:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079c00/0x1cd7fe5368736259 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d3a0 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619928.242794:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079c00, rc 0) 00000100:00100000:1.0:1788619928.242863:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e647ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571642624:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5279us (5479us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619928.242872:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1399 00000100:00100000:0.0:1788619928.243361:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c8594a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589218816:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619928.243382:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008447b954/0x1cd7fe5368736260 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d3a7 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.243400:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008447b954/0x1cd7fe5368736260 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d3a7 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.243409:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079000, rc 0) 00000100:00100000:0.0:1788619928.243447:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5024700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571642752:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3544us (3695us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619928.243453:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1139 00000100:00100000:3.0:1788619928.247430:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619928.247631:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571642880 00000100:00100000:0.0:1788619928.247641:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571642880 00000100:00100000:0.0:1788619928.247706:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1400 00000100:00100000:0.0:1788619928.247716:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63fad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571642880:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619928.247737:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619928.247758:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4400/0x1cd7fe5368736267 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d3ae expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.247796:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6a00/0x1cd7fe53687361fe lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d353 expref: 11 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619928.247838:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5dc9500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589218944:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619928.248136:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590c00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619928.250829:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619928.251018:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5dc9500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589218944:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619928.251040:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 794416 00010000:00010000:1.0:1788619928.251056:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4400/0x1cd7fe5368736267 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d3ae expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619928.251076:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619928.251079:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619928.251084:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4400/0x1cd7fe5368736267 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d3ae expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619928.251099:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d4400, rc 0) 00000100:00100000:1.0:1788619928.251152:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63fad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571642880:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3438us (3731us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619928.251160:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1400 00000100:00100000:3.0:1788619928.251268:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571643008 00000100:00100000:3.0:1788619928.251276:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571643008 00000100:00100000:3.0:1788619928.251319:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1140 00000100:00100000:3.0:1788619928.251349:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e6479180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571643008:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619928.251361:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619928.251375:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002257c85c/0x1cd7fe536873626e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d3b5 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619928.251430:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63faa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589219072:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619928.251478:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c300 going to sleep for 11 seconds 00000100:00100000:3.0:1788619928.253890:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63faa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589219072:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619928.253937:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002257c85c/0x1cd7fe536873626e lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d3b5 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.253962:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002257c85c/0x1cd7fe536873626e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d3b5 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.253974:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f71e00, rc 0) 00000100:00100000:3.0:1788619928.254031:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e6479180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571643008:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2703us (3211us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619928.254039:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1140 00000100:00100000:2.0:1788619928.258476:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.258556:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571643264 00000100:00100000:3.0:1788619928.258566:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571643264 00000100:00100000:3.0:1788619928.258654:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1401 00000100:00100000:3.0:1788619928.258666:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e6478a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+12:14908:x1875503571643264:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:3.0:1788619928.258723:0:9417:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619928.258749:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079600/0x1cd7fe5368736236 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d37d expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.258771:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619928.258776:0:9417:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788619928.259000:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71c00/0x1cd7fe5368736275 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.259015:0:9417:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5a00/0x1cd7fe536873620c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d35a expref: 12 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.259024:0:9417:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71c00/0x1cd7fe5368736275 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.259058:0:9417:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5a00/0x1cd7fe536873620c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d35a expref: 12 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.259072:0:9417:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5a00/0x1cd7fe536873620c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d35a expref: 12 pid: 5844 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619928.259085:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63faa00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589219200:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619928.259172:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c300 going to sleep for 11 seconds 00000100:00100000:2.0:1788619928.263753:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.263866:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63faa00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589219200:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619928.263897:0:9417:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71c00/0x1cd7fe5368736275 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619928.264134:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571643392 00000100:00100000:0.0:1788619928.264142:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571643392 00010000:00010000:0.0:1788619928.264180:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5a00/0x1cd7fe536873620c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d35a expref: 13 pid: 5844 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619928.264249:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 463 00000100:00100000:0.0:1788619928.264259:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e6479c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571643392:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619928.264274:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619928.264281:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5a00/0x1cd7fe536873620c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d35a expref: 13 pid: 5844 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.264293:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5a00/0x1cd7fe536873620c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d35a expref: 13 pid: 5844 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.264319:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5a00/0x1cd7fe536873620c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d35a expref: 13 pid: 5844 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.264373:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619928.264378:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71c00/0x1cd7fe5368736275 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.264390:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71c00/0x1cd7fe5368736275 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.264398:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71c00/0x1cd7fe5368736275 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.264435:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619928.264485:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e6479c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2355:x1875503571643392:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 227us (741us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619928.264493:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 463 00010000:00010000:2.0:1788619928.264619:0:9417:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71c00/0x1cd7fe5368736275 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.264628:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71c00/0x1cd7fe5368736275 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788619928.264741:0:9417:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967795 00000004:00010000:2.0:1788619928.264791:0:9417:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 794416 -> 794416 00000004:00010000:2.0:1788619928.264795:0:9417:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619928 -> 1788619928 00000004:00010000:2.0:1788619928.264798:0:9417:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619928 -> 1788619927 00000004:00010000:2.0:1788619928.264800:0:9417:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619928 -> 1788619928 00000004:00010000:2.0:1788619928.264803:0:9417:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1158 -> 1158 00000004:00010000:2.0:1788619928.264810:0:9417:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000060d3cce7 reply state 000000006a0c6a62 transno 4294967796 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71c00/0x1cd7fe5368736275 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619928.264819:0:9417:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788619928.264894:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e6478a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+14:14908:x1875503571643264:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 6230us (6430us total) trans 4294967796 rc 0/0 00000100:00100000:2.0:1788619928.264902:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1401 00000020:00080000:2.0:1788619928.264905:0:9417:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1788619928.268580:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.268722:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571643520 00000100:00100000:3.0:1788619928.268731:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571643520 00002000:00010000:3.0:1788619928.268771:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (2030185->18446744073709551615) 00002000:00100000:3.0:1788619928.268777:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78e6479f80 with x1875503571643520 ext(2030185->18446744073709551615) 00002000:00010000:3.0:1788619928.268790:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78e6479f80. 00000100:00100000:3.0:1788619928.268824:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 649 00000100:00100000:3.0:1788619928.268836:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e6479f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571643520:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:3.0:1788619928.269077:0:6702:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1845232 -> 2030185 00002000:00010000:3.0:1788619928.269081:0:6702:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619928 -> 1788619928 00002000:00010000:3.0:1788619928.269083:0:6702:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619928 -> 1788619926 00002000:00010000:3.0:1788619928.269085:0:6702:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619928 -> 1788619928 00002000:00010000:3.0:1788619928.269088:0:6702:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 135 -> 135 00000100:00100000:3.0:1788619928.269157:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e6479f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571643520:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 323us (587us total) trans 4294967493 rc 0/0 00000100:00100000:3.0:1788619928.269165:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 649 00002000:00010000:3.0:1788619928.269168:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (2030185->18446744073709551615) 00002000:00100000:3.0:1788619928.269171:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78e6479f80 with x1875503571643520 ext(2030185->18446744073709551615) 00002000:00010000:3.0:1788619928.269176:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78e6479f80. 00000100:00100000:1.0:1788619928.272214:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619928.272314:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571643648 00000100:00100000:2.0:1788619928.272358:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571643648 00000100:00100000:2.0:1788619928.272425:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1402 00000100:00100000:2.0:1788619928.272435:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8595500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+12:14908:x1875503571643648:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619928.272456:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619928.272477:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6200/0x1cd7fe536873627c lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d3bc expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.272564:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7e00/0x1cd7fe5368736283 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.272578:0:11397:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71c00/0x1cd7fe5368736275 lrc: 2/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.272597:0:11397:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7e00/0x1cd7fe5368736283 lrc: 4/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x13/0x8 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.272616:0:11397:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71c00/0x1cd7fe5368736275 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000020 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.272627:0:11397:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7e00/0x1cd7fe5368736283 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x13/0x8 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.273052:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71c00/0x1cd7fe5368736275 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000020 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.273061:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71c00/0x1cd7fe5368736275 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000020 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.273068:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71c00/0x1cd7fe5368736275 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000020 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.273073:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c5f71c00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788619928.273077:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71c00/0x1cd7fe5368736275 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000020 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.273084:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71c00/0x1cd7fe5368736275 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000020 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.273098:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619928.273101:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788619928.273103:0:3303:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7e00/0x1cd7fe5368736283 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x13/0x8 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.273116:0:3303:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7e00/0x1cd7fe5368736283 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40290000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.273123:0:3303:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7e00/0x1cd7fe5368736283 lrc: 4/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40290000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.273151:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71c00/0x1cd7fe5368736275 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000020 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.273158:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71c00/0x1cd7fe5368736275 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000020 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.273214:0:11397:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7e00/0x1cd7fe5368736283 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.273220:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7e00/0x1cd7fe5368736283 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788619928.273249:0:11397:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 2030185 00000004:00200000:2.0:1788619928.273253:0:11397:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788619928.273271:0:11397:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7e00/0x1cd7fe5368736283 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.273297:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6200/0x1cd7fe536873627c lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d3bc expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.273309:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7e00/0x1cd7fe5368736283 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d3bc expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.273317:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619928.273320:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788619928.274508:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d7e00, rc 0) 00000100:00100000:2.0:1788619928.274585:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8595500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571643648:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2148us (2377us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619928.274602:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1402 00000100:00100000:1.0:1788619928.277942:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619928.277980:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619928.278042:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571643776 00000100:00100000:2.0:1788619928.278050:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571643776 00000100:00100000:2.0:1788619928.278104:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1403 00000100:00100000:2.0:1788619928.278113:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8596a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571643776:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619928.278130:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619928.278146:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7200/0x1cd7fe536873628a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d3c3 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.278178:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6a00/0x1cd7fe53687361fe lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d353 expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619928.278214:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e6478e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589219328:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619928.278289:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164e40 going to sleep for 11 seconds 00000100:00100000:3.0:1788619928.279917:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571643904 00000100:00100000:3.0:1788619928.279928:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571643904 00000100:00100000:3.0:1788619928.279982:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1141 00000100:00100000:3.0:1788619928.279991:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8595c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571643904:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619928.280004:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619928.280018:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002257c85c/0x1cd7fe5368736291 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d3ca expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619928.280065:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63faa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589219456:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619928.280127:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c300 going to sleep for 11 seconds 00000100:00100000:3.0:1788619928.280973:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e6478e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589219328:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619928.280995:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 794416 00010000:00010000:3.0:1788619928.281011:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7200/0x1cd7fe536873628a lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d3c3 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.281030:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619928.281034:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619928.281040:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7200/0x1cd7fe536873628a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d3c3 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.281054:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d7200, rc 0) 00000100:00100000:3.0:1788619928.281146:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8596a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571643776:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3033us (3212us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619928.281154:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1403 00002000:00010000:3.0:1788619928.284312:0:7654:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619926 -> 1788619927 00000100:00100000:3.0:1788619928.284526:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63faa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589219456:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619928.284559:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002257c85c/0x1cd7fe5368736291 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d3ca expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.284582:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002257c85c/0x1cd7fe5368736291 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d3ca expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.284603:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f71e00, rc 0) 00000100:00100000:3.0:1788619928.284707:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8595c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571643904:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4716us (6724us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619928.284716:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1141 00000100:00100000:2.0:1788619928.287710:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619928.287753:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619928.287834:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571644160 00000100:00100000:0.0:1788619928.287844:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571644160 00000100:00100000:0.0:1788619928.287924:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1142 00000100:00100000:0.0:1788619928.287934:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e647b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571644160:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619928.287952:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619928.287971:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000df72df5a/0x1cd7fe5368736298 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d3d8 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619928.288032:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5dc8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589219584:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619928.288113:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590b40 going to sleep for 11 seconds 00000100:00100000:3.0:1788619928.288385:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571644032 00000100:00100000:3.0:1788619928.288390:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571644032 00000100:00100000:3.0:1788619928.288432:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1404 00000100:00100000:3.0:1788619928.288441:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e647bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571644032:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619928.288452:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619928.288465:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70800/0x1cd7fe536873629f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d3d1 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.288496:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6a00/0x1cd7fe53687361fe lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d353 expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619928.288521:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63faa00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589219712:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619928.288554:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c300 going to sleep for 11 seconds 00000100:00100000:3.0:1788619928.290940:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63faa00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589219712:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619928.290969:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 794416 00010000:00010000:3.0:1788619928.290988:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70800/0x1cd7fe536873629f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d3d1 expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.291010:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619928.291013:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619928.291018:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70800/0x1cd7fe536873629f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d3d1 expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.291034:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f70800, rc 0) 00000100:00100000:3.0:1788619928.291142:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e647bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571644032:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2702us (3388us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619928.291150:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1404 00000100:00100000:0.0:1788619928.291349:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5dc8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589219584:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619928.291369:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000df72df5a/0x1cd7fe5368736298 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d3d8 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.291387:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000df72df5a/0x1cd7fe5368736298 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d3d8 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.291398:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d5200, rc 0) 00000100:00100000:0.0:1788619928.291433:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e647b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571644160:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3501us (3734us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619928.291440:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1142 00000100:00100000:1.0:1788619928.295082:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619928.295154:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619928.295219:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571644288 00000100:00100000:2.0:1788619928.295229:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571644288 00000100:00100000:2.0:1788619928.295294:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1405 00000100:00100000:2.0:1788619928.295305:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8595f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571644288:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619928.295358:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619928.295379:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6400/0x1cd7fe53687362a6 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d3df expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.295432:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6a00/0x1cd7fe53687361fe lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d353 expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619928.295475:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e6478e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589219840:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619928.295549:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619928.297607:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e6478e00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589219840:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619928.297639:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 794416 00010000:00010000:1.0:1788619928.297657:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6400/0x1cd7fe53687362a6 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d3df expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619928.297680:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619928.297683:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619928.297689:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6400/0x1cd7fe53687362a6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d3df expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619928.297704:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d6400, rc 0) 00000100:00100000:1.0:1788619928.297759:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8595f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571644288:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2456us (2603us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619928.297767:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1405 00000100:00100000:0.0:1788619928.299535:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571644416 00000100:00100000:0.0:1788619928.299550:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571644416 00000100:00100000:0.0:1788619928.299654:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1143 00000100:00100000:0.0:1788619928.299664:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8594a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571644416:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619928.299683:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619928.299700:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bdacd512/0x1cd7fe53687362ad lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d3e6 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619928.299760:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5dc8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589219968:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619928.299843:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590b40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619928.303995:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5dc8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589219968:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619928.304031:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bdacd512/0x1cd7fe53687362ad lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d3e6 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.304054:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bdacd512/0x1cd7fe53687362ad lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d3e6 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.304067:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d4c00, rc 0) 00000100:00100000:0.0:1788619928.304152:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8594a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571644416:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4489us (9080us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619928.304162:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1143 00000100:00100000:2.0:1788619928.307229:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.307321:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571644544 00000100:00100000:3.0:1788619928.307594:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571644544 00000100:00100000:0.0:1788619928.307711:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1406 00000100:00100000:0.0:1788619928.307722:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e6478e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571644544:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619928.307742:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619928.307763:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4600/0x1cd7fe53687362b4 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d3ed expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.307793:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6a00/0x1cd7fe53687361fe lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d353 expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.307805:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4600/0x1cd7fe53687362b4 lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d3ed expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.307840:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6a00/0x1cd7fe53687361fe lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d353 expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.307866:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6a00/0x1cd7fe53687361fe lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d353 expref: 12 pid: 9417 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619928.307880:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5dc8a80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589220096:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619928.308126:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590b40 going to sleep for 11 seconds 00000100:00100000:2.0:1788619928.310688:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.310756:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571644672 00000100:00100000:3.0:1788619928.310764:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571644672 00010000:00010000:3.0:1788619928.310801:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6a00/0x1cd7fe53687361fe lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d353 expref: 13 pid: 9417 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619928.310874:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 464 00000100:00100000:3.0:1788619928.310882:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+13:2355:x1875503571644672:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619928.310896:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619928.310932:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6a00/0x1cd7fe53687361fe lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d353 expref: 13 pid: 9417 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.310945:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6a00/0x1cd7fe53687361fe lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d353 expref: 13 pid: 9417 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.310965:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619928.310970:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4600/0x1cd7fe53687362b4 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d3ed expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.310980:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4600/0x1cd7fe53687362b4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d3ed expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.310989:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4600/0x1cd7fe53687362b4 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d3ed expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.311023:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4600/0x1cd7fe53687362b4 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d3ed expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619928.311041:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63faa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589220224:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619928.311120:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c300 going to sleep for 11 seconds 00010000:00010000:0.0:1788619928.311461:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6a00/0x1cd7fe53687361fe lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d353 expref: 13 pid: 9417 timeout: 520 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619928.311478:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5dc8a80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589220096:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619928.311503:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4600/0x1cd7fe53687362b4 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d3ed expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.311514:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619928.311516:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619928.311519:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d4600, rc 0) 00000100:00100000:0.0:1788619928.311562:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e6478e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571644544:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3842us (4343us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619928.311570:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1406 00000100:00100000:0.0:1788619928.314926:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63faa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589220224:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619928.314953:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619928.315001:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571644672:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 4118us (4322us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619928.315011:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 464 00000100:00100000:3.0:1788619928.317225:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619928.317316:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571644800 00000100:00100000:2.0:1788619928.317359:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571644800 00002000:00100000:2.0:1788619928.317419:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c63fb480 x1875503571644800/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619928.317429:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c63fb480 with x1875503571644800 00000100:00100000:2.0:1788619928.317457:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 854 00000100:00100000:2.0:1788619928.317466:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63fb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571644800:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619928.317736:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63fb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571644800:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 270us (520us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619928.317744:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 854 00002000:00100000:2.0:1788619928.317751:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c63fb480 x1875503571644800/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/4536 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619928.317759:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c63fb480 with x1875503571644800 00000100:00100000:1.0:1788619928.320914:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619928.321485:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571644928 00000100:00100000:0.0:1788619928.321496:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571644928 00002000:00100000:2.0:1788619928.321611:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5f24a80 x1875503571644928/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619928.321621:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5f24a80 with x1875503571644928 00000100:00100000:2.0:1788619928.321702:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 855 00000100:00100000:2.0:1788619928.321713:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f24a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571644928:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619928.322020:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f24a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571644928:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 308us (1116us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619928.322027:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 855 00002000:00100000:2.0:1788619928.322034:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5f24a80 x1875503571644928/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/4536 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619928.322043:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5f24a80 with x1875503571644928 00000100:00100000:3.0:1788619928.325121:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619928.325216:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571645056 00000100:00100000:1.0:1788619928.325226:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571645056 00002000:00100000:1.0:1788619928.325275:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c63f8000 x1875503571645056/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619928.325286:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c63f8000 with x1875503571645056 00000100:00100000:1.0:1788619928.325320:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 856 00000100:00100000:1.0:1788619928.325362:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63f8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2353:x1875503571645056:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619928.325636:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63f8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2353:x1875503571645056:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 274us (523us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619928.325644:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 856 00002000:00100000:1.0:1788619928.325651:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c63f8000 x1875503571645056/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/4536 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619928.325665:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c63f8000 with x1875503571645056 00000100:00100000:3.0:1788619928.329050:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619928.329318:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571645184 00000100:00100000:2.0:1788619928.329368:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571645184 00002000:00100000:2.0:1788619928.329429:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c63faa00 x1875503571645184/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619928.329440:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c63faa00 with x1875503571645184 00000100:00100000:2.0:1788619928.329478:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 857 00000100:00100000:2.0:1788619928.329489:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63faa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2352:x1875503571645184:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619928.329775:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63faa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2352:x1875503571645184:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 287us (734us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619928.329783:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 857 00002000:00100000:2.0:1788619928.329790:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c63faa00 x1875503571645184/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/4536 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619928.329798:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c63faa00 with x1875503571645184 00000100:00100000:2.0:1788619928.333421:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.333716:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571645312 00000100:00100000:3.0:1788619928.333726:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571645312 00000100:00100000:3.0:1788619928.333788:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1144 00000100:00100000:3.0:1788619928.333798:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571645312:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619928.333817:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619928.333837:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d234829/0x1cd7fe53687362bb lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d3f4 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619928.333912:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63f8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589220352:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619928.334003:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619928.336206:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63f8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589220352:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619928.336246:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009d234829/0x1cd7fe53687362bb lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d3f4 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.336271:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d234829/0x1cd7fe53687362bb lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d3f4 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.336285:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f70800, rc 0) 00000100:00100000:2.0:1788619928.336394:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571645312:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2597us (2991us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619928.336402:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1144 00000100:00100000:2.0:1788619928.339206:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.339288:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571645440 00000100:00100000:3.0:1788619928.339296:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571645440 00000100:00100000:3.0:1788619928.339386:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1145 00000100:00100000:3.0:1788619928.339398:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571645440:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619928.339426:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619928.339447:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002257c85c/0x1cd7fe53687362c2 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d3fb expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619928.339521:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63fb100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589220480:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619928.339627:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c240 going to sleep for 11 seconds 00000100:00100000:0.0:1788619928.341996:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63fb100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589220480:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619928.342032:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002257c85c/0x1cd7fe53687362c2 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d3fb expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.342055:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002257c85c/0x1cd7fe53687362c2 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d3fb expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.342068:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f71e00, rc 0) 00000100:00100000:0.0:1788619928.342119:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571645440:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2723us (2923us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619928.342128:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1145 00000100:00100000:1.0:1788619928.344603:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619928.344707:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571645568 00000100:00100000:0.0:1788619928.344716:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571645568 00000100:00100000:0.0:1788619928.344768:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1146 00000100:00100000:0.0:1788619928.344776:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f27100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571645568:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619928.344793:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619928.344809:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bdacd512/0x1cd7fe53687362c9 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d402 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619928.344872:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5dc8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589220608:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619928.344954:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590b40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619928.347683:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5dc8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589220608:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619928.347722:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bdacd512/0x1cd7fe53687362c9 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d402 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.347746:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bdacd512/0x1cd7fe53687362c9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d402 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.347758:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d4c00, rc 0) 00000100:00100000:0.0:1788619928.347817:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f27100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571645568:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3040us (3232us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619928.347826:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1146 00000100:00100000:3.0:1788619928.350484:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619928.350569:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571645696 00000100:00100000:0.0:1788619928.350579:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571645696 00000100:00100000:0.0:1788619928.350646:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1147 00000100:00100000:0.0:1788619928.350656:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63fb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571645696:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619928.350674:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619928.350692:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000df72df5a/0x1cd7fe53687362d0 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d409 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619928.350747:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5dc8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589220736:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619928.350831:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590b40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619928.352736:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5dc8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589220736:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619928.352780:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000df72df5a/0x1cd7fe53687362d0 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d409 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.352803:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000df72df5a/0x1cd7fe53687362d0 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d409 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.352815:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d5200, rc 0) 00000100:00100000:0.0:1788619928.352879:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63fb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571645696:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2224us (2405us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619928.352888:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1147 00010000:00010000:2.0:1788619928.354825:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788619928.356043:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.356154:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571645824 00000100:00100000:3.0:1788619928.356164:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571645824 00000100:00100000:3.0:1788619928.356219:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1148 00000100:00100000:3.0:1788619928.356230:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a59180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571645824:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619928.356247:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619928.356265:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000709ffba6/0x1cd7fe53687362d7 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d410 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619928.356321:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63f8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589220864:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619928.356550:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48ccc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619928.359504:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63f8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589220864:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619928.359545:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000709ffba6/0x1cd7fe53687362d7 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d410 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.359569:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000709ffba6/0x1cd7fe53687362d7 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d410 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.359581:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f70400, rc 0) 00000100:00100000:3.0:1788619928.359689:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a59180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571645824:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3460us (3653us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619928.359697:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1148 00000100:00100000:1.0:1788619928.362640:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.362726:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571645952 00000100:00100000:3.0:1788619928.362736:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571645952 00000100:00100000:3.0:1788619928.362823:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1149 00000100:00100000:3.0:1788619928.362833:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f24700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571645952:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619928.362999:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619928.363018:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e9c3b213/0x1cd7fe53687362de lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d417 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619928.363076:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63f8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589220992:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619928.363162:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48ccc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619928.365575:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63f8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589220992:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619928.365626:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e9c3b213/0x1cd7fe53687362de lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d417 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.365651:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e9c3b213/0x1cd7fe53687362de lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d417 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.365663:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f71200, rc 0) 00000100:00100000:3.0:1788619928.365763:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f24700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571645952:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2930us (3132us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619928.365772:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1149 00000100:00100000:2.0:1788619928.369227:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619928.369317:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571646080 00000100:00100000:0.0:1788619928.369352:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571646080 00000100:00100000:0.0:1788619928.369419:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1150 00000100:00100000:0.0:1788619928.369430:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571646080:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619928.369449:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619928.369469:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000df72df5a/0x1cd7fe53687362e5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d41e expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619928.369530:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5dc8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589221120:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619928.369615:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590b40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619928.372237:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5dc8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589221120:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619928.372278:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000df72df5a/0x1cd7fe53687362e5 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d41e expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.372304:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000df72df5a/0x1cd7fe53687362e5 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d41e expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.372317:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d5200, rc 0) 00000100:00100000:1.0:1788619928.372720:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571646080:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3290us (3501us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619928.372730:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1150 00000100:00100000:2.0:1788619928.375596:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619928.375687:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571646208 00000100:00100000:0.0:1788619928.375698:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571646208 00000100:00100000:0.0:1788619928.375781:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1151 00000100:00100000:0.0:1788619928.375792:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a59880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571646208:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619928.375812:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619928.375833:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bdacd512/0x1cd7fe53687362ec lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d425 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619928.375915:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5dc8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589221248:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619928.376001:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590b40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619928.378312:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5dc8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589221248:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619928.378374:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bdacd512/0x1cd7fe53687362ec lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d425 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.378405:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bdacd512/0x1cd7fe53687362ec lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d425 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.378491:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d4c00, rc 0) 00000100:00100000:0.0:1788619928.378599:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a59880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571646208:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2809us (3019us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619928.378609:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1151 00000100:00100000:1.0:1788619928.381261:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.381360:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571646336 00000100:00100000:3.0:1788619928.381370:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571646336 00000100:00100000:3.0:1788619928.381517:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1152 00000100:00100000:3.0:1788619928.381527:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f24000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571646336:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619928.381544:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619928.381561:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e9c3b213/0x1cd7fe53687362f3 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d42c expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619928.381628:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63f8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589221376:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619928.381715:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48ccc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619928.384109:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63f8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589221376:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619928.384146:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e9c3b213/0x1cd7fe53687362f3 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d42c expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.384169:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e9c3b213/0x1cd7fe53687362f3 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d42c expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.384182:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f71200, rc 0) 00000100:00100000:3.0:1788619928.384238:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f24000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571646336:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2713us (2986us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619928.384245:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1152 00000100:00100000:2.0:1788619928.386983:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.387305:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571646464 00000100:00100000:3.0:1788619928.387315:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571646464 00000100:00100000:2.0:1788619928.387626:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1153 00000100:00100000:2.0:1788619928.387638:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:14908:x1875503571646464:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619928.387656:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619928.387674:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029d0975e/0x1cd7fe53687362fa lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d433 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.387707:0:7654:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000c7d29fbb/0x1cd7fe5368736125 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2b2 expref: 12 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.387717:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=1s v=(1 12) 00010000:00010000:2.0:1788619928.387722:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 12) 00010000:00010000:2.0:1788619928.387728:0:7654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029d0975e/0x1cd7fe53687362fa lrc: 4/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1744896->1748991] (req 1744896->1748991) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d433 expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.387760:0:7654:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000c7d29fbb/0x1cd7fe5368736125 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2b2 expref: 12 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.387774:0:7654:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000c7d29fbb/0x1cd7fe5368736125 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2b2 expref: 12 pid: 7654 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619928.387787:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a5a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589221504:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619928.387897:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164d80 going to sleep for 11 seconds 00000100:00100000:0.0:1788619928.391809:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a5a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589221504:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619928.391846:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000029d0975e/0x1cd7fe53687362fa lrc: 3/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1744896->1748991] (req 1744896->1748991) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d433 expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.391880:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d7400, rc 0) 00000100:00100000:0.0:1788619928.392257:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+6:14908:x1875503571646464:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4619us (5284us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619928.392267:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1153 00000100:00100000:2.0:1788619928.392439:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619928.392643:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571646592 00000100:00100000:0.0:1788619928.392650:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571646592 00002000:00100000:0.0:1788619928.392688:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1183744->1245183) req@ffff8e77c6a59f80 x1875503571646592/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619928.392697:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a59f80 with x1875503571646592 ext(1183744->1245183) 00010000:00010000:0.0:1788619928.392704:0:6703:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000c7d29fbb/0x1cd7fe5368736125 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2b2 expref: 13 pid: 7654 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.392723:0:6703:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000c7d29fbb/0x1cd7fe5368736125 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2b2 expref: 13 pid: 7654 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788619928.392731:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff8e77c6a59f80 00000100:00100000:0.0:1788619928.392753:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 22 00000100:00100000:0.0:1788619928.392760:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a59f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571646592:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:2.0:1788619928.401105:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a59f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+15:2354:x1875503571646592:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 8334us (8656us total) trans 4294967494 rc 0/0 00000100:00100000:2.0:1788619928.401120:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 22 00002000:00100000:2.0:1788619928.401134:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1183744->1245183) req@ffff8e77c6a59f80 x1875503571646592/t4294967494(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:24/0 lens 488/448 e 0 to 0 dl 1788619939 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619928.401145:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a59f80 with x1875503571646592 ext(1183744->1245183) 00010000:00010000:2.0:1788619928.401156:0:6703:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000c7d29fbb/0x1cd7fe5368736125 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2b2 expref: 15 pid: 7654 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.401175:0:6703:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000c7d29fbb/0x1cd7fe5368736125 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2b2 expref: 15 pid: 7654 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788619928.401182:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff8e77c6a59f80 00000100:00100000:2.0:1788619928.404982:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619928.405079:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571646720 00000100:00100000:1.0:1788619928.405091:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571646720 00010000:00010000:1.0:1788619928.405130:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000c7d29fbb/0x1cd7fe5368736125 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2b2 expref: 15 pid: 7654 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619928.405207:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 465 00000100:00100000:1.0:1788619928.405217:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a58380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+15:2353:x1875503571646720:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619928.405233:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619928.405257:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000c7d29fbb/0x1cd7fe5368736125 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2b2 expref: 15 pid: 7654 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619928.405272:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000c7d29fbb/0x1cd7fe5368736125 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2b2 expref: 15 pid: 7654 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619928.405296:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7d29fbb/0x1cd7fe5368736125 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d2b2 expref: 15 pid: 7654 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619928.405314:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 12) 00010000:00010000:1.0:1788619928.405319:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 12) 00010000:00010000:1.0:1788619928.405562:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000029d0975e/0x1cd7fe53687362fa lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1744896->1748991) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d433 expref: 5 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619928.405574:0:5829:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029d0975e/0x1cd7fe53687362fa lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1744896->1748991) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d433 expref: 5 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619928.405624:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000029d0975e/0x1cd7fe53687362fa lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1744896->1748991) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d433 expref: 5 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619928.405638:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f26680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1875503589221632:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619928.405727:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc300 going to sleep for 11 seconds 00000100:00100000:1.0:1788619928.409668:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619928.409752:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571646848 00000100:00100000:2.0:1788619928.409763:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571646848 00002000:00100000:2.0:1788619928.409814:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1744896->1748991) req@ffff8e77c5f25180 x1875503571646848/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619928.409825:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5f25180 with x1875503571646848 ext(1744896->1748991) 00002000:00010000:2.0:1788619928.409831:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5f25180 00000100:00100000:2.0:1788619928.409873:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 650 00000100:00100000:2.0:1788619928.409884:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f25180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571646848:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619928.410157:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f25180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571646848:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 275us (499us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619928.410162:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 650 00002000:00100000:2.0:1788619928.410169:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1744896->1748991) req@ffff8e77c5f25180 x1875503571646848/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/4536 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619928.410177:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5f25180 with x1875503571646848 ext(1744896->1748991) 00002000:00010000:2.0:1788619928.410181:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5f25180 00000100:00100000:2.0:1788619928.412781:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.412866:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571646976 00000100:00100000:3.0:1788619928.412875:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571646976 00002000:00100000:3.0:1788619928.412930:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1748992->1753087) req@ffff8e77c6a59500 x1875503571646976/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619928.412941:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a59500 with x1875503571646976 ext(1748992->1753087) 00002000:00010000:3.0:1788619928.412946:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a59500 00000100:00100000:3.0:1788619928.412977:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 651 00000100:00100000:3.0:1788619928.412986:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571646976:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619928.413216:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571646976:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 230us (442us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619928.413223:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 651 00002000:00100000:3.0:1788619928.413230:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1748992->1753087) req@ffff8e77c6a59500 x1875503571646976/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/4536 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619928.413237:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a59500 with x1875503571646976 ext(1748992->1753087) 00002000:00010000:3.0:1788619928.413240:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a59500 00000100:00100000:1.0:1788619928.416446:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.416516:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571647104 00000100:00100000:3.0:1788619928.416528:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571647104 00002000:00100000:3.0:1788619928.416579:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1753088->1757183) req@ffff8e77c5f25c00 x1875503571647104/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619928.416601:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5f25c00 with x1875503571647104 ext(1753088->1757183) 00002000:00010000:3.0:1788619928.416606:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5f25c00 00000100:00100000:3.0:1788619928.416679:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 652 00000100:00100000:3.0:1788619928.416689:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f25c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571647104:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619928.416963:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f25c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571647104:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 275us (527us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619928.416972:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 652 00002000:00100000:3.0:1788619928.416979:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1753088->1757183) req@ffff8e77c5f25c00 x1875503571647104/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/4536 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619928.416988:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5f25c00 with x1875503571647104 ext(1753088->1757183) 00002000:00010000:3.0:1788619928.416992:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5f25c00 00000100:00100000:0.0:1788619928.417084:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f26680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1875503589221632:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619928.417105:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619928.417377:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a58380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+14:2353:x1875503571646720:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 12160us (12404us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619928.417387:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 465 00000100:00100000:2.0:1788619928.422540:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.422655:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571647232 00000100:00100000:3.0:1788619928.422666:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571647232 00002000:00100000:3.0:1788619928.422721:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1757184->1761279) req@ffff8e77c6a5aa00 x1875503571647232/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619928.422734:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a5aa00 with x1875503571647232 ext(1757184->1761279) 00002000:00010000:3.0:1788619928.422739:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a5aa00 00000100:00100000:3.0:1788619928.422812:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 653 00000100:00100000:3.0:1788619928.422822:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571647232:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619928.423081:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571647232:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 259us (551us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619928.423090:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 653 00002000:00100000:3.0:1788619928.423098:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1757184->1761279) req@ffff8e77c6a5aa00 x1875503571647232/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/4536 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619928.423106:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a5aa00 with x1875503571647232 ext(1757184->1761279) 00002000:00010000:3.0:1788619928.423110:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a5aa00 00000100:00100000:0.0:1788619928.426556:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.426661:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571647360 00000100:00100000:3.0:1788619928.426671:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571647360 00002000:00100000:3.0:1788619928.426721:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1761280->1765375) req@ffff8e78c5dc8a80 x1875503571647360/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619928.426733:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5dc8a80 with x1875503571647360 ext(1761280->1765375) 00002000:00010000:3.0:1788619928.426740:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5dc8a80 00000100:00100000:3.0:1788619928.426775:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 654 00000100:00100000:3.0:1788619928.426784:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dc8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571647360:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619928.427217:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dc8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571647360:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 431us (669us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619928.427229:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 654 00002000:00100000:3.0:1788619928.427237:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1761280->1765375) req@ffff8e78c5dc8a80 x1875503571647360/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/4536 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619928.427245:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5dc8a80 with x1875503571647360 ext(1761280->1765375) 00002000:00010000:3.0:1788619928.427250:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5dc8a80 00000100:00100000:1.0:1788619928.430271:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.430389:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571647488 00000100:00100000:3.0:1788619928.430400:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571647488 00002000:00100000:3.0:1788619928.430518:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1765376->1769471) req@ffff8e77c5f24e00 x1875503571647488/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619928.430529:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5f24e00 with x1875503571647488 ext(1765376->1769471) 00002000:00010000:3.0:1788619928.430535:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5f24e00 00000100:00100000:3.0:1788619928.430621:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 655 00000100:00100000:3.0:1788619928.430631:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f24e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571647488:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619928.430884:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f24e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571647488:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 253us (621us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619928.430891:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 655 00002000:00100000:3.0:1788619928.430898:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1765376->1769471) req@ffff8e77c5f24e00 x1875503571647488/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/4536 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619928.430906:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5f24e00 with x1875503571647488 ext(1765376->1769471) 00002000:00010000:3.0:1788619928.430910:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5f24e00 00000100:00100000:1.0:1788619928.433986:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.434066:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571647616 00000100:00100000:3.0:1788619928.434075:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571647616 00000100:00100000:3.0:1788619928.434154:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1407 00000100:00100000:3.0:1788619928.434163:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f25500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+12:14908:x1875503571647616:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619928.434185:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619928.434203:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71200/0x1cd7fe5368736301 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d43a expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.434283:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70400/0x1cd7fe5368736308 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.434299:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70400/0x1cd7fe5368736308 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.434311:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70400/0x1cd7fe5368736308 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788619928.434367:0:9417:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 2030185 00000004:00200000:3.0:1788619928.434371:0:9417:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788619928.434390:0:9417:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70400/0x1cd7fe5368736308 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.434424:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71200/0x1cd7fe5368736301 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d43a expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.434443:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70400/0x1cd7fe5368736308 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d43a expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.434450:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619928.434453:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788619928.434457:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f70400, rc 0) 00000100:00100000:3.0:1788619928.434518:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f25500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+13:14908:x1875503571647616:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 356us (540us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619928.434524:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1407 00000100:00100000:0.0:1788619928.439911:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619928.440095:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571647744 00000100:00100000:1.0:1788619928.440105:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571647744 00000100:00100000:1.0:1788619928.440668:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1408 00000100:00100000:1.0:1788619928.440682:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dc9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571647744:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619928.440703:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619928.440724:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe536873630f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d441 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619928.440765:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe536873630f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d441 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788619928.440777:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 794416 00010000:00010000:1.0:1788619928.440785:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe536873630f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d441 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619928.440798:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619928.440801:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619928.440805:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079000, rc 0) 00000100:00100000:1.0:1788619928.440875:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dc9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+14:2354:x1875503571647744:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 197us (975us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619928.440882:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1408 00000100:00100000:2.0:1788619928.442272:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.443876:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571647872 00000100:00100000:3.0:1788619928.443888:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571647872 00000100:00100000:2.0:1788619928.444196:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1154 00000100:00100000:2.0:1788619928.444207:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+14:2355:x1875503571647872:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619928.444224:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619928.444242:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8dcc7dd/0x1cd7fe5368736316 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d448 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.444274:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 12) 00010000:00010000:2.0:1788619928.444279:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 12) 00010000:00010000:2.0:1788619928.444289:0:6698:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8dcc7dd/0x1cd7fe5368736316 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d448 expref: 15 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.444305:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b8dcc7dd/0x1cd7fe5368736316 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d448 expref: 15 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.444318:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d7800, rc 0) 00000100:00100000:2.0:1788619928.444405:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+15:2355:x1875503571647872:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 200us (2138us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619928.444421:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1154 00000100:00100000:2.0:1788619928.447488:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.447752:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571648000 00000100:00100000:3.0:1788619928.447762:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571648000 00002000:00100000:3.0:1788619928.447807:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1462272->1466367) req@ffff8e79010fd500 x1875503571648000/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619928.447818:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e79010fd500 with x1875503571648000 ext(1462272->1466367) 00002000:00010000:3.0:1788619928.447823:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e79010fd500 00000100:00100000:3.0:1788619928.447894:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 656 00000100:00100000:3.0:1788619928.447903:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010fd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571648000:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619928.448154:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010fd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571648000:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 251us (676us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619928.448162:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 656 00002000:00100000:3.0:1788619928.448169:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1462272->1466367) req@ffff8e79010fd500 x1875503571648000/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/4536 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619928.448177:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e79010fd500 with x1875503571648000 ext(1462272->1466367) 00002000:00010000:3.0:1788619928.448180:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e79010fd500 00000100:00100000:0.0:1788619928.451716:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.451796:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571648128 00000100:00100000:3.0:1788619928.451806:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571648128 00002000:00100000:3.0:1788619928.451904:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1466368->1470463) req@ffff8e78c5dc8380 x1875503571648128/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619928.451916:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5dc8380 with x1875503571648128 ext(1466368->1470463) 00002000:00010000:3.0:1788619928.451922:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5dc8380 00000100:00100000:3.0:1788619928.451996:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 657 00000100:00100000:3.0:1788619928.452005:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dc8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571648128:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619928.452246:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dc8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571648128:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 242us (604us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619928.452254:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 657 00002000:00100000:3.0:1788619928.452260:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1466368->1470463) req@ffff8e78c5dc8380 x1875503571648128/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/4536 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619928.452269:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5dc8380 with x1875503571648128 ext(1466368->1470463) 00002000:00010000:3.0:1788619928.452273:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5dc8380 00000100:00100000:1.0:1788619928.456070:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.456189:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571648256 00000100:00100000:3.0:1788619928.456199:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571648256 00002000:00100000:3.0:1788619928.456251:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1470464->1474559) req@ffff8e77c5f26a00 x1875503571648256/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619928.456262:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5f26a00 with x1875503571648256 ext(1470464->1474559) 00002000:00010000:3.0:1788619928.456268:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5f26a00 00000100:00100000:3.0:1788619928.456380:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 658 00000100:00100000:3.0:1788619928.456389:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f26a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571648256:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619928.456650:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f26a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571648256:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 261us (591us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619928.456658:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 658 00002000:00100000:3.0:1788619928.456665:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1470464->1474559) req@ffff8e77c5f26a00 x1875503571648256/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/4536 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619928.456673:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5f26a00 with x1875503571648256 ext(1470464->1474559) 00002000:00010000:3.0:1788619928.456676:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5f26a00 00000100:00100000:1.0:1788619928.459781:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.459869:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571648384 00000100:00100000:3.0:1788619928.459881:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571648384 00002000:00100000:3.0:1788619928.459936:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1474560->1478655) req@ffff8e77c5f27b80 x1875503571648384/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619928.459949:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5f27b80 with x1875503571648384 ext(1474560->1478655) 00002000:00010000:3.0:1788619928.459954:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5f27b80 00000100:00100000:3.0:1788619928.459988:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 659 00000100:00100000:3.0:1788619928.459998:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f27b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571648384:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619928.460254:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f27b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571648384:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 256us (483us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619928.460262:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 659 00002000:00100000:3.0:1788619928.460270:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1474560->1478655) req@ffff8e77c5f27b80 x1875503571648384/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/4536 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619928.460278:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5f27b80 with x1875503571648384 ext(1474560->1478655) 00002000:00010000:3.0:1788619928.460282:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5f27b80 00000100:00100000:0.0:1788619928.463615:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.463749:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571648512 00000100:00100000:3.0:1788619928.463760:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571648512 00002000:00100000:3.0:1788619928.463811:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1478656->1482751) req@ffff8e78c5dc9500 x1875503571648512/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619928.463821:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5dc9500 with x1875503571648512 ext(1478656->1482751) 00002000:00010000:3.0:1788619928.463828:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5dc9500 00000100:00100000:3.0:1788619928.463875:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 660 00000100:00100000:3.0:1788619928.463886:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dc9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571648512:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619928.464145:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dc9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571648512:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 260us (538us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619928.464156:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 660 00002000:00100000:3.0:1788619928.464163:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1478656->1482751) req@ffff8e78c5dc9500 x1875503571648512/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/4536 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619928.464171:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5dc9500 with x1875503571648512 ext(1478656->1482751) 00002000:00010000:3.0:1788619928.464174:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5dc9500 00000100:00100000:1.0:1788619928.468166:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619928.468257:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571648640 00000100:00100000:2.0:1788619928.468268:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571648640 00002000:00100000:2.0:1788619928.468317:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1482752->1486847) req@ffff8e77c5f26680 x1875503571648640/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619928.468346:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5f26680 with x1875503571648640 ext(1482752->1486847) 00002000:00010000:2.0:1788619928.468351:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5f26680 00000100:00100000:2.0:1788619928.468380:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 661 00000100:00100000:2.0:1788619928.468389:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f26680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571648640:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619928.468777:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f26680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571648640:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 387us (619us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619928.468784:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 661 00002000:00100000:2.0:1788619928.468793:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1482752->1486847) req@ffff8e77c5f26680 x1875503571648640/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/4536 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619928.468800:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5f26680 with x1875503571648640 ext(1482752->1486847) 00002000:00010000:2.0:1788619928.468804:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5f26680 00000100:00100000:1.0:1788619928.473796:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619928.474021:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571648768 00000100:00100000:0.0:1788619928.474032:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571648768 00000100:00100000:0.0:1788619928.474098:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1409 00000100:00100000:0.0:1788619928.474108:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5019c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+14:14908:x1875503571648768:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619928.474127:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619928.474130:0:5844:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:0.0:1788619928.474175:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079000/0x1cd7fe536873630f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d441 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.474193:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619928.474199:0:5844:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788619928.474215:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe536873631d lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d44f expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.474240:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4600/0x1cd7fe53687362b4 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d3ed expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.474249:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe536873631d lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d44f expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.474283:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4600/0x1cd7fe53687362b4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d3ed expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.474297:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4600/0x1cd7fe53687362b4 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d3ed expref: 13 pid: 5844 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619928.474311:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6530000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589221760:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619928.474595:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590b40 going to sleep for 11 seconds 00000100:00100000:2.0:1788619928.477256:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6530000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589221760:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619928.477293:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe536873631d lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d44f expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.477312:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619928.477315:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619928.477320:0:5844:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe536873631d lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d44f expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.477354:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d4c00, rc 0) 00000100:00100000:2.0:1788619928.477418:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5019c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+14:14908:x1875503571648768:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3312us (3633us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619928.477427:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1409 00000100:00100000:2.0:1788619928.478603:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.478673:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571648896 00000100:00100000:3.0:1788619928.478682:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571648896 00010000:00010000:3.0:1788619928.478720:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4600/0x1cd7fe53687362b4 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d3ed expref: 14 pid: 5844 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619928.478759:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 466 00000100:00100000:3.0:1788619928.478769:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcac850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571648896:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619928.478783:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619928.478809:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4600/0x1cd7fe53687362b4 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d3ed expref: 14 pid: 5844 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.478823:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4600/0x1cd7fe53687362b4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d3ed expref: 14 pid: 5844 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.478844:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4600/0x1cd7fe53687362b4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d3ed expref: 14 pid: 5844 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.478870:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619928.478877:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe536873631d lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d44f expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.478893:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe536873631d lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d44f expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.478911:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe536873631d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d44f expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.478950:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe536873631d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d44f expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619928.478962:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63f8700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589221888:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619928.479034:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c780 going to sleep for 11 seconds 00000100:00100000:2.0:1788619928.481219:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63f8700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589221888:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619928.481244:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619928.481300:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcac850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571648896:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 2530us (2713us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619928.481311:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 466 00000100:00100000:1.0:1788619928.483832:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619928.483945:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571649024 00000100:00100000:0.0:1788619928.483955:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571649024 00000100:00100000:0.0:1788619928.484028:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 858 00000100:00100000:0.0:1788619928.484039:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+14:2355:x1875503571649024:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:0.0:1788619928.484232:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 794416 -> 794416 00000004:00010000:0.0:1788619928.484236:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619928 -> 1788619928 00000004:00010000:0.0:1788619928.484239:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619927 -> 1788619927 00000004:00010000:0.0:1788619928.484241:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619928 -> 1788619928 00000004:00010000:0.0:1788619928.484244:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1158 -> 1158 00000100:00100000:0.0:1788619928.484318:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+15:2355:x1875503571649024:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 281us (495us total) trans 4294967797 rc 0/0 00000100:00100000:0.0:1788619928.484367:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 858 00000100:00100000:1.0:1788619928.488382:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.488527:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571649152 00000100:00100000:3.0:1788619928.488538:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571649152 00000100:00100000:3.0:1788619928.488603:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1410 00000100:00100000:3.0:1788619928.488614:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571649152:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619928.488636:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619928.488658:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71200/0x1cd7fe5368736324 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d456 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.488695:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe536873631d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d44f expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619928.488738:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63f8700 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589222016:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619928.488817:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c780 going to sleep for 11 seconds 00000100:00100000:2.0:1788619928.490419:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63f8700 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589222016:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619928.490444:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 794416 00010000:00010000:2.0:1788619928.490463:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71200/0x1cd7fe5368736324 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d456 expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.490483:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619928.490486:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619928.490492:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71200/0x1cd7fe5368736324 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d456 expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.490506:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f71200, rc 0) 00000100:00100000:2.0:1788619928.490566:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571649152:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 1954us (2193us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619928.490574:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1410 00000100:00100000:1.0:1788619928.493387:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619928.493548:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571649280 00000100:00100000:0.0:1788619928.493558:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571649280 00000100:00100000:0.0:1788619928.493619:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1155 00000100:00100000:0.0:1788619928.493629:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:14908:x1875503571649280:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619928.493646:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619928.493663:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dbe89f01/0x1cd7fe536873632b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d45d expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.493691:0:6696:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000029d0975e/0x1cd7fe53687362fa lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1744896->1748991) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d433 expref: 7 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.493701:0:6696:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000b8dcc7dd/0x1cd7fe5368736316 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d448 expref: 14 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.493709:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 12) 00010000:00010000:0.0:1788619928.493713:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 12) 00010000:00010000:0.0:1788619928.493718:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dbe89f01/0x1cd7fe536873632b lrc: 5/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [1409024->1413119] (req 1409024->1413119) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d45d expref: 7 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.493748:0:6696:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000b8dcc7dd/0x1cd7fe5368736316 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d448 expref: 14 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.493761:0:6696:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000b8dcc7dd/0x1cd7fe5368736316 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d448 expref: 14 pid: 6698 timeout: 521 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619928.493780:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6530e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589222144:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619928.493868:0:6696:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000029d0975e/0x1cd7fe53687362fa lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1744896->1748991) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d433 expref: 7 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.493878:0:6696:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000029d0975e/0x1cd7fe53687362fa lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1744896->1748991) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d433 expref: 7 pid: 7654 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619928.493887:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6533800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589222272:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619928.495212:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590780 going to sleep for 11 seconds 00000100:00100000:3.0:1788619928.497401:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619928.497467:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571649408 00000100:00100000:1.0:1788619928.497475:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571649408 00010000:00010000:1.0:1788619928.497508:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000b8dcc7dd/0x1cd7fe5368736316 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d448 expref: 15 pid: 6698 timeout: 521 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619928.497549:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 467 00000100:00100000:1.0:1788619928.497559:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63f8700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+15:2355:x1875503571649408:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619928.497575:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619928.497610:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000b8dcc7dd/0x1cd7fe5368736316 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d448 expref: 15 pid: 6698 timeout: 521 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619928.497624:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000b8dcc7dd/0x1cd7fe5368736316 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d448 expref: 15 pid: 6698 timeout: 521 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619928.497647:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619928.497703:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63f8700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+15:2355:x1875503571649408:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 145us (309us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619928.497710:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 467 00010000:00010000:0.0:1788619928.497841:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8dcc7dd/0x1cd7fe5368736316 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d448 expref: 14 pid: 6698 timeout: 521 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619928.497867:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6530e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589222144:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619928.499660:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619928.499683:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6533800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589222272:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619928.499721:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000dbe89f01/0x1cd7fe536873632b lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1409024->1413119] (req 1409024->1413119) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d45d expref: 7 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.499739:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d5a00, rc 0) 00000100:00100000:0.0:1788619928.499792:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:14908:x1875503571649280:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6163us (6413us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619928.499800:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1155 00000100:00100000:3.0:1788619928.499817:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571649536 00000100:00100000:3.0:1788619928.499824:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571649536 00010000:00010000:3.0:1788619928.499852:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000029d0975e/0x1cd7fe53687362fa lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1744896->1748991) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d433 expref: 7 pid: 7654 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619928.500093:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 468 00000100:00100000:3.0:1788619928.500101:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571649536:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619928.500111:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619928.500127:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000029d0975e/0x1cd7fe53687362fa lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1744896->1748991) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d433 expref: 7 pid: 7654 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.500135:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000029d0975e/0x1cd7fe53687362fa lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1744896->1748991) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d433 expref: 7 pid: 7654 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.500150:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029d0975e/0x1cd7fe53687362fa lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1744896->1748991) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d433 expref: 7 pid: 7654 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.500160:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 12) 00010000:00010000:3.0:1788619928.500163:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 12) 00010000:00010000:3.0:1788619928.500169:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000dbe89f01/0x1cd7fe536873632b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1409024->1413119) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d45d expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.500175:0:13330:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dbe89f01/0x1cd7fe536873632b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1409024->1413119) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d45d expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.500200:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000dbe89f01/0x1cd7fe536873632b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1409024->1413119) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d45d expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619928.500210:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df386680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589222400:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619928.500270:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c300 going to sleep for 11 seconds 00000100:00100000:3.0:1788619928.502383:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df386680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589222400:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619928.502409:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619928.502452:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571649536:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 2351us (2799us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619928.502460:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 468 00000100:00100000:1.0:1788619928.503821:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619928.503934:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571649664 00000100:00100000:2.0:1788619928.503942:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571649664 00002000:00100000:2.0:1788619928.503986:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1409024->1413119) req@ffff8e77c5019500 x1875503571649664/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619928.503994:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5019500 with x1875503571649664 ext(1409024->1413119) 00002000:00010000:2.0:1788619928.503998:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5019500 00000100:00100000:2.0:1788619928.504080:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 662 00000100:00100000:2.0:1788619928.504087:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5019500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571649664:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619928.504303:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5019500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571649664:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 214us (487us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619928.504310:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 662 00002000:00100000:2.0:1788619928.504316:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1409024->1413119) req@ffff8e77c5019500 x1875503571649664/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/4536 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619928.504349:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5019500 with x1875503571649664 ext(1409024->1413119) 00002000:00010000:2.0:1788619928.504354:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5019500 00000100:00100000:1.0:1788619928.507256:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619928.507629:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571649792 00000100:00100000:2.0:1788619928.507638:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571649792 00002000:00100000:2.0:1788619928.507688:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1413120->1417215) req@ffff8e77c501aa00 x1875503571649792/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619928.507699:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c501aa00 with x1875503571649792 ext(1413120->1417215) 00002000:00010000:2.0:1788619928.507705:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c501aa00 00000100:00100000:2.0:1788619928.507760:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 663 00000100:00100000:2.0:1788619928.507770:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571649792:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619928.508021:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571649792:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 252us (774us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619928.508028:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 663 00002000:00100000:2.0:1788619928.508035:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1413120->1417215) req@ffff8e77c501aa00 x1875503571649792/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/4536 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619928.508044:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c501aa00 with x1875503571649792 ext(1413120->1417215) 00002000:00010000:2.0:1788619928.508047:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c501aa00 00000100:00100000:1.0:1788619928.512648:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619928.512765:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571649920 00000100:00100000:2.0:1788619928.512775:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571649920 00002000:00100000:2.0:1788619928.512824:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1417216->1421311) req@ffff8e77c501a300 x1875503571649920/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619928.512835:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c501a300 with x1875503571649920 ext(1417216->1421311) 00002000:00010000:2.0:1788619928.512841:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c501a300 00000100:00100000:2.0:1788619928.512910:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 664 00000100:00100000:2.0:1788619928.512921:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571649920:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619928.513199:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571649920:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 279us (560us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619928.513207:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 664 00002000:00100000:2.0:1788619928.513214:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1417216->1421311) req@ffff8e77c501a300 x1875503571649920/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/4536 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619928.513223:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c501a300 with x1875503571649920 ext(1417216->1421311) 00002000:00010000:2.0:1788619928.513227:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c501a300 00000100:00100000:1.0:1788619928.516756:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.516867:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571650048 00000100:00100000:3.0:1788619928.516877:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571650048 00002000:00100000:3.0:1788619928.516922:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1421312->1425407) req@ffff8e77c501ad80 x1875503571650048/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619928.516934:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c501ad80 with x1875503571650048 ext(1421312->1425407) 00002000:00010000:3.0:1788619928.516944:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c501ad80 00000100:00100000:3.0:1788619928.516976:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 665 00000100:00100000:3.0:1788619928.516985:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571650048:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619928.517226:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571650048:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 241us (478us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619928.517233:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 665 00002000:00100000:3.0:1788619928.517240:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1421312->1425407) req@ffff8e77c501ad80 x1875503571650048/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/4536 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619928.517248:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c501ad80 with x1875503571650048 ext(1421312->1425407) 00002000:00010000:3.0:1788619928.517251:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c501ad80 00000100:00100000:1.0:1788619928.521124:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619928.521316:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571650176 00000100:00100000:2.0:1788619928.521360:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571650176 00002000:00100000:2.0:1788619928.521411:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1425408->1429503) req@ffff8e77c5018000 x1875503571650176/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619928.521421:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5018000 with x1875503571650176 ext(1425408->1429503) 00002000:00010000:2.0:1788619928.521427:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5018000 00000100:00100000:2.0:1788619928.521450:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 666 00000100:00100000:2.0:1788619928.521459:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5018000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571650176:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619928.521722:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5018000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571650176:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 263us (606us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619928.521730:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 666 00002000:00100000:2.0:1788619928.521737:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1425408->1429503) req@ffff8e77c5018000 x1875503571650176/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/4536 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619928.521744:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5018000 with x1875503571650176 ext(1425408->1429503) 00002000:00010000:2.0:1788619928.521748:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5018000 00000100:00100000:1.0:1788619928.526226:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.526636:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571650304 00000100:00100000:3.0:1788619928.526646:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571650304 00002000:00100000:3.0:1788619928.526709:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1409024->1429503) req@ffff8e77c501b800 x1875503571650304/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619928.526721:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c501b800 with x1875503571650304 ext(1409024->1429503) 00002000:00010000:3.0:1788619928.526727:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c501b800 00000100:00100000:3.0:1788619928.526762:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 667 00000100:00100000:3.0:1788619928.526771:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571650304:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:3.0:1788619928.530043:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571650304:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 3261us (3814us total) trans 4294967495 rc 0/0 00000100:00100000:3.0:1788619928.530057:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 667 00002000:00100000:3.0:1788619928.530070:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1409024->1429503) req@ffff8e77c501b800 x1875503571650304/t4294967495(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/448 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619928.530079:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c501b800 with x1875503571650304 ext(1409024->1429503) 00002000:00010000:3.0:1788619928.530086:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c501b800 00000100:00100000:1.0:1788619928.532358:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.532433:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571650432 00000100:00100000:3.0:1788619928.532443:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571650432 00000100:00100000:3.0:1788619928.532495:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1156 00000100:00100000:3.0:1788619928.532575:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5018e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571650432:12345-192.168.201.5@tcp:16:fsx.0 00000001:00080000:1.0:1788619928.579822:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967490 is committed 00000001:00080000:1.0:1788619928.579856:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967494 is committed 00000001:00080000:1.0:1788619928.579868:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967495 is committed 00000100:00100000:2.0:1788619928.580157:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5018e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571650432:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 47648us (47802us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619928.580170:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1156 00000100:00100000:0.0:1788619928.582910:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619928.583255:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571650560 00000100:00100000:1.0:1788619928.583266:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571650560 00000100:00100000:1.0:1788619928.583415:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1411 00000100:00100000:1.0:1788619928.583427:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6533800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571650560:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619928.583447:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619928.583469:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368736332 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d464 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619928.583630:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe536873631d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d44f expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619928.583672:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5018380 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589222528:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619928.583802:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619928.585952:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5018380 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589222528:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619928.585975:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 794416 00010000:00010000:2.0:1788619928.585992:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368736332 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d464 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.586011:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619928.586014:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619928.586019:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe5368736332 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d464 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.586033:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079400, rc 0) 00000100:00100000:2.0:1788619928.586163:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6533800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571650560:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2738us (3262us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619928.586171:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1411 00000100:00100000:0.0:1788619928.588917:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619928.589054:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571650688 00000100:00100000:2.0:1788619928.589062:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571650688 00000100:00100000:2.0:1788619928.589116:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1412 00000100:00100000:2.0:1788619928.589126:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6530e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571650688:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619928.589144:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619928.589163:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7000/0x1cd7fe5368736339 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d46b expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.589198:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe536873631d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d44f expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619928.589240:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010ff480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589222656:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619928.589309:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164300 going to sleep for 11 seconds 00000100:00100000:2.0:1788619928.591724:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010ff480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589222656:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619928.591753:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 794416 00010000:00010000:2.0:1788619928.591774:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7000/0x1cd7fe5368736339 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d46b expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.591795:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619928.591798:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619928.591803:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7000/0x1cd7fe5368736339 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d46b expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.591817:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d7000, rc 0) 00000100:00100000:2.0:1788619928.591990:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6530e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571650688:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2864us (3082us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619928.592000:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1412 00000100:00100000:0.0:1788619928.595079:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619928.595165:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571650816 00000100:00100000:2.0:1788619928.595173:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571650816 00000100:00100000:2.0:1788619928.595261:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1157 00000100:00100000:2.0:1788619928.595271:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6530700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571650816:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619928.595288:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619928.595304:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e4266ce/0x1cd7fe5368736340 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d472 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619928.595400:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010ff480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589222784:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619928.595494:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164300 going to sleep for 11 seconds 00002000:00010000:2.0:1788619928.597549:0:7654:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619927 -> 1788619928 00002000:00010000:2.0:1788619928.597560:0:7654:0:(ofd_lvb.c:284:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks: 135 -> 263 00000100:00100000:2.0:1788619928.597593:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010ff480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589222784:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619928.597631:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003e4266ce/0x1cd7fe5368736340 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d472 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.597728:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e4266ce/0x1cd7fe5368736340 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d472 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.597742:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d6200, rc 0) 00000100:00100000:2.0:1788619928.597797:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6530700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571650816:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2527us (2725us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619928.597806:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1157 00000100:00100000:3.0:1788619928.600477:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619928.600570:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571650944 00000100:00100000:1.0:1788619928.600579:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571650944 00000100:00100000:1.0:1788619928.600646:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1158 00000100:00100000:1.0:1788619928.600663:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df386680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571650944:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619928.600679:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619928.600696:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000855993/0x1cd7fe5368736347 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d479 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619928.600751:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5018380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589222912:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619928.600830:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccc00 going to sleep for 11 seconds 00000100:00100000:1.0:1788619928.603188:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5018380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589222912:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619928.603224:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000000855993/0x1cd7fe5368736347 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d479 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619928.603248:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000855993/0x1cd7fe5368736347 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d479 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619928.603261:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079c00, rc 0) 00000100:00100000:1.0:1788619928.603321:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df386680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571650944:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2659us (2851us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619928.603351:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1158 00000100:00100000:0.0:1788619928.606644:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.606742:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571651072 00000100:00100000:3.0:1788619928.606751:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571651072 00000100:00100000:3.0:1788619928.606806:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1413 00000100:00100000:3.0:1788619928.606816:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6531500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571651072:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619928.606836:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619928.606868:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70200/0x1cd7fe536873634e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d480 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.606903:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe536873631d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d44f expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619928.606943:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df384700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589223040:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619928.607027:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48cc00 going to sleep for 11 seconds 00000004:00010000:3.0:1788619928.609385:0:5845:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619927 -> 1788619928 00000100:00100000:3.0:1788619928.609413:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df384700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589223040:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619928.609433:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 794416 00010000:00010000:3.0:1788619928.609450:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70200/0x1cd7fe536873634e lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d480 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.609469:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619928.609471:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619928.609477:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70200/0x1cd7fe536873634e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d480 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.609490:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f70200, rc 0) 00000100:00100000:3.0:1788619928.609536:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6531500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571651072:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2722us (2903us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619928.609543:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1413 00000100:00100000:1.0:1788619928.612919:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619928.613095:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571651200 00000100:00100000:0.0:1788619928.613105:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571651200 00002000:00100000:0.0:1788619928.613166:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1359872->1363967) req@ffff8e77c5018380 x1875503571651200/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619928.613177:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5018380 with x1875503571651200 ext(1359872->1363967) 00002000:00010000:0.0:1788619928.613183:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5018380 00000100:00100000:0.0:1788619928.613221:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 668 00000100:00100000:0.0:1788619928.613230:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5018380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571651200:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619928.613521:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5018380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571651200:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 290us (617us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619928.613529:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 668 00002000:00100000:0.0:1788619928.613537:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1359872->1363967) req@ffff8e77c5018380 x1875503571651200/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/4536 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619928.613545:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5018380 with x1875503571651200 ext(1359872->1363967) 00002000:00010000:0.0:1788619928.613549:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5018380 00000100:00100000:3.0:1788619928.617477:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619928.617669:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571651328 00000100:00100000:2.0:1788619928.617680:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571651328 00002000:00100000:2.0:1788619928.617725:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1363968->1368063) req@ffff8e78df384700 x1875503571651328/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619928.617737:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df384700 with x1875503571651328 ext(1363968->1368063) 00002000:00010000:2.0:1788619928.617742:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df384700 00000100:00100000:2.0:1788619928.617768:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 669 00000100:00100000:2.0:1788619928.617776:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df384700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571651328:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619928.618046:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df384700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571651328:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 270us (576us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619928.618056:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 669 00002000:00100000:2.0:1788619928.618062:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1363968->1368063) req@ffff8e78df384700 x1875503571651328/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/4536 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619928.618070:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df384700 with x1875503571651328 ext(1363968->1368063) 00002000:00010000:2.0:1788619928.618074:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df384700 00000100:00100000:3.0:1788619928.621740:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619928.621839:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571651456 00000100:00100000:2.0:1788619928.621849:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571651456 00002000:00100000:2.0:1788619928.621915:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1368064->1372159) req@ffff8e78df385500 x1875503571651456/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619928.621927:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df385500 with x1875503571651456 ext(1368064->1372159) 00002000:00010000:2.0:1788619928.621933:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df385500 00000100:00100000:2.0:1788619928.622006:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 670 00000100:00100000:2.0:1788619928.622016:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df385500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571651456:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619928.622271:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df385500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571651456:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 256us (540us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619928.622280:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 670 00002000:00100000:2.0:1788619928.622287:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1368064->1372159) req@ffff8e78df385500 x1875503571651456/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/4536 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619928.622295:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df385500 with x1875503571651456 ext(1368064->1372159) 00002000:00010000:2.0:1788619928.622298:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df385500 00000100:00100000:1.0:1788619928.625982:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.626091:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571651584 00000100:00100000:3.0:1788619928.626101:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571651584 00002000:00100000:3.0:1788619928.626148:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1372160->1376255) req@ffff8e77c6002300 x1875503571651584/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619928.626160:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6002300 with x1875503571651584 ext(1372160->1376255) 00002000:00010000:3.0:1788619928.626165:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6002300 00000100:00100000:3.0:1788619928.626233:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 671 00000100:00100000:3.0:1788619928.626242:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6002300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571651584:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619928.626518:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6002300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571651584:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 275us (681us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619928.626526:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 671 00002000:00100000:3.0:1788619928.626534:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1372160->1376255) req@ffff8e77c6002300 x1875503571651584/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/4536 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619928.626543:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6002300 with x1875503571651584 ext(1372160->1376255) 00002000:00010000:3.0:1788619928.626547:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6002300 00000100:00100000:1.0:1788619928.629922:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619928.630027:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571651712 00000100:00100000:2.0:1788619928.630037:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571651712 00002000:00100000:2.0:1788619928.630087:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1376256->1380351) req@ffff8e77c6003b80 x1875503571651712/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619928.630104:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6003b80 with x1875503571651712 ext(1376256->1380351) 00002000:00010000:2.0:1788619928.630109:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6003b80 00000100:00100000:2.0:1788619928.630179:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 672 00000100:00100000:2.0:1788619928.630190:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6003b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571651712:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619928.630432:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6003b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571651712:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 243us (518us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619928.630440:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 672 00002000:00100000:2.0:1788619928.630448:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1376256->1380351) req@ffff8e77c6003b80 x1875503571651712/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/4536 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619928.630455:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6003b80 with x1875503571651712 ext(1376256->1380351) 00002000:00010000:2.0:1788619928.630459:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6003b80 00000100:00100000:0.0:1788619928.634092:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.634197:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571651840 00000100:00100000:3.0:1788619928.634207:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571651840 00002000:00100000:3.0:1788619928.634258:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1380352->1384447) req@ffff8e77c6532d80 x1875503571651840/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619928.634270:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6532d80 with x1875503571651840 ext(1380352->1384447) 00002000:00010000:3.0:1788619928.634276:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6532d80 00000100:00100000:3.0:1788619928.634360:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 673 00000100:00100000:3.0:1788619928.634369:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6532d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571651840:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619928.634639:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6532d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571651840:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 270us (555us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619928.634646:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 673 00002000:00100000:3.0:1788619928.634654:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1380352->1384447) req@ffff8e77c6532d80 x1875503571651840/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/4536 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619928.634662:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6532d80 with x1875503571651840 ext(1380352->1384447) 00002000:00010000:3.0:1788619928.634666:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6532d80 00000100:00100000:1.0:1788619928.638111:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619928.638219:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571651968 00000100:00100000:2.0:1788619928.638228:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571651968 00002000:00100000:2.0:1788619928.638277:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1384448->1388543) req@ffff8e77c6003480 x1875503571651968/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619928.638288:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6003480 with x1875503571651968 ext(1384448->1388543) 00002000:00010000:2.0:1788619928.638293:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6003480 00000100:00100000:2.0:1788619928.638341:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 674 00000100:00100000:2.0:1788619928.638351:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6003480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571651968:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619928.638718:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6003480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571651968:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 368us (615us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619928.638726:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 674 00002000:00100000:2.0:1788619928.638734:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1384448->1388543) req@ffff8e77c6003480 x1875503571651968/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/4536 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619928.638742:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6003480 with x1875503571651968 ext(1384448->1388543) 00002000:00010000:2.0:1788619928.638746:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6003480 00000100:00100000:1.0:1788619928.641735:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.641819:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571652096 00000100:00100000:3.0:1788619928.641829:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571652096 00002000:00100000:3.0:1788619928.641928:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1388544->1392639) req@ffff8e77c6001180 x1875503571652096/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619928.641940:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6001180 with x1875503571652096 ext(1388544->1392639) 00002000:00010000:3.0:1788619928.641945:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6001180 00000100:00100000:3.0:1788619928.642023:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 675 00000100:00100000:3.0:1788619928.642033:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6001180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571652096:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619928.642211:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6001180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571652096:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 179us (484us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619928.642218:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 675 00002000:00100000:3.0:1788619928.642224:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1388544->1392639) req@ffff8e77c6001180 x1875503571652096/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/4536 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619928.642233:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6001180 with x1875503571652096 ext(1388544->1392639) 00002000:00010000:3.0:1788619928.642236:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6001180 00000100:00100000:0.0:1788619928.645481:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619928.645596:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571652224 00000100:00100000:2.0:1788619928.645607:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571652224 00002000:00100000:2.0:1788619928.645658:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1392640->1396735) req@ffff8e77c6533480 x1875503571652224/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619928.645670:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6533480 with x1875503571652224 ext(1392640->1396735) 00002000:00010000:2.0:1788619928.645677:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6533480 00000100:00100000:2.0:1788619928.645753:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 676 00000100:00100000:2.0:1788619928.645764:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6533480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571652224:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619928.646032:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6533480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571652224:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 269us (559us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619928.646039:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 676 00002000:00100000:2.0:1788619928.646046:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1392640->1396735) req@ffff8e77c6533480 x1875503571652224/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/4536 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619928.646054:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6533480 with x1875503571652224 ext(1392640->1396735) 00002000:00010000:2.0:1788619928.646057:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6533480 00000100:00100000:1.0:1788619928.649477:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.649613:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571652352 00000100:00100000:3.0:1788619928.649623:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571652352 00002000:00100000:3.0:1788619928.649674:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1396736->1400831) req@ffff8e77c6000380 x1875503571652352/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619928.649685:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6000380 with x1875503571652352 ext(1396736->1400831) 00002000:00010000:3.0:1788619928.649691:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6000380 00000100:00100000:3.0:1788619928.649727:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 677 00000100:00100000:3.0:1788619928.649737:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6000380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571652352:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619928.649943:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6000380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571652352:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 207us (473us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619928.649951:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 677 00002000:00100000:3.0:1788619928.649958:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1396736->1400831) req@ffff8e77c6000380 x1875503571652352/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/4536 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619928.649966:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6000380 with x1875503571652352 ext(1396736->1400831) 00002000:00010000:3.0:1788619928.649970:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6000380 00000100:00100000:1.0:1788619928.653600:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619928.653706:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571652480 00000100:00100000:2.0:1788619928.653716:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571652480 00002000:00100000:2.0:1788619928.653763:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1400832->1404927) req@ffff8e77c6001880 x1875503571652480/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619928.653774:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6001880 with x1875503571652480 ext(1400832->1404927) 00002000:00010000:2.0:1788619928.653780:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6001880 00000100:00100000:2.0:1788619928.653864:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 678 00000100:00100000:2.0:1788619928.653873:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6001880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571652480:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619928.654059:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6001880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571652480:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 187us (479us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619928.654066:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 678 00002000:00100000:2.0:1788619928.654073:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1400832->1404927) req@ffff8e77c6001880 x1875503571652480/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/4536 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619928.654081:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6001880 with x1875503571652480 ext(1400832->1404927) 00002000:00010000:2.0:1788619928.654084:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6001880 00000100:00100000:0.0:1788619928.657201:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.657286:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571652608 00000100:00100000:3.0:1788619928.657296:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571652608 00002000:00100000:3.0:1788619928.657365:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1404928->1409023) req@ffff8e77c6532300 x1875503571652608/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619928.657376:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6532300 with x1875503571652608 ext(1404928->1409023) 00002000:00010000:3.0:1788619928.657381:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6532300 00000100:00100000:3.0:1788619928.657415:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 679 00000100:00100000:3.0:1788619928.657424:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6532300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571652608:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619928.657628:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6532300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571652608:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 205us (436us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619928.657636:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 679 00002000:00100000:3.0:1788619928.657643:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1404928->1409023) req@ffff8e77c6532300 x1875503571652608/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/4536 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619928.657651:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6532300 with x1875503571652608 ext(1404928->1409023) 00002000:00010000:3.0:1788619928.657656:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6532300 00000100:00100000:1.0:1788619928.661497:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.661619:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571652736 00000100:00100000:3.0:1788619928.661629:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571652736 00002000:00100000:3.0:1788619928.661678:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1359872->1417215) req@ffff8e77c6002a00 x1875503571652736/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619928.661690:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6002a00 with x1875503571652736 ext(1359872->1417215) 00002000:00010000:3.0:1788619928.661695:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6002a00 00000100:00100000:3.0:1788619928.661728:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 680 00000100:00100000:3.0:1788619928.661738:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6002a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571652736:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:2.0:1788619928.669886:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6002a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571652736:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 8139us (8389us total) trans 4294967496 rc 0/0 00000100:00100000:2.0:1788619928.669900:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 680 00002000:00100000:2.0:1788619928.669913:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1359872->1417215) req@ffff8e77c6002a00 x1875503571652736/t4294967496(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/448 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619928.669923:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6002a00 with x1875503571652736 ext(1359872->1417215) 00002000:00010000:2.0:1788619928.669929:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6002a00 00000100:00100000:1.0:1788619928.673166:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619928.673267:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571652864 00000100:00100000:0.0:1788619928.673276:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571652864 00000100:00100000:0.0:1788619928.673370:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1159 00000100:00100000:0.0:1788619928.673380:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6000000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571652864:12345-192.168.201.5@tcp:16:fsx.0 00000100:00100000:0.0:1788619928.701519:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6000000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571652864:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 28130us (28351us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619928.701534:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1159 00000001:00080000:1.0:1788619928.701756:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967496 is committed 00000100:00100000:1.0:1788619928.704418:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.704558:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571652992 00000100:00100000:3.0:1788619928.704568:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571652992 00000100:00100000:3.0:1788619928.704630:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1414 00000100:00100000:3.0:1788619928.704639:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6001f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571652992:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619928.704659:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619928.704680:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71000/0x1cd7fe5368736355 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d487 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.704716:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe536873631d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d44f expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619928.704755:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df385f80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589223168:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619928.704836:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c780 going to sleep for 11 seconds 00000100:00100000:3.0:1788619928.707243:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df385f80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589223168:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619928.707268:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 794416 00010000:00010000:3.0:1788619928.707289:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71000/0x1cd7fe5368736355 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d487 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.707314:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619928.707317:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619928.707344:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71000/0x1cd7fe5368736355 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d487 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.707359:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f71000, rc 0) 00000100:00100000:3.0:1788619928.707424:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6001f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571652992:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2785us (3028us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619928.707433:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1414 00000100:00100000:1.0:1788619928.710539:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619928.710655:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571653120 00000100:00100000:2.0:1788619928.710665:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571653120 00000100:00100000:2.0:1788619928.710762:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1415 00000100:00100000:2.0:1788619928.710774:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6000a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571653120:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619928.710794:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619928.710814:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6200/0x1cd7fe536873635c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d48e expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.710871:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe536873631d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d44f expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619928.710924:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010ff480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589223296:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619928.711013:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164480 going to sleep for 11 seconds 00000100:00100000:2.0:1788619928.713204:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010ff480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589223296:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619928.713229:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 794416 00010000:00010000:2.0:1788619928.713247:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6200/0x1cd7fe536873635c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d48e expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.713266:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619928.713269:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619928.713275:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6200/0x1cd7fe536873635c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d48e expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.713289:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d6200, rc 0) 00000100:00100000:2.0:1788619928.713376:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6000a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571653120:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2604us (2846us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619928.713383:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1415 00000100:00100000:3.0:1788619928.716074:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619928.716667:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571653248 00000100:00100000:0.0:1788619928.716678:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571653248 00000100:00100000:1.0:1788619928.716865:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1416 00000100:00100000:1.0:1788619928.716877:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df385f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571653248:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619928.716898:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619928.716932:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8078200/0x1cd7fe5368736363 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d495 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619928.716972:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe536873631d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d44f expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619928.717012:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6000e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589223424:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619928.717100:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebcc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619928.719993:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6000e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589223424:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619928.720016:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 794416 00010000:00010000:2.0:1788619928.720034:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8078200/0x1cd7fe5368736363 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d495 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.720055:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619928.720058:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619928.720064:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8078200/0x1cd7fe5368736363 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d495 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.720078:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8078200, rc 0) 00000100:00100000:2.0:1788619928.720135:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df385f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571653248:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3259us (4068us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619928.720143:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1416 00000100:00100000:3.0:1788619928.722912:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619928.722997:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571653376 00000100:00100000:2.0:1788619928.723007:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571653376 00000100:00100000:2.0:1788619928.723063:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1417 00000100:00100000:2.0:1788619928.723072:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df387100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571653376:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619928.723091:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619928.723112:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7200/0x1cd7fe536873636a lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d49c expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.723141:0:11397:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe536873631d lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d44f expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.723151:0:11397:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7200/0x1cd7fe536873636a lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d49c expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.723182:0:11397:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe536873631d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d44f expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.723195:0:11397:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe536873631d lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d44f expref: 14 pid: 5844 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619928.723213:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010ff480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589223552:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619928.723295:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164480 going to sleep for 11 seconds 00000100:00100000:2.0:1788619928.725168:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010ff480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589223552:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619928.725203:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7200/0x1cd7fe536873636a lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d49c expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.725221:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619928.725224:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619928.725229:0:11397:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7200/0x1cd7fe536873636a lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d49c expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.725239:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d7200, rc 0) 00000100:00100000:2.0:1788619928.725289:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df387100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571653376:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2217us (2387us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619928.725298:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1417 00000100:00100000:1.0:1788619928.727240:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619928.727616:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571653504 00000100:00100000:0.0:1788619928.727626:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571653504 00010000:00010000:0.0:1788619928.727668:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe536873631d lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d44f expref: 15 pid: 5844 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619928.727710:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 469 00000100:00100000:0.0:1788619928.727719:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6003100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+15:2354:x1875503571653504:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619928.727733:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619928.727759:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe536873631d lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d44f expref: 15 pid: 5844 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.727773:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe536873631d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d44f expref: 15 pid: 5844 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.727793:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4c00/0x1cd7fe536873631d lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d44f expref: 15 pid: 5844 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.727811:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619928.727817:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7200/0x1cd7fe536873636a lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d49c expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.727829:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7200/0x1cd7fe536873636a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d49c expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.727839:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7200/0x1cd7fe536873636a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d49c expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.727887:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7200/0x1cd7fe536873636a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d49c expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619928.727899:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6533100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589223680:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619928.727985:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590780 going to sleep for 11 seconds 00000100:00100000:0.0:1788619928.730774:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6533100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589223680:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619928.730800:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619928.730858:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6003100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+14:2354:x1875503571653504:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3138us (3625us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619928.730868:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 469 00000100:00100000:1.0:1788619928.732818:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.733001:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571653632 00000100:00100000:3.0:1788619928.733010:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571653632 00002000:00100000:3.0:1788619928.733059:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6002680 x1875503571653632/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619928.733070:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6002680 with x1875503571653632 00000100:00100000:3.0:1788619928.733107:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 859 00000100:00100000:3.0:1788619928.733116:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6002680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571653632:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619928.733356:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6002680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571653632:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 240us (544us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619928.733364:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 859 00002000:00100000:3.0:1788619928.733371:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6002680 x1875503571653632/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/4536 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619928.733378:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6002680 with x1875503571653632 00000100:00100000:1.0:1788619928.737384:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619928.737503:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571653760 00000100:00100000:2.0:1788619928.737512:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571653760 00002000:00100000:2.0:1788619928.737561:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6003800 x1875503571653760/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619928.737572:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6003800 with x1875503571653760 00000100:00100000:2.0:1788619928.737608:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 860 00000100:00100000:2.0:1788619928.737617:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6003800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571653760:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619928.737827:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6003800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571653760:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 210us (454us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619928.737837:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 860 00002000:00100000:2.0:1788619928.737844:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6003800 x1875503571653760/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/4536 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619928.737865:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6003800 with x1875503571653760 00000100:00100000:1.0:1788619928.745237:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619928.745370:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571653888 00000100:00100000:0.0:1788619928.745380:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571653888 00002000:00100000:0.0:1788619928.745440:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6000700 x1875503571653888/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619928.745450:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6000700 with x1875503571653888 00000100:00100000:0.0:1788619928.745486:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 861 00000100:00100000:0.0:1788619928.745495:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6000700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2353:x1875503571653888:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619928.745706:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6000700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2353:x1875503571653888:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 211us (479us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619928.745714:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 861 00002000:00100000:0.0:1788619928.745720:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6000700 x1875503571653888/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/4536 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619928.745728:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6000700 with x1875503571653888 00000100:00100000:0.0:1788619928.749147:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.749241:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571654016 00000100:00100000:3.0:1788619928.749251:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571654016 00002000:00100000:3.0:1788619928.749298:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6533100 x1875503571654016/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619928.749309:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6533100 with x1875503571654016 00000100:00100000:3.0:1788619928.749382:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 862 00000100:00100000:3.0:1788619928.749392:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6533100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2352:x1875503571654016:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619928.749615:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6533100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2352:x1875503571654016:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 223us (479us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619928.749623:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 862 00002000:00100000:3.0:1788619928.749629:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6533100 x1875503571654016/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/4536 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619928.749637:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6533100 with x1875503571654016 00000100:00100000:0.0:1788619928.753670:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619928.754041:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571654144 00000100:00100000:2.0:1788619928.754052:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571654144 00002000:00100000:2.0:1788619928.754101:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6530380 x1875503571654144/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 16872/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619928.754112:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6530380 with x1875503571654144 00000100:00100000:2.0:1788619928.754191:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 863 00000100:00100000:2.0:1788619928.754201:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6530380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2353:x1875503571654144:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:2.0:1788619928.754605:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 794416 -> 794416 00000004:00010000:2.0:1788619928.754609:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619928 -> 1788619928 00000004:00010000:2.0:1788619928.754611:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619928 -> 1788619927 00000004:00010000:2.0:1788619928.754614:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619928 -> 1788619928 00000004:00010000:2.0:1788619928.754617:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1158 -> 1158 00000100:00100000:2.0:1788619928.754726:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6530380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2353:x1875503571654144:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 525us (1065us total) trans 4294967798 rc 0/0 00000100:00100000:2.0:1788619928.754734:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 863 00002000:00100000:2.0:1788619928.754742:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6530380 x1875503571654144/t4294967798(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 16872/448 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619928.754751:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6530380 with x1875503571654144 00000100:00100000:1.0:1788619928.758171:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.758262:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571654272 00000100:00100000:3.0:1788619928.758272:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571654272 00000100:00100000:3.0:1788619928.758387:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1418 00000100:00100000:3.0:1788619928.758398:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6001c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571654272:12345-192.168.201.5@tcp:16:fsx.0 00000001:00080000:2.0:1788619928.780715:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967797 is committed 00000001:00080000:2.0:1788619928.780740:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967798 is committed 00000100:00100000:0.0:1788619928.781093:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6001c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571654272:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 22691us (22926us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619928.781108:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1418 00000100:00100000:1.0:1788619928.784912:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619928.784956:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619928.785060:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571654400 00000100:00100000:2.0:1788619928.785070:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571654400 00000100:00100000:2.0:1788619928.785137:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1419 00000100:00100000:2.0:1788619928.785148:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6002d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571654400:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619928.785172:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619928.785191:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6200/0x1cd7fe5368736371 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d4a3 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.785227:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7200/0x1cd7fe536873636a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d49c expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619928.785266:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010ff480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589223808:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619928.785358:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164600 going to sleep for 11 seconds 00000004:00010000:2.0:1788619928.787369:0:5846:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619927 -> 1788619928 00000100:00100000:2.0:1788619928.787399:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010ff480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589223808:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619928.787433:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 794416 00010000:00010000:2.0:1788619928.787451:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6200/0x1cd7fe5368736371 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4a3 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.787471:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619928.787474:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619928.787479:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6200/0x1cd7fe5368736371 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4a3 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.787493:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d6200, rc 0) 00000100:00100000:2.0:1788619928.787541:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6002d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571654400:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2396us (2695us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619928.787553:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1419 00000100:00100000:0.0:1788619928.789025:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571654528 00000100:00100000:0.0:1788619928.789037:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571654528 00000100:00100000:0.0:1788619928.789136:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1160 00000100:00100000:0.0:1788619928.789146:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6000e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571654528:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619928.789161:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619928.789177:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bdacd512/0x1cd7fe5368736378 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d4aa expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619928.789235:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6531f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589223936:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619928.789318:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590d80 going to sleep for 11 seconds 00002000:00010000:0.0:1788619928.791557:0:7654:0:(ofd_lvb.c:284:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks: 263 -> 391 00000100:00100000:0.0:1788619928.791597:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6531f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589223936:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619928.791628:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bdacd512/0x1cd7fe5368736378 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4aa expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.791650:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bdacd512/0x1cd7fe5368736378 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4aa expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.791670:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d4c00, rc 0) 00000100:00100000:0.0:1788619928.791757:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6000e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571654528:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2611us (6799us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619928.791766:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1160 00000100:00100000:2.0:1788619928.794440:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.794627:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571654656 00000100:00100000:3.0:1788619928.794637:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571654656 00000100:00100000:3.0:1788619928.794697:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1420 00000100:00100000:3.0:1788619928.794707:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010ff480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571654656:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619928.794729:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619928.794751:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71000/0x1cd7fe536873637f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d4b1 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.794789:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7200/0x1cd7fe536873636a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d49c expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619928.794830:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df387800 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589224064:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619928.794931:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c780 going to sleep for 11 seconds 00000100:00100000:1.0:1788619928.797162:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.797309:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571654784 00000100:00100000:3.0:1788619928.797317:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571654784 00000100:00100000:3.0:1788619928.797416:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1161 00000100:00100000:3.0:1788619928.797426:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571654784:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619928.797441:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619928.797458:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1c6e115/0x1cd7fe5368736386 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d4b8 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619928.797512:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df385180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589224192:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619928.797656:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c300 going to sleep for 11 seconds 00000100:00100000:3.0:1788619928.801227:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df387800 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589224064:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619928.801252:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 794416 00010000:00010000:3.0:1788619928.801270:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71000/0x1cd7fe536873637f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4b1 expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.801291:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619928.801294:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619928.801300:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71000/0x1cd7fe536873637f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4b1 expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.801315:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f71000, rc 0) 00000100:00100000:3.0:1788619928.801518:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010ff480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571654656:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6810us (7087us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619928.801529:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1420 00000100:00100000:3.0:1788619928.802509:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df385180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589224192:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619928.802538:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b1c6e115/0x1cd7fe5368736386 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4b8 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.802559:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1c6e115/0x1cd7fe5368736386 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4b8 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.802571:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f70200, rc 0) 00000100:00100000:3.0:1788619928.802633:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571654784:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5207us (5477us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619928.802641:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1161 00000100:00100000:2.0:1788619928.806401:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.806507:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571654912 00000100:00100000:3.0:1788619928.806517:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571654912 00000100:00100000:3.0:1788619928.806568:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1421 00000100:00100000:3.0:1788619928.806590:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010fe680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571654912:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619928.806610:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619928.806629:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71a00/0x1cd7fe536873638d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d4bf expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.806667:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7200/0x1cd7fe536873636a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d49c expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619928.806705:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df385180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589224320:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619928.806783:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c300 going to sleep for 11 seconds 00000100:00100000:2.0:1788619928.808267:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.808369:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571655040 00000100:00100000:3.0:1788619928.808376:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571655040 00000100:00100000:3.0:1788619928.808425:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1162 00000100:00100000:3.0:1788619928.808435:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010fdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571655040:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619928.808448:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619928.808462:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d786ac5b/0x1cd7fe5368736394 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d4c6 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619928.808508:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df387800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589224448:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619928.808561:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c780 going to sleep for 11 seconds 00000100:00100000:2.0:1788619928.810725:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df387800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589224448:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619928.810772:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d786ac5b/0x1cd7fe5368736394 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4c6 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.810796:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d786ac5b/0x1cd7fe5368736394 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4c6 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.810809:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f70c00, rc 0) 00000100:00100000:2.0:1788619928.810981:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010fdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571655040:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2547us (2719us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619928.810989:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1162 00000100:00100000:2.0:1788619928.811057:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df385180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589224320:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619928.811068:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 794416 00010000:00010000:2.0:1788619928.811082:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71a00/0x1cd7fe536873638d lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4bf expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.811096:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619928.811099:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619928.811104:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71a00/0x1cd7fe536873638d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4bf expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.811120:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f71a00, rc 0) 00000100:00100000:2.0:1788619928.811143:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010fe680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571654912:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4567us (4752us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619928.811149:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1421 00000100:00100000:2.0:1788619928.814538:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619928.814652:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571655168 00000100:00100000:1.0:1788619928.814662:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571655168 00000100:00100000:1.0:1788619928.814756:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1422 00000100:00100000:1.0:1788619928.814767:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010ff100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571655168:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619928.814786:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619928.814810:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe536873639b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d4cd expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619928.814841:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7200/0x1cd7fe536873636a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d49c expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619928.814863:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe536873639b lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4cd expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619928.814897:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7200/0x1cd7fe536873636a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d49c expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619928.814920:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7200/0x1cd7fe536873636a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d49c expref: 13 pid: 11397 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619928.814933:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be2a00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589224576:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619928.815017:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb540 going to sleep for 11 seconds 00000100:00100000:1.0:1788619928.817767:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619928.817881:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be2a00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589224576:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619928.817927:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe536873639b lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4cd expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.817946:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619928.817949:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619928.817954:0:5846:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe536873639b lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4cd expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.817966:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c8079400, rc 0) 00000100:00100000:2.0:1788619928.818026:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010ff100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:14908:x1875503571655168:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3261us (3498us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619928.818034:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1422 00000100:00100000:3.0:1788619928.818834:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571655296 00000100:00100000:3.0:1788619928.818857:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571655296 00010000:00010000:3.0:1788619928.818897:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7200/0x1cd7fe536873636a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d49c expref: 14 pid: 11397 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619928.818944:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 470 00000100:00100000:3.0:1788619928.818953:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be1180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571655296:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619928.818967:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619928.818995:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7200/0x1cd7fe536873636a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d49c expref: 14 pid: 11397 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.819008:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7200/0x1cd7fe536873636a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d49c expref: 14 pid: 11397 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.819031:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7200/0x1cd7fe536873636a lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d49c expref: 14 pid: 11397 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.819052:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619928.819058:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe536873639b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4cd expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.819069:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe536873639b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4cd expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.819079:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe536873639b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4cd expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.819114:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe536873639b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4cd expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619928.819128:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df386d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589224704:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619928.819201:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c300 going to sleep for 11 seconds 00000100:00100000:3.0:1788619928.824154:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df386d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589224704:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619928.824183:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619928.824237:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be1180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571655296:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 5285us (6480us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619928.824248:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 470 00000100:00100000:3.0:1788619928.828818:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619928.828962:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571655424 00000100:00100000:2.0:1788619928.828971:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571655424 00002000:00100000:2.0:1788619928.829024:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df386d80 x1875503571655424/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619928.829036:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df386d80 with x1875503571655424 00000100:00100000:2.0:1788619928.829068:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 864 00000100:00100000:2.0:1788619928.829076:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df386d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571655424:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619928.839554:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df386d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571655424:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 10465us (10733us total) trans 0 rc 794624/794624 00000100:00100000:2.0:1788619928.839570:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 864 00002000:00100000:2.0:1788619928.839594:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df386d80 x1875503571655424/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:24/0 lens 488/440 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 794624/794624 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619928.839605:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df386d80 with x1875503571655424 00000100:00100000:1.0:1788619928.843520:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619928.843647:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571655552 00000100:00100000:0.0:1788619928.843657:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571655552 00000100:00100000:0.0:1788619928.843724:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1163 00000100:00100000:0.0:1788619928.843735:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571655552:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619928.843753:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619928.843770:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025706efa/0x1cd7fe53687363a2 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d4d4 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619928.843834:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6531f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589224832:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619928.843930:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590d80 going to sleep for 11 seconds 00000100:00100000:0.0:1788619928.846671:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6531f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589224832:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619928.846714:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000025706efa/0x1cd7fe53687363a2 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4d4 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.846742:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025706efa/0x1cd7fe53687363a2 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4d4 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.846757:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d5400, rc 0) 00000100:00100000:0.0:1788619928.846860:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571655552:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3125us (3350us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619928.846869:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1163 00000100:00100000:2.0:1788619928.849999:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619928.850088:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571655680 00000100:00100000:0.0:1788619928.850099:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571655680 00000100:00100000:0.0:1788619928.850187:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1164 00000100:00100000:0.0:1788619928.850198:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010fe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571655680:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619928.850215:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619928.850233:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bdacd512/0x1cd7fe53687363a9 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d4db expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619928.850292:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6531f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589224960:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619928.850530:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590d80 going to sleep for 11 seconds 00000100:00100000:0.0:1788619928.853111:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6531f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589224960:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619928.853151:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bdacd512/0x1cd7fe53687363a9 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4db expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.853176:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bdacd512/0x1cd7fe53687363a9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4db expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.853191:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d4c00, rc 0) 00000100:00100000:0.0:1788619928.853254:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010fe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571655680:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3058us (3265us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619928.853263:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1164 00000100:00100000:1.0:1788619928.856217:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.856310:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571655808 00000100:00100000:3.0:1788619928.856322:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571655808 00000100:00100000:3.0:1788619928.856432:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1423 00000100:00100000:3.0:1788619928.856443:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571655808:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619928.856464:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619928.856485:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70600/0x1cd7fe53687363b0 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d4e2 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.856518:0:11397:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe536873639b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4cd expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.856529:0:11397:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70600/0x1cd7fe53687363b0 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4e2 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.856567:0:11397:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe536873639b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4cd expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.856592:0:11397:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe536873639b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4cd expref: 9 pid: 5846 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619928.856606:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df387480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589225088:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619928.856692:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c300 going to sleep for 11 seconds 00000100:00100000:2.0:1788619928.858725:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df387480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589225088:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619928.858760:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70600/0x1cd7fe53687363b0 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4e2 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.858780:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619928.858782:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619928.858787:0:11397:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70600/0x1cd7fe53687363b0 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4e2 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.858796:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f70600, rc 0) 00000100:00100000:2.0:1788619928.858866:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571655808:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2425us (2660us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619928.858875:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1423 00000100:00100000:2.0:1788619928.863690:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619928.863823:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571655936 00000100:00100000:1.0:1788619928.863833:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571655936 00010000:00010000:1.0:1788619928.863886:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe536873639b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4cd expref: 10 pid: 5846 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619928.863936:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 471 00000100:00100000:1.0:1788619928.863947:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571655936:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619928.863962:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619928.863990:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe536873639b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4cd expref: 10 pid: 5846 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619928.864004:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe536873639b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4cd expref: 10 pid: 5846 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619928.864028:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c8079400/0x1cd7fe536873639b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4cd expref: 10 pid: 5846 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619928.864047:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619928.864052:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70600/0x1cd7fe53687363b0 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4e2 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619928.864066:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70600/0x1cd7fe53687363b0 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4e2 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619928.864083:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70600/0x1cd7fe53687363b0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4e2 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619928.864121:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70600/0x1cd7fe53687363b0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4e2 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619928.864135:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be3b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589225216:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619928.864223:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb000 going to sleep for 11 seconds 00000100:00100000:1.0:1788619928.867672:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619928.868109:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be3b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589225216:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619928.868135:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619928.868192:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571655936:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 4246us (4511us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619928.868206:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 471 00000100:00100000:3.0:1788619928.868764:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571656064 00000100:00100000:3.0:1788619928.868773:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571656064 00002000:00100000:3.0:1788619928.868820:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be1f80 x1875503571656064/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619928.868830:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be1f80 with x1875503571656064 00000100:00100000:3.0:1788619928.868874:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 865 00000100:00100000:3.0:1788619928.868884:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571656064:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619928.869239:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571656064:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 356us (1575us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619928.869247:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 865 00002000:00100000:3.0:1788619928.869254:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be1f80 x1875503571656064/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/4536 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619928.869261:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be1f80 with x1875503571656064 00000100:00100000:1.0:1788619928.876108:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619928.876213:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571656192 00000100:00100000:2.0:1788619928.876224:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571656192 00002000:00100000:2.0:1788619928.876286:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be0380 x1875503571656192/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619928.876298:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be0380 with x1875503571656192 00000100:00100000:2.0:1788619928.876402:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 866 00000100:00100000:2.0:1788619928.876422:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571656192:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619928.876771:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571656192:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 358us (673us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619928.876780:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 866 00002000:00100000:2.0:1788619928.876788:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be0380 x1875503571656192/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/4536 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619928.876796:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be0380 with x1875503571656192 00000100:00100000:1.0:1788619928.881828:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619928.881874:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619928.881944:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571656448 00000100:00100000:0.0:1788619928.881955:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571656448 00000100:00100000:0.0:1788619928.882014:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1165 00000100:00100000:0.0:1788619928.882024:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571656448:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619928.882043:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619928.882060:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bdacd512/0x1cd7fe53687363b7 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d4f0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619928.882118:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6531f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589225344:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619928.882187:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590d80 going to sleep for 11 seconds 00000100:00100000:3.0:1788619928.884834:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571656320 00000100:00100000:3.0:1788619928.884861:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571656320 00000100:00100000:3.0:1788619928.884980:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1424 00000100:00100000:3.0:1788619928.884990:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571656320:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619928.885011:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619928.885031:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71000/0x1cd7fe53687363be lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d4e9 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.885067:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70600/0x1cd7fe53687363b0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4e2 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619928.885112:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df385880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589225472:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619928.885210:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48cc00 going to sleep for 11 seconds 00000100:00100000:3.0:1788619928.889833:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df385880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589225472:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619928.889871:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 794416 00010000:00010000:3.0:1788619928.889891:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71000/0x1cd7fe53687363be lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4e9 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.889912:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619928.889914:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619928.889920:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71000/0x1cd7fe53687363be lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4e9 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.889939:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f71000, rc 0) 00000100:00100000:3.0:1788619928.890008:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571656320:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5018us (8191us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619928.890016:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1424 00000100:00100000:1.0:1788619928.890268:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6531f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589225344:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619928.890290:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bdacd512/0x1cd7fe53687363b7 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4f0 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619928.890308:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bdacd512/0x1cd7fe53687363b7 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4f0 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619928.890319:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d4c00, rc 0) 00000100:00100000:1.0:1788619928.890389:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571656448:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8366us (8513us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619928.890396:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1165 00000100:00100000:1.0:1788619928.894147:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619928.894248:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571656576 00000100:00100000:2.0:1788619928.894258:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571656576 00000100:00100000:2.0:1788619928.894322:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1425 00000100:00100000:2.0:1788619928.894371:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571656576:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619928.894392:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619928.894422:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7a00/0x1cd7fe53687363c5 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d4f7 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.894459:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70600/0x1cd7fe53687363b0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4e2 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619928.894503:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fc700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589225600:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619928.894600:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81649c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619928.897218:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619928.897430:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fc700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589225600:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619928.897453:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 794416 00010000:00010000:1.0:1788619928.897472:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7a00/0x1cd7fe53687363c5 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4f7 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619928.897494:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619928.897497:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619928.897503:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7a00/0x1cd7fe53687363c5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4f7 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619928.897518:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d7a00, rc 0) 00000100:00100000:1.0:1788619928.897593:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571656576:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3224us (3454us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619928.897603:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1425 00000100:00100000:3.0:1788619928.897911:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571656704 00000100:00100000:3.0:1788619928.897918:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571656704 00000100:00100000:3.0:1788619928.897959:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1166 00000100:00100000:3.0:1788619928.897967:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010fea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571656704:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619928.897979:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619928.897995:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1c6e115/0x1cd7fe53687363cc lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d4fe expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619928.898044:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df385880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589225728:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619928.898110:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48cc00 going to sleep for 11 seconds 00000100:00100000:3.0:1788619928.902376:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df385880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589225728:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619928.902427:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b1c6e115/0x1cd7fe53687363cc lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4fe expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.902453:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1c6e115/0x1cd7fe53687363cc lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4fe expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.902468:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f70200, rc 0) 00000100:00100000:3.0:1788619928.902535:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010fea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571656704:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4568us (5327us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619928.902547:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1166 00000100:00100000:1.0:1788619928.905482:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.905623:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571656832 00000100:00100000:3.0:1788619928.905632:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571656832 00000100:00100000:3.0:1788619928.905690:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1426 00000100:00100000:3.0:1788619928.905701:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571656832:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619928.905720:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619928.905742:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71000/0x1cd7fe53687363d3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d505 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.905774:0:11397:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70600/0x1cd7fe53687363b0 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4e2 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.905785:0:11397:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71000/0x1cd7fe53687363d3 lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d505 expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.905823:0:11397:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70600/0x1cd7fe53687363b0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4e2 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.905836:0:11397:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70600/0x1cd7fe53687363b0 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4e2 expref: 13 pid: 11397 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619928.905863:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df385880 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589225856:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619928.905958:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48cc00 going to sleep for 11 seconds 00000100:00100000:3.0:1788619928.909689:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df385880 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589225856:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619928.909730:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71000/0x1cd7fe53687363d3 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d505 expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.909755:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619928.909758:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619928.909763:0:11397:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71000/0x1cd7fe53687363d3 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d505 expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.909773:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f71000, rc 0) 00000100:00100000:3.0:1788619928.909836:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+10:14908:x1875503571656832:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4137us (4363us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619928.909852:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1426 00000100:00100000:1.0:1788619928.911077:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.911162:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571656960 00000100:00100000:3.0:1788619928.911170:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571656960 00002000:00100000:3.0:1788619928.911218:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be2d80 x1875503571656960/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619928.911228:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be2d80 with x1875503571656960 00010000:00010000:3.0:1788619928.911236:0:5857:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70600/0x1cd7fe53687363b0 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4e2 expref: 14 pid: 11397 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.911251:0:5857:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70600/0x1cd7fe53687363b0 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4e2 expref: 14 pid: 11397 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788619928.911258:0:5857:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e77c4be2d80 00000100:00100000:3.0:1788619928.911286:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 36 00000100:00100000:3.0:1788619928.911295:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571656960:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:0.0:1788619928.916234:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 794416 -> 794416 00000004:00010000:0.0:1788619928.916244:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619928 -> 1788619928 00000004:00010000:0.0:1788619928.916246:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619928 -> 1788619927 00000004:00010000:0.0:1788619928.916249:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619928 -> 1788619928 00000004:00010000:0.0:1788619928.916251:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1158 -> 1158 00000100:00100000:0.0:1788619928.916323:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571656960:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 5024us (5249us total) trans 4294967799 rc 0/0 00000100:00100000:0.0:1788619928.916354:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 36 00002000:00100000:0.0:1788619928.916367:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be2d80 x1875503571656960/t4294967799(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:24/0 lens 488/448 e 0 to 0 dl 1788619939 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619928.916376:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be2d80 with x1875503571656960 00010000:00010000:0.0:1788619928.916385:0:5857:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70600/0x1cd7fe53687363b0 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4e2 expref: 16 pid: 11397 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619928.916415:0:5857:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70600/0x1cd7fe53687363b0 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4e2 expref: 16 pid: 11397 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788619928.916424:0:5857:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e77c4be2d80 00000100:00100000:2.0:1788619928.920776:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.920969:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571657088 00000100:00100000:3.0:1788619928.920980:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571657088 00010000:00010000:3.0:1788619928.921022:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70600/0x1cd7fe53687363b0 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4e2 expref: 16 pid: 11397 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619928.921099:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 472 00000100:00100000:3.0:1788619928.921110:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010fd180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571657088:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619928.921125:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619928.921150:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70600/0x1cd7fe53687363b0 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4e2 expref: 16 pid: 11397 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.921165:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70600/0x1cd7fe53687363b0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4e2 expref: 16 pid: 11397 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.921188:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70600/0x1cd7fe53687363b0 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d4e2 expref: 16 pid: 11397 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.921203:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619928.921209:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71000/0x1cd7fe53687363d3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d505 expref: 9 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.921221:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71000/0x1cd7fe53687363d3 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d505 expref: 9 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.921229:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71000/0x1cd7fe53687363d3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d505 expref: 9 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.921266:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71000/0x1cd7fe53687363d3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d505 expref: 9 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619928.921285:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df385880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589225984:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619928.921600:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48cc00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619928.924491:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.924640:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df385880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589225984:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619928.924662:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619928.924705:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010fd180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571657088:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3596us (3937us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619928.924715:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 472 00000100:00100000:1.0:1788619928.925152:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571657216 00000100:00100000:1.0:1788619928.925159:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571657216 00000100:00100000:1.0:1788619928.925207:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1167 00000100:00100000:1.0:1788619928.925215:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010fc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571657216:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619928.925231:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619928.925250:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011e2507b/0x1cd7fe53687363da lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d50c expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619928.925304:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589226112:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619928.925381:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb540 going to sleep for 11 seconds 00000100:00100000:2.0:1788619928.927498:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589226112:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619928.927536:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000011e2507b/0x1cd7fe53687363da lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d50c expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.927559:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011e2507b/0x1cd7fe53687363da lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d50c expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.927572:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f73800, rc 0) 00000100:00100000:2.0:1788619928.927677:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010fc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571657216:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2461us (3195us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619928.927686:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1167 00000100:00100000:2.0:1788619928.933742:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619928.933841:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571657344 00000100:00100000:0.0:1788619928.933862:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571657344 00002000:00100000:0.0:1788619928.933920:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79010ffb80 x1875503571657344/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:24/0 lens 504/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619928.933932:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79010ffb80 with x1875503571657344 00000100:00100000:0.0:1788619928.933967:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 867 00000100:00100000:0.0:1788619928.933978:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010ffb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571657344:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619928.943189:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010ffb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571657344:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 9199us (9445us total) trans 0 rc 794624/794624 00000100:00100000:0.0:1788619928.943203:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 867 00002000:00100000:0.0:1788619928.943216:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79010ffb80 x1875503571657344/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:24/0 lens 504/440 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 794624/794624 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619928.943228:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79010ffb80 with x1875503571657344 00000100:00100000:2.0:1788619928.946376:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619928.946679:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571657472 00000100:00100000:0.0:1788619928.946688:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571657472 00002000:00100000:0.0:1788619928.946738:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79010fce00 x1875503571657472/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619928.946749:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79010fce00 with x1875503571657472 00000100:00100000:0.0:1788619928.946787:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 868 00000100:00100000:0.0:1788619928.946797:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010fce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571657472:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619928.947067:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010fce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571657472:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 270us (700us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619928.947074:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 868 00002000:00100000:0.0:1788619928.947081:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79010fce00 x1875503571657472/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:24/0 lens 488/440 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619928.947094:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79010fce00 with x1875503571657472 00000100:00100000:1.0:1788619928.949456:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619928.949601:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571657600 00000100:00100000:0.0:1788619928.949610:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571657600 00002000:00100000:0.0:1788619928.949656:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be3480 x1875503571657600/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619928.949666:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be3480 with x1875503571657600 00000100:00100000:0.0:1788619928.949702:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 869 00000100:00100000:0.0:1788619928.949711:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571657600:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619928.950109:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571657600:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 397us (660us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619928.950118:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 869 00002000:00100000:0.0:1788619928.950124:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be3480 x1875503571657600/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:24/0 lens 488/440 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619928.950132:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be3480 with x1875503571657600 00000100:00100000:1.0:1788619928.952691:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619928.952782:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571657728 00000100:00100000:2.0:1788619928.952792:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571657728 00002000:00100000:2.0:1788619928.952841:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be0a80 x1875503571657728/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619928.952913:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be0a80 with x1875503571657728 00000100:00100000:2.0:1788619928.952978:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 870 00000100:00100000:2.0:1788619928.952989:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571657728:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619928.953287:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571657728:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 298us (603us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619928.953294:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 870 00002000:00100000:2.0:1788619928.953302:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be0a80 x1875503571657728/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:24/0 lens 488/440 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619928.953309:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be0a80 with x1875503571657728 00000100:00100000:3.0:1788619928.955881:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619928.956139:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571657856 00000100:00100000:0.0:1788619928.956148:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571657856 00000100:00100000:1.0:1788619928.956293:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1168 00000100:00100000:1.0:1788619928.956304:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df385880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571657856:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619928.956323:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619928.956356:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe53687363e1 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d513 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619928.956429:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589226240:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619928.956514:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeba80 going to sleep for 11 seconds 00000100:00100000:2.0:1788619928.958679:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589226240:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619928.958723:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe53687363e1 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d513 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.958747:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe53687363e1 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d513 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619928.958760:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c600, rc 0) 00000100:00100000:2.0:1788619928.958856:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df385880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571657856:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2553us (2981us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619928.958863:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1168 00000100:00100000:3.0:1788619928.961591:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619928.961672:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571657984 00000100:00100000:1.0:1788619928.961683:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571657984 00000100:00100000:1.0:1788619928.961745:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1169 00000100:00100000:1.0:1788619928.961756:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df387b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571657984:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619928.961774:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619928.961791:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003b84bbc9/0x1cd7fe53687363e8 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d51a expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619928.961845:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589226368:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619928.962150:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619928.964290:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589226368:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619928.964353:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003b84bbc9/0x1cd7fe53687363e8 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d51a expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619928.964377:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b84bbc9/0x1cd7fe53687363e8 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d51a expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619928.964394:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683de00, rc 0) 00000100:00100000:1.0:1788619928.964448:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df387b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571657984:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2694us (2880us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619928.964456:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1169 00000100:00100000:2.0:1788619928.970129:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619928.970320:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571658112 00000100:00100000:3.0:1788619928.970371:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571658112 00000100:00100000:3.0:1788619928.970474:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1170 00000100:00100000:3.0:1788619928.970484:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900986050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:14908:x1875503571658112:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619928.970503:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619928.970523:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000033c29ece/0x1cd7fe53687363ef lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d521 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.970558:0:6698:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000dbe89f01/0x1cd7fe536873632b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1409024->1413119) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d45d expref: 5 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.970568:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 12) 00010000:00010000:3.0:1788619928.970572:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(3 12) 00010000:00010000:3.0:1788619928.970589:0:6698:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000033c29ece/0x1cd7fe53687363ef lrc: 4/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1507328->1527807] (req 1507328->1527807) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d521 expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.970630:0:6698:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000dbe89f01/0x1cd7fe536873632b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1409024->1413119) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d45d expref: 5 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.970643:0:6698:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000dbe89f01/0x1cd7fe536873632b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1409024->1413119) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d45d expref: 5 pid: 6696 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619928.970656:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df384a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589226496:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619928.970733:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48cc00 going to sleep for 11 seconds 00000100:00100000:0.0:1788619928.973957:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619928.974092:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571658240 00000100:00100000:1.0:1788619928.974102:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571658240 00010000:00010000:1.0:1788619928.974144:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000dbe89f01/0x1cd7fe536873632b lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1409024->1413119) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d45d expref: 6 pid: 6696 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619928.974192:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 473 00000100:00100000:1.0:1788619928.974202:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcad850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571658240:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619928.974217:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619928.974240:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000dbe89f01/0x1cd7fe536873632b lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1409024->1413119) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d45d expref: 6 pid: 6696 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619928.974255:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000dbe89f01/0x1cd7fe536873632b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1409024->1413119) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d45d expref: 6 pid: 6696 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619928.974279:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(3 12) 00010000:00010000:1.0:1788619928.974284:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(3 12) 00010000:00010000:1.0:1788619928.974294:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000033c29ece/0x1cd7fe53687363ef lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1507328->1527807) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d521 expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619928.974304:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000033c29ece/0x1cd7fe53687363ef lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1507328->1527807) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d521 expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619928.974377:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000033c29ece/0x1cd7fe53687363ef lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1507328->1527807) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d521 expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619928.974392:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be2a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589226624:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619928.974474:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb9c0 going to sleep for 11 seconds 00010000:00010000:3.0:1788619928.974560:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dbe89f01/0x1cd7fe536873632b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1409024->1413119) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d45d expref: 6 pid: 6696 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619928.974587:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df384a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589226496:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619928.974609:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000033c29ece/0x1cd7fe53687363ef lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1507328->1527807) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d521 expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619928.974620:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f70600, rc 0) 00000100:00100000:3.0:1788619928.974647:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900986050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:14908:x1875503571658112:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4165us (4530us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619928.974654:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1170 00000100:00100000:1.0:1788619928.977625:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be2a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589226624:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619928.977648:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619928.977704:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcad850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571658240:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3502us (3756us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619928.977715:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 473 00000100:00100000:0.0:1788619928.985868:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619928.986014:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571658368 00000100:00100000:2.0:1788619928.986024:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571658368 00002000:00100000:2.0:1788619928.986077:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1048576->2031615) req@ffff8e77c6530700 x1875503571658368/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:24/0 lens 488/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619928.986088:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6530700 with x1875503571658368 ext(1048576->2031615) 00002000:00010000:2.0:1788619928.986094:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6530700 00000100:00100000:2.0:1788619928.986128:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 681 00000100:00100000:2.0:1788619928.986138:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6530700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571658368:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619928.992053:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619928.992130:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571658496 00000100:00100000:1.0:1788619928.992140:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571658496 00002000:00100000:1.0:1788619928.992186:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6531f80 x1875503571658496/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:24/0 lens 504/0 e 0 to 0 dl 1788619939 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619928.992197:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6531f80 with x1875503571658496 00000100:00100000:1.0:1788619928.992236:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 871 00000100:00100000:1.0:1788619928.992245:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6531f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571658496:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619928.998746:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6531f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571658496:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 6490us (6689us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619928.998760:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 871 00002000:00100000:1.0:1788619928.998774:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6531f80 x1875503571658496/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:24/0 lens 504/440 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619928.998784:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6531f80 with x1875503571658496 00000100:00100000:2.0:1788619929.008388:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6530700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571658368:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 22240us (22517us total) trans 0 rc 983040/983040 00000100:00100000:2.0:1788619929.008403:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 681 00002000:00100000:2.0:1788619929.008416:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1048576->2031615) req@ffff8e77c6530700 x1875503571658368/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:24/0 lens 488/440 e 0 to 0 dl 1788619939 ref 1 fl Complete:/600/0 rc 983040/983040 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619929.008426:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6530700 with x1875503571658368 ext(1048576->2031615) 00002000:00010000:2.0:1788619929.008433:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6530700 00000100:00100000:3.0:1788619929.012251:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619929.012306:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619929.012547:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571658752 00000100:00100000:0.0:1788619929.012556:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571658752 00000100:00100000:0.0:1788619929.012611:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1171 00000100:00100000:0.0:1788619929.012621:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df386680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571658752:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619929.012638:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619929.012655:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096deb5d6/0x1cd7fe53687363f6 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d52f expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.012682:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(3 12) 00010000:00010000:0.0:1788619929.012686:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(3 12) 00010000:00010000:0.0:1788619929.012697:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096deb5d6/0x1cd7fe53687363f6 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d52f expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.012715:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000096deb5d6/0x1cd7fe53687363f6 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d52f expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.012729:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d5c00, rc 0) 00000100:00100000:0.0:1788619929.012792:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df386680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571658752:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 171us (549us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619929.012798:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1171 00000100:00100000:2.0:1788619929.012881:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571658624 00000100:00100000:2.0:1788619929.012885:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571658624 00000100:00100000:2.0:1788619929.012928:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1427 00000100:00100000:2.0:1788619929.012936:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df385880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571658624:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619929.012949:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619929.012962:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6a00/0x1cd7fe53687363fd lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d528 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.012990:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6a00/0x1cd7fe53687363fd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d528 expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788619929.013001:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 794416 00010000:00010000:2.0:1788619929.013007:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6a00/0x1cd7fe53687363fd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d528 expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.013019:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619929.013023:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619929.013026:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d6a00, rc 0) 00000100:00100000:2.0:1788619929.013049:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df385880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571658624:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 115us (742us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619929.013055:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1427 00000100:00100000:0.0:1788619929.017560:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619929.019646:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571659008 00000100:00100000:1.0:1788619929.019659:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571659008 00000100:00100000:1.0:1788619929.019748:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1428 00000100:00100000:1.0:1788619929.019758:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6530000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:14908:x1875503571659008:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:1.0:1788619929.019807:0:5846:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619929.019832:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70400/0x1cd7fe5368736308 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d43a expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.020045:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619929.020050:0:5846:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788619929.020234:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368736404 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.020249:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7e00/0x1cd7fe5368736283 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d3bc expref: 15 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.020269:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368736404 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.020303:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7e00/0x1cd7fe5368736283 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d3bc expref: 15 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.020317:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7e00/0x1cd7fe5368736283 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d3bc expref: 15 pid: 11397 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619929.020478:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be1180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589226752:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619929.020564:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619929.023172:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619929.023278:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571659136 00000100:00100000:0.0:1788619929.023286:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571659136 00010000:00010000:0.0:1788619929.023322:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7e00/0x1cd7fe5368736283 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d3bc expref: 16 pid: 11397 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619929.023387:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 474 00000100:00100000:0.0:1788619929.023397:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df387b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571659136:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619929.023422:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619929.023429:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7e00/0x1cd7fe5368736283 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d3bc expref: 16 pid: 11397 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.023442:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7e00/0x1cd7fe5368736283 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d3bc expref: 16 pid: 11397 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.023463:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619929.023468:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368736404 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.023479:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368736404 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x50290000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.023487:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368736404 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x50290000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.023506:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619929.023552:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df387b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571659136:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 157us (390us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619929.023560:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 474 00010000:00010000:1.0:1788619929.024107:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7e00/0x1cd7fe5368736283 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d3bc expref: 15 pid: 11397 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619929.024126:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be1180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589226752:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619929.024147:0:5846:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368736404 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.024158:0:5846:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368736404 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.024164:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368736404 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788619929.024290:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967800 00000004:00010000:1.0:1788619929.024360:0:5846:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 794416 -> 794416 00000004:00010000:1.0:1788619929.024364:0:5846:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619928 -> 1788619928 00000004:00010000:1.0:1788619929.024367:0:5846:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619927 -> 1788619927 00000004:00010000:1.0:1788619929.024369:0:5846:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619928 -> 1788619928 00000004:00010000:1.0:1788619929.024371:0:5846:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1158 -> 1158 00000004:00010000:1.0:1788619929.024378:0:5846:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000008066c31 reply state 00000000e3132af5 transno 4294967801 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368736404 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619929.024388:0:5846:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788619929.024479:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6530000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:14908:x1875503571659008:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 4722us (6927us total) trans 4294967801 rc 0/0 00000100:00100000:1.0:1788619929.024487:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1428 00000020:00080000:1.0:1788619929.024491:0:5846:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1788619929.029093:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619929.029187:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571659264 00000100:00100000:1.0:1788619929.029197:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571659264 00000100:00100000:1.0:1788619929.029258:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1429 00000100:00100000:1.0:1788619929.029269:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df384a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+12:14908:x1875503571659264:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619929.029293:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619929.029296:0:11397:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:1.0:1788619929.029360:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71000/0x1cd7fe53687363d3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d505 expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.029379:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619929.029385:0:11397:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788619929.029398:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe536873640b lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d536 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.029433:0:11397:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6a00/0x1cd7fe53687363fd lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d528 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.029443:0:11397:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe536873640b lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d536 expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.029473:0:11397:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6a00/0x1cd7fe53687363fd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d528 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.029485:0:11397:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6a00/0x1cd7fe53687363fd lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d528 expref: 14 pid: 5845 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619929.029498:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be1180 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589226880:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619929.029568:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb0c0 going to sleep for 11 seconds 00010000:00010000:0.0:1788619929.030378:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368736404 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.030388:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368736404 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.030396:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368736404 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.030412:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c683de00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788619929.030418:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368736404 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.030427:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368736404 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.030444:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619929.030446:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788619929.030451:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368736404 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.030457:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368736404 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619929.032298:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be1180 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589226880:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619929.032358:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe536873640b lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d536 expref: 11 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.032376:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619929.032379:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619929.032384:0:11397:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe536873640b lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d536 expref: 11 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.032393:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683cc00, rc 0) 00000100:00100000:1.0:1788619929.032450:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df384a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571659264:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3183us (3367us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619929.032460:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1429 00000100:00100000:0.0:1788619929.035096:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619929.035175:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571659392 00000100:00100000:2.0:1788619929.035184:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571659392 00010000:00010000:2.0:1788619929.035221:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6a00/0x1cd7fe53687363fd lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d528 expref: 15 pid: 5845 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619929.035264:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 475 00000100:00100000:2.0:1788619929.035273:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff6680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571659392:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619929.035285:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619929.035309:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6a00/0x1cd7fe53687363fd lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d528 expref: 15 pid: 5845 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.035321:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6a00/0x1cd7fe53687363fd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d528 expref: 15 pid: 5845 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.035368:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6a00/0x1cd7fe53687363fd lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d528 expref: 15 pid: 5845 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.035384:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619929.035389:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe536873640b lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d536 expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.035409:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe536873640b lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d536 expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.035419:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe536873640b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d536 expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.035458:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe536873640b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d536 expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619929.035474:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fc000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589227008:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619929.035549:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164a80 going to sleep for 11 seconds 00000100:00100000:2.0:1788619929.039033:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fc000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589227008:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619929.039060:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619929.039126:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff6680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571659392:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3852us (4038us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619929.039136:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 475 00000100:00100000:3.0:1788619929.040998:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619929.041446:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571659520 00000100:00100000:2.0:1788619929.041457:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571659520 00000100:00100000:2.0:1788619929.041515:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1172 00000100:00100000:2.0:1788619929.041525:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df386a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:14908:x1875503571659520:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619929.041543:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619929.041564:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000614d4133/0x1cd7fe5368736412 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d53d expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.041606:0:7654:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000033c29ece/0x1cd7fe53687363ef lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1507328->1527807) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d521 expref: 7 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.041616:0:7654:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000096deb5d6/0x1cd7fe53687363f6 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d52f expref: 5 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.041624:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(3 12) 00010000:00010000:2.0:1788619929.041628:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(3 12) 00010000:00010000:2.0:1788619929.041633:0:7654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000614d4133/0x1cd7fe5368736412 lrc: 5/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d53d expref: 7 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.041666:0:7654:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000096deb5d6/0x1cd7fe53687363f6 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d52f expref: 5 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.041678:0:7654:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000096deb5d6/0x1cd7fe53687363f6 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d52f expref: 5 pid: 6696 timeout: 521 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619929.041691:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589227136:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619929.041767:0:7654:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000033c29ece/0x1cd7fe53687363ef lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1507328->1527807) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d521 expref: 7 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.041776:0:7654:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000033c29ece/0x1cd7fe53687363ef lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1507328->1527807) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d521 expref: 7 pid: 6698 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619929.041785:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589227264:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619929.041803:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164a80 going to sleep for 11 seconds 00000100:00100000:2.0:1788619929.044441:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589227136:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619929.044454:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589227264:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619929.044485:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000614d4133/0x1cd7fe5368736412 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d53d expref: 7 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.044507:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d6c00, rc 0) 00000100:00100000:2.0:1788619929.044564:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df386a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+7:14908:x1875503571659520:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3040us (3574us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619929.044582:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1172 00000100:00100000:0.0:1788619929.049072:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619929.049120:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619929.049191:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571659648 00000100:00100000:3.0:1788619929.049201:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571659648 00010000:00010000:3.0:1788619929.049239:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000033c29ece/0x1cd7fe53687363ef lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1507328->1527807) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d521 expref: 7 pid: 6698 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619929.049273:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571659776 00000100:00100000:3.0:1788619929.049277:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571659776 00010000:00010000:3.0:1788619929.049292:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000096deb5d6/0x1cd7fe53687363f6 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d52f expref: 6 pid: 6696 timeout: 521 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619929.049321:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 476 00000100:00100000:3.0:1788619929.049355:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff7480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571659648:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619929.049370:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619929.049391:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000033c29ece/0x1cd7fe53687363ef lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1507328->1527807) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d521 expref: 7 pid: 6698 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.049414:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000033c29ece/0x1cd7fe53687363ef lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1507328->1527807) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d521 expref: 7 pid: 6698 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.049440:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000033c29ece/0x1cd7fe53687363ef lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1507328->1527807) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d521 expref: 7 pid: 6698 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.049458:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619929.049506:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff7480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571659648:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 154us (444us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619929.049513:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 476 00000100:00100000:3.0:1788619929.049530:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 477 00000100:00100000:3.0:1788619929.049538:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571659776:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619929.049545:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619929.049555:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000096deb5d6/0x1cd7fe53687363f6 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d52f expref: 6 pid: 6696 timeout: 521 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.049565:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000096deb5d6/0x1cd7fe53687363f6 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d52f expref: 6 pid: 6696 timeout: 521 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.049589:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096deb5d6/0x1cd7fe53687363f6 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d52f expref: 6 pid: 6696 timeout: 521 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.049602:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(3 12) 00010000:00010000:3.0:1788619929.049606:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(3 12) 00010000:00010000:3.0:1788619929.049615:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000614d4133/0x1cd7fe5368736412 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d53d expref: 5 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.049624:0:13330:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000614d4133/0x1cd7fe5368736412 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d53d expref: 5 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.049658:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000614d4133/0x1cd7fe5368736412 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d53d expref: 5 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619929.049672:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df384000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589227392:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619929.049723:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48cc00 going to sleep for 11 seconds 00000100:00100000:0.0:1788619929.052957:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619929.052994:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619929.053071:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571659904 00000100:00100000:1.0:1788619929.053079:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571659904 00002000:00010000:1.0:1788619929.053116:0:5858:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:1.0:1788619929.053120:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5ff5f80 with x1875503571659904 00000100:00100000:1.0:1788619929.053153:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 872 00000100:00100000:1.0:1788619929.053161:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571659904:12345-192.168.201.5@tcp:10:fsx.0 00000004:00010000:1.0:1788619929.053357:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 794416 -> 978388 00000004:00010000:1.0:1788619929.053361:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619928 -> 1788619928 00000004:00010000:1.0:1788619929.053363:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619927 -> 1788619927 00000004:00010000:1.0:1788619929.053366:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619928 -> 1788619928 00000004:00010000:1.0:1788619929.053368:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1158 -> 1158 00000100:00100000:1.0:1788619929.053438:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571659904:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 277us (443us total) trans 4294967802 rc 0/0 00000100:00100000:1.0:1788619929.053444:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 872 00002000:00010000:1.0:1788619929.053448:0:5858:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:1.0:1788619929.053451:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5ff5f80 with x1875503571659904 00000100:00100000:2.0:1788619929.054372:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571660032 00000100:00100000:2.0:1788619929.054380:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571660032 00002000:00010000:2.0:1788619929.054417:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:2.0:1788619929.054422:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5ff6a00 with x1875503571660032 ext(0->18446744073709551615) 00002000:00010000:2.0:1788619929.054431:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5ff6a00. 00000100:00100000:2.0:1788619929.054459:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 682 00000100:00100000:2.0:1788619929.054467:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571660032:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:2.0:1788619929.054774:0:6703:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 2030185 -> 0 00002000:00010000:2.0:1788619929.054779:0:6703:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619928 -> 1788619928 00002000:00010000:2.0:1788619929.054781:0:6703:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619928 -> 1788619927 00002000:00010000:2.0:1788619929.054783:0:6703:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619928 -> 1788619928 00002000:00010000:2.0:1788619929.054785:0:6703:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 391 -> 391 00000100:00100000:2.0:1788619929.054832:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571660032:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 366us (1884us total) trans 4294967497 rc 0/0 00000100:00100000:2.0:1788619929.055015:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 682 00002000:00010000:2.0:1788619929.055019:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:2.0:1788619929.055022:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5ff6a00 with x1875503571660032 ext(0->18446744073709551615) 00002000:00010000:2.0:1788619929.055027:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5ff6a00. 00000100:00100000:3.0:1788619929.057078:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df384000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589227392:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619929.057103:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619929.057298:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571659776:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 7759us (8176us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619929.057307:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 477 00000100:00100000:0.0:1788619929.057564:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619929.058758:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571660160 00000100:00100000:2.0:1788619929.058769:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571660160 00000100:00100000:2.0:1788619929.058920:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1430 00000100:00100000:2.0:1788619929.058931:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571660160:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619929.058951:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619929.058971:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6a00/0x1cd7fe5368736419 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d544 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.059062:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7c00/0x1cd7fe5368736420 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.059079:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7c00/0x1cd7fe5368736420 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.059091:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7c00/0x1cd7fe5368736420 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788619929.059118:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 978388 00000004:00200000:2.0:1788619929.059123:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788619929.059143:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7c00/0x1cd7fe5368736420 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.059171:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6a00/0x1cd7fe5368736419 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d544 expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.059188:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7c00/0x1cd7fe5368736420 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d544 expref: 15 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.059197:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619929.059200:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788619929.059204:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d7c00, rc 0) 00000100:00100000:2.0:1788619929.059264:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:14908:x1875503571660160:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 335us (1700us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619929.059270:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1430 00000100:00100000:1.0:1788619929.062040:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619929.062350:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619929.062368:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571660416 00000100:00100000:1.0:1788619929.062376:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571660416 00000100:00100000:1.0:1788619929.062428:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1173 00000100:00100000:1.0:1788619929.062437:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571660416:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619929.062453:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619929.062469:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a803caae/0x1cd7fe5368736427 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d552 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619929.062522:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589227520:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619929.062598:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619929.064492:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571660288 00000100:00100000:0.0:1788619929.064505:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571660288 00000100:00100000:0.0:1788619929.064564:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1431 00000100:00100000:0.0:1788619929.064604:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571660288:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619929.064622:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619929.064641:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5000/0x1cd7fe536873642e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d54b expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.064675:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe536873640b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d536 expref: 11 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619929.064708:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5ff7800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589227648:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619929.064884:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590a80 going to sleep for 11 seconds 00000004:00010000:3.0:1788619929.067089:0:9417:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619927 -> 1788619928 00000100:00100000:3.0:1788619929.067121:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5ff7800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589227648:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619929.067143:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 978388 00010000:00010000:3.0:1788619929.067160:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5000/0x1cd7fe536873642e lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d54b expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.067178:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619929.067181:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619929.067186:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5000/0x1cd7fe536873642e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d54b expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.067202:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d5000, rc 0) 00000100:00100000:3.0:1788619929.067291:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571660288:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2718us (5259us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619929.067300:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1431 00002000:00010000:0.0:1788619929.067600:0:6696:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619927 -> 1788619928 00000100:00100000:0.0:1788619929.067616:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589227520:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619929.067637:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a803caae/0x1cd7fe5368736427 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d552 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.067655:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a803caae/0x1cd7fe5368736427 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d552 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.067665:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d000, rc 0) 00000100:00100000:0.0:1788619929.067689:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571660416:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5254us (5340us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619929.067696:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1173 00000100:00100000:1.0:1788619929.070693:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619929.070761:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619929.070857:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571660544 00000100:00100000:3.0:1788619929.070868:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571660544 00000100:00100000:3.0:1788619929.070930:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1432 00000100:00100000:3.0:1788619929.070941:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571660544:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619929.070961:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619929.070984:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70600/0x1cd7fe5368736435 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d559 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.071026:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe536873640b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d536 expref: 11 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619929.071068:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df384000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589227776:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619929.071151:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48cc00 going to sleep for 11 seconds 00000100:00100000:3.0:1788619929.077642:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df384000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589227776:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619929.077668:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 978388 00010000:00010000:3.0:1788619929.077689:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70600/0x1cd7fe5368736435 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d559 expref: 16 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.077712:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619929.077715:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619929.077720:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70600/0x1cd7fe5368736435 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d559 expref: 16 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.077734:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f70600, rc 0) 00000100:00100000:3.0:1788619929.077796:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571660544:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6857us (7113us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619929.077805:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1432 00000100:00100000:2.0:1788619929.079225:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571660672 00000100:00100000:2.0:1788619929.079237:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571660672 00000100:00100000:2.0:1788619929.079403:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1174 00000100:00100000:2.0:1788619929.079415:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571660672:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619929.079430:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619929.079446:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041680db8/0x1cd7fe536873643c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d560 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619929.079500:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589227904:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619929.079723:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164a80 going to sleep for 11 seconds 00000100:00100000:2.0:1788619929.082022:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589227904:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619929.082060:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000041680db8/0x1cd7fe536873643c lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d560 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.082085:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041680db8/0x1cd7fe536873643c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d560 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.082098:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d7200, rc 0) 00000100:00100000:2.0:1788619929.082284:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571660672:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2870us (11522us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619929.082292:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1174 00000100:00100000:0.0:1788619929.085104:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619929.085168:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619929.085228:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571660928 00000100:00100000:1.0:1788619929.085237:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571660928 00000100:00100000:1.0:1788619929.085290:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1175 00000100:00100000:1.0:1788619929.085300:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571660928:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619929.085318:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619929.085364:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001391204f/0x1cd7fe5368736443 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d56e expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619929.085435:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589228032:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619929.085513:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebc00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619929.087415:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571660800 00000100:00100000:2.0:1788619929.087426:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571660800 00000100:00100000:2.0:1788619929.087501:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1433 00000100:00100000:2.0:1788619929.087510:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571660800:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619929.087526:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619929.087543:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6400/0x1cd7fe536873644a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d567 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.087587:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe536873640b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d536 expref: 11 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619929.087619:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fdc00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589228160:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619929.087679:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164a80 going to sleep for 11 seconds 00000100:00100000:2.0:1788619929.089545:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589228032:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619929.089598:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001391204f/0x1cd7fe5368736443 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d56e expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.089621:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001391204f/0x1cd7fe5368736443 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d56e expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.089633:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683da00, rc 0) 00000100:00100000:2.0:1788619929.089813:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571660928:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4513us (4642us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619929.089821:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1175 00000100:00100000:2.0:1788619929.091243:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fdc00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589228160:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619929.091264:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 978388 00010000:00010000:2.0:1788619929.091280:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6400/0x1cd7fe536873644a lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d567 expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.091297:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619929.091300:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619929.091306:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6400/0x1cd7fe536873644a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d567 expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.091318:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d6400, rc 0) 00000100:00100000:2.0:1788619929.091390:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571660800:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3881us (6295us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619929.091634:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1433 00000100:00100000:1.0:1788619929.095242:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619929.095377:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619929.095565:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571661056 00000100:00100000:3.0:1788619929.095586:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571661056 00000100:00100000:3.0:1788619929.095676:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1434 00000100:00100000:3.0:1788619929.095687:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571661056:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619929.095708:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619929.095729:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71a00/0x1cd7fe5368736451 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d575 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.095765:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe536873640b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d536 expref: 11 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619929.095807:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df384000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589228288:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619929.095900:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48cc00 going to sleep for 11 seconds 00000100:00100000:3.0:1788619929.098275:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df384000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589228288:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619929.098301:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 978388 00010000:00010000:3.0:1788619929.098321:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71a00/0x1cd7fe5368736451 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d575 expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.098374:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619929.098376:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619929.098382:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71a00/0x1cd7fe5368736451 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d575 expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.098405:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f71a00, rc 0) 00000100:00100000:3.0:1788619929.098453:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571661056:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2768us (3076us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619929.098462:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1434 00000100:00100000:2.0:1788619929.098799:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571661184 00000100:00100000:2.0:1788619929.098807:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571661184 00000100:00100000:2.0:1788619929.098849:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1176 00000100:00100000:2.0:1788619929.098858:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571661184:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619929.098871:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619929.098886:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041680db8/0x1cd7fe5368736458 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d57c expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619929.098955:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589228416:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619929.099013:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164a80 going to sleep for 11 seconds 00000100:00100000:2.0:1788619929.103362:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589228416:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619929.103415:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000041680db8/0x1cd7fe5368736458 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d57c expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.103439:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041680db8/0x1cd7fe5368736458 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d57c expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.103453:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d7200, rc 0) 00000100:00100000:2.0:1788619929.103514:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571661184:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4656us (8279us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619929.103523:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1176 00000100:00100000:1.0:1788619929.109805:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619929.109845:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619929.110021:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571661312 00000100:00100000:3.0:1788619929.110032:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571661312 00000100:00100000:3.0:1788619929.110092:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1435 00000100:00100000:3.0:1788619929.110103:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa40000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571661312:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619929.110124:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619929.110145:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71a00/0x1cd7fe536873645f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d583 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.110186:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe536873640b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d536 expref: 11 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619929.110226:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df384000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589228544:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619929.110315:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48cc00 going to sleep for 11 seconds 00000100:00100000:0.0:1788619929.110615:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571661440 00000100:00100000:0.0:1788619929.110621:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571661440 00000100:00100000:0.0:1788619929.110661:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1177 00000100:00100000:0.0:1788619929.110669:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa43100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571661440:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619929.110678:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619929.110689:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000df72df5a/0x1cd7fe5368736466 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d58a expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619929.110734:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5ff4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589228672:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619929.110772:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590780 going to sleep for 11 seconds 00000100:00100000:3.0:1788619929.113555:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df384000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589228544:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619929.113594:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 978388 00010000:00010000:3.0:1788619929.113614:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71a00/0x1cd7fe536873645f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d583 expref: 16 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.113634:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619929.113637:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619929.113643:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71a00/0x1cd7fe536873645f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d583 expref: 16 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.113657:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f71a00, rc 0) 00000100:00100000:3.0:1788619929.113713:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa40000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571661312:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3611us (3917us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619929.113723:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1435 00000100:00100000:0.0:1788619929.115962:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5ff4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589228672:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619929.116002:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000df72df5a/0x1cd7fe5368736466 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d58a expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.116025:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000df72df5a/0x1cd7fe5368736466 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d58a expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.116038:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d5200, rc 0) 00000100:00100000:0.0:1788619929.116095:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa43100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571661440:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5426us (6248us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619929.116103:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1177 00000100:00100000:2.0:1788619929.119405:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619929.119755:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571661568 00000100:00100000:0.0:1788619929.119767:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571661568 00000100:00100000:0.0:1788619929.119830:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1436 00000100:00100000:0.0:1788619929.119851:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010fdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571661568:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619929.119875:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619929.119898:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5c00/0x1cd7fe536873646d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d591 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.119939:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe536873640b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d536 expref: 11 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619929.119982:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5ff4380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589228800:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619929.120060:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590780 going to sleep for 11 seconds 00000100:00100000:1.0:1788619929.122502:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619929.122738:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5ff4380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589228800:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619929.122759:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 978388 00010000:00010000:0.0:1788619929.122777:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5c00/0x1cd7fe536873646d lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d591 expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.122798:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619929.122802:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619929.122808:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5c00/0x1cd7fe536873646d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d591 expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.122824:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d5c00, rc 0) 00000100:00100000:0.0:1788619929.122886:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010fdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571661568:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3047us (3499us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619929.122895:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1436 00000100:00100000:2.0:1788619929.123173:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571661696 00000100:00100000:2.0:1788619929.123179:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571661696 00000100:00100000:2.0:1788619929.123233:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1178 00000100:00100000:2.0:1788619929.123243:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa41c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571661696:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619929.123257:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619929.123273:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041680db8/0x1cd7fe5368736474 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d598 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619929.123351:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589228928:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619929.123631:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164a80 going to sleep for 11 seconds 00000100:00100000:2.0:1788619929.126682:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589228928:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619929.126718:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000041680db8/0x1cd7fe5368736474 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d598 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.126745:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041680db8/0x1cd7fe5368736474 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d598 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.126758:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d7200, rc 0) 00000100:00100000:2.0:1788619929.126818:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa41c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571661696:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3577us (4325us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619929.126827:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1178 00000100:00100000:0.0:1788619929.130169:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619929.130248:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571661824 00000100:00100000:3.0:1788619929.130257:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571661824 00000100:00100000:3.0:1788619929.130319:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1437 00000100:00100000:3.0:1788619929.130549:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+12:14908:x1875503571661824:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619929.130584:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619929.130605:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71a00/0x1cd7fe536873647b lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d59f expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.130695:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70600/0x1cd7fe5368736482 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.130711:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70600/0x1cd7fe5368736482 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.130723:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70600/0x1cd7fe5368736482 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788619929.130752:0:9417:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 978388 00000004:00200000:3.0:1788619929.130756:0:9417:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788619929.130773:0:9417:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70600/0x1cd7fe5368736482 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.130799:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71a00/0x1cd7fe536873647b lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d59f expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.130815:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70600/0x1cd7fe5368736482 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d59f expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.130823:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619929.130825:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788619929.130830:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f70600, rc 0) 00000100:00100000:3.0:1788619929.130958:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+13:14908:x1875503571661824:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 413us (797us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619929.130965:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1437 00000100:00100000:1.0:1788619929.135844:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619929.135994:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571661952 00000100:00100000:0.0:1788619929.136005:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571661952 00000100:00100000:0.0:1788619929.136073:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 873 00000100:00100000:0.0:1788619929.136083:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa43800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+13:2355:x1875503571661952:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:0.0:1788619929.136387:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 978388 -> 978388 00000004:00010000:0.0:1788619929.136391:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619928 -> 1788619928 00000004:00010000:0.0:1788619929.136393:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619928 -> 1788619927 00000004:00010000:0.0:1788619929.136406:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619928 -> 1788619928 00000004:00010000:0.0:1788619929.136408:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1158 -> 1158 00000100:00100000:0.0:1788619929.136471:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa43800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+14:2355:x1875503571661952:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 389us (636us total) trans 4294967803 rc 0/0 00000100:00100000:0.0:1788619929.136478:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 873 00000100:00100000:1.0:1788619929.140015:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619929.140056:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619929.140127:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571662208 00000100:00100000:2.0:1788619929.140138:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571662208 00000100:00100000:2.0:1788619929.140187:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1179 00000100:00100000:2.0:1788619929.140197:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571662208:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619929.140214:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619929.140230:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b195ac97/0x1cd7fe5368736489 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d5ad expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619929.140287:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589229056:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619929.140391:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164a80 going to sleep for 11 seconds 00000100:00100000:3.0:1788619929.140460:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571662080 00000100:00100000:3.0:1788619929.140464:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571662080 00000100:00100000:3.0:1788619929.140506:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1438 00000100:00100000:3.0:1788619929.140514:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa41500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2352:x1875503571662080:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619929.140525:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619929.140538:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70200/0x1cd7fe5368736490 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d5a6 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.140566:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe536873640b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d536 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619929.140608:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df384000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589229184:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619929.140634:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48cc00 going to sleep for 11 seconds 00000004:00010000:3.0:1788619929.143582:0:11397:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619927 -> 1788619928 00000004:00010000:3.0:1788619929.143592:0:11397:0:(mdt_lvb.c:195:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime: 1788619928 -> 1788619929 00000100:00100000:3.0:1788619929.143618:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df384000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589229184:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619929.143641:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 978388 00010000:00010000:3.0:1788619929.143659:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70200/0x1cd7fe5368736490 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d5a6 expref: 16 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.143679:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619929.143682:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619929.143688:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70200/0x1cd7fe5368736490 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d5a6 expref: 16 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.143703:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f70200, rc 0) 00000100:00100000:3.0:1788619929.143767:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa41500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2352:x1875503571662080:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3253us (3760us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619929.143775:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1438 00002000:00010000:2.0:1788619929.145305:0:6697:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime: 1788619928 -> 1788619929 00000100:00100000:2.0:1788619929.145353:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589229056:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619929.145387:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b195ac97/0x1cd7fe5368736489 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d5ad expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.145421:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b195ac97/0x1cd7fe5368736489 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d5ad expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.145434:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d6800, rc 0) 00000100:00100000:2.0:1788619929.145490:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571662208:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5294us (5431us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619929.145497:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1179 00000100:00100000:1.0:1788619929.149681:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619929.149727:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619929.149795:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571662464 00000100:00100000:3.0:1788619929.149805:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571662464 00000100:00100000:3.0:1788619929.149880:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1180 00000100:00100000:3.0:1788619929.149890:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa40700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571662464:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619929.149922:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619929.149941:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c3618b1f/0x1cd7fe5368736497 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d5bb expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619929.149996:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df384000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589229312:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619929.150080:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48cc00 going to sleep for 11 seconds 00000100:00100000:0.0:1788619929.150744:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571662336 00000100:00100000:0.0:1788619929.150750:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571662336 00000100:00100000:0.0:1788619929.150796:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1439 00000100:00100000:0.0:1788619929.150805:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa42d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2353:x1875503571662336:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619929.150817:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619929.150833:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5200/0x1cd7fe536873649e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d5b4 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.150876:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe536873640b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d536 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619929.151135:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5ff4000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589229440:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619929.151373:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590780 going to sleep for 11 seconds 00000100:00100000:3.0:1788619929.155438:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df384000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589229312:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619929.155480:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c3618b1f/0x1cd7fe5368736497 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d5bb expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.155505:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c3618b1f/0x1cd7fe5368736497 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d5bb expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.155519:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f71a00, rc 0) 00000100:00100000:3.0:1788619929.160082:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa40700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571662464:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 10181us (10340us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619929.160096:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1180 00000100:00100000:0.0:1788619929.172233:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5ff4000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589229440:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619929.172262:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 978388 00010000:00010000:0.0:1788619929.172283:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5200/0x1cd7fe536873649e lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d5b4 expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.172304:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619929.172307:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619929.172312:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5200/0x1cd7fe536873649e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d5b4 expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.172354:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d5200, rc 0) 00000100:00100000:1.0:1788619929.172649:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa42d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2353:x1875503571662336:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 21844us (22976us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619929.172659:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1439 00000100:00100000:3.0:1788619929.176197:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619929.176237:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619929.176364:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571662720 00000100:00100000:1.0:1788619929.176376:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571662720 00000100:00100000:1.0:1788619929.176443:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1181 00000100:00100000:1.0:1788619929.176453:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df385c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571662720:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619929.176472:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619929.176490:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe53687364a5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d5c9 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619929.176551:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78dfa40e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589229568:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619929.176638:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb840 going to sleep for 11 seconds 00000100:00100000:3.0:1788619929.179465:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78dfa40e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589229568:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619929.179504:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe53687364a5 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d5c9 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.179530:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe53687364a5 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d5c9 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.179543:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c600, rc 0) 00000100:00100000:3.0:1788619929.179608:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df385c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571662720:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3155us (3368us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619929.179617:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1181 00000100:00100000:0.0:1788619929.179775:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571662592 00000100:00100000:0.0:1788619929.179783:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571662592 00000100:00100000:0.0:1788619929.179843:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1440 00000100:00100000:0.0:1788619929.179853:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df384000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571662592:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619929.179869:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619929.179889:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4400/0x1cd7fe53687364ac lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d5c2 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.179925:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe536873640b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d536 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619929.179975:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5ff4000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589229696:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619929.180031:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590780 going to sleep for 11 seconds 00000100:00100000:0.0:1788619929.182893:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5ff4000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589229696:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619929.182928:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 978388 00010000:00010000:0.0:1788619929.182954:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4400/0x1cd7fe53687364ac lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d5c2 expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.182975:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619929.182978:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619929.182984:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4400/0x1cd7fe53687364ac lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d5c2 expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.183000:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d4400, rc 0) 00000100:00100000:0.0:1788619929.183060:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df384000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571662592:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3208us (6872us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619929.183068:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1440 00000100:00100000:3.0:1788619929.186701:0:3317:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df387800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3317:1875503589230080:0@lo:400:kworker.0 00000100:00100000:3.0:1788619929.186767:0:3317:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619929.186805:0:3317:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1788619929.186810:0:3317:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df385180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3317:1875503589229952:0@lo:400:kworker.0 00000100:00100000:3.0:1788619929.186831:0:3317:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619929.186859:0:3317:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c75f5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3317:1875503589230464:0@lo:400:kworker.0 00000100:00100000:3.0:1788619929.186883:0:3317:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619929.186994:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589230080 00000100:00100000:3.0:1788619929.187004:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589230080 00000100:00080000:3.0:1788619929.187032:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788619929.187069:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589229952 00000100:00100000:3.0:1788619929.187072:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589229952 00000100:00100000:3.0:1788619929.187094:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 192 00000100:00100000:3.0:1788619929.187102:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3317:x1875503589230080:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788619929.187152:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3317:x1875503589230080:12345-0@lo:400:kworker.0 Request processed in 51us (384us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619929.187159:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 192 00000100:00100000:3.0:1788619929.187198:0:3317:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df387800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3317:1875503589230080:0@lo:400:kworker.0 00000100:00100000:3.0:1788619929.187223:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 193 00000100:00100000:3.0:1788619929.187229:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3317:x1875503589229952:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788619929.187262:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3317:x1875503589229952:12345-0@lo:400:kworker.0 Request processed in 33us (430us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619929.187268:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 193 00000100:00100000:3.0:1788619929.187292:0:3317:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df385180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3317:1875503589229952:0@lo:400:kworker.0 00000100:00100000:2.0:1788619929.187381:0:3319:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df387480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:3319:1875503589229824:0@lo:400:kworker.0 00000100:00100000:2.0:1788619929.187413:0:3319:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788619929.187441:0:3319:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5dcad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3319:1875503589230336:0@lo:400:kworker.0 00000100:00100000:2.0:1788619929.187462:0:3319:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788619929.187478:0:3319:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1788619929.187483:0:3319:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fefd5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3319:1875503589230208:0@lo:400:kworker.0 00000100:00100000:2.0:1788619929.187503:0:3319:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788619929.187530:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589230464 00000100:00100000:2.0:1788619929.187534:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589230464 00000100:00100000:2.0:1788619929.187567:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589230208 00000100:00100000:2.0:1788619929.187582:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589230208 00000100:00100000:2.0:1788619929.187606:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 130 00000100:00100000:2.0:1788619929.187612:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3317:x1875503589230464:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788619929.187645:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3317:x1875503589230464:12345-0@lo:400:kworker.0 Request processed in 33us (761us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619929.187651:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 130 00000100:00100000:2.0:1788619929.187664:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 131 00000100:00100000:2.0:1788619929.187670:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3319:x1875503589230208:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788619929.187701:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3319:x1875503589230208:12345-0@lo:400:kworker.0 Request processed in 31us (197us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619929.187706:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 131 00000100:00080000:2.0:1788619929.187721:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1788619929.187724:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1788619929.187741:0:3319:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fefd5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3319:1875503589230208:0@lo:400:kworker.0 00000100:00100000:1.0:1788619929.188029:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589230336 00000100:00100000:1.0:1788619929.188034:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589230336 00000100:00100000:1.0:1788619929.188075:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 194 00000100:00100000:1.0:1788619929.188081:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010fc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3319:x1875503589230336:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788619929.188116:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010fc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3319:x1875503589230336:12345-0@lo:400:kworker.0 Request processed in 36us (653us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619929.188125:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 194 00000100:00100000:1.0:1788619929.188176:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619929.188194:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571662848 00000100:00100000:1.0:1788619929.188198:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571662848 00000100:00100000:1.0:1788619929.188218:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1441 00000100:00100000:1.0:1788619929.188225:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa42300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+15:14908:x1875503571662848:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619929.188237:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619929.188257:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687364b3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d5d0 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.188284:0:11397:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe536873640b lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d536 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.188295:0:11397:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687364b3 lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d5d0 expref: 16 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.188323:0:11397:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe536873640b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d536 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.188366:0:11397:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe536873640b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d536 expref: 13 pid: 11397 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619929.188377:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78dfa40380 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589230592:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619929.188431:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebc00 going to sleep for 11 seconds 00000100:00100000:3.0:1788619929.188887:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589229824 00000100:00100000:3.0:1788619929.188893:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589229824 00000100:00100000:3.0:1788619929.188940:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 294 00000100:00080000:2.0:1788619929.189442:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1788619929.189478:0:3319:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5dcad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3319:1875503589230336:0@lo:400:kworker.0 00000100:00100000:3.0:1788619929.189602:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5ce0850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+19:3319:x1875503589229824:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788619929.189666:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5ce0850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+19:3319:x1875503589229824:12345-0@lo:400:kworker.0 Request processed in 719us (2252us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619929.189673:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 294 00000100:00100000:3.0:1788619929.189718:0:3317:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c75f5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3317:1875503589230464:0@lo:400:kworker.0 00000100:00100000:2.0:1788619929.189847:0:3319:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df387480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:3319:1875503589229824:0@lo:400:kworker.0 00000100:00100000:1.0:1788619929.192410:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619929.192637:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571662976 00000100:00100000:2.0:1788619929.192646:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571662976 00010000:00010000:2.0:1788619929.192682:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe536873640b lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d536 expref: 14 pid: 11397 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619929.192715:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 478 00000100:00100000:2.0:1788619929.192724:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa43b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+14:2355:x1875503571662976:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619929.192741:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619929.192764:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe536873640b lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d536 expref: 14 pid: 11397 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.192776:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe536873640b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d536 expref: 14 pid: 11397 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.192804:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619929.192809:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687364b3 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d5d0 expref: 16 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.192823:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687364b3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d5d0 expref: 16 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.192832:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687364b3 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d5d0 expref: 16 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.192880:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687364b3 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d5d0 expref: 16 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619929.192893:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fdf80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589230720:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619929.192979:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164a80 going to sleep for 11 seconds 00010000:00010000:3.0:1788619929.194052:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe536873640b lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d536 expref: 14 pid: 11397 timeout: 521 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619929.194073:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78dfa40380 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589230592:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619929.194100:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687364b3 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d5d0 expref: 16 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.194115:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619929.194118:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619929.194122:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683da00, rc 0) 00000100:00100000:3.0:1788619929.194564:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa42300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+16:14908:x1875503571662848:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6337us (6387us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619929.194583:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1441 00000100:00100000:2.0:1788619929.198255:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fdf80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589230720:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619929.198291:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619929.198719:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa43b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+13:2355:x1875503571662976:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 5993us (6325us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619929.198731:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 478 00000100:00100000:1.0:1788619929.200868:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619929.201221:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571663104 00000100:00100000:0.0:1788619929.201231:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571663104 00002000:00100000:0.0:1788619929.201281:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78dfa42680 x1875503571663104/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:25/0 lens 488/0 e 0 to 0 dl 1788619940 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619929.201293:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78dfa42680 with x1875503571663104 00000100:00100000:0.0:1788619929.201349:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 874 00000100:00100000:0.0:1788619929.201358:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa42680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571663104:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619929.201638:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa42680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571663104:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 280us (779us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619929.201645:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 874 00002000:00100000:0.0:1788619929.201653:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78dfa42680 x1875503571663104/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:25/0 lens 488/4536 e 0 to 0 dl 1788619940 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619929.201661:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78dfa42680 with x1875503571663104 00000100:00100000:0.0:1788619929.205695:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619929.205775:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571663232 00000100:00100000:1.0:1788619929.205785:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571663232 00002000:00100000:1.0:1788619929.205828:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5ff4000 x1875503571663232/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:25/0 lens 488/0 e 0 to 0 dl 1788619940 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619929.205896:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5ff4000 with x1875503571663232 00000100:00100000:1.0:1788619929.205955:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 875 00000100:00100000:1.0:1788619929.205965:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571663232:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619929.206250:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571663232:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 285us (566us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619929.206257:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 875 00002000:00100000:1.0:1788619929.206264:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5ff4000 x1875503571663232/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:25/0 lens 488/4536 e 0 to 0 dl 1788619940 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619929.206272:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5ff4000 with x1875503571663232 00000100:00100000:3.0:1788619929.208950:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619929.209065:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571663360 00000100:00100000:0.0:1788619929.209076:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571663360 00002000:00100000:0.0:1788619929.209127:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c75f5880 x1875503571663360/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:25/0 lens 488/0 e 0 to 0 dl 1788619940 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619929.209139:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c75f5880 with x1875503571663360 00000100:00100000:0.0:1788619929.209182:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 876 00000100:00100000:0.0:1788619929.209192:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571663360:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619929.209476:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571663360:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 285us (536us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619929.209484:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 876 00002000:00100000:0.0:1788619929.209492:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c75f5880 x1875503571663360/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:25/0 lens 488/4536 e 0 to 0 dl 1788619940 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619929.209500:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c75f5880 with x1875503571663360 00000100:00100000:2.0:1788619929.212396:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619929.212479:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571663488 00000100:00100000:0.0:1788619929.212489:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571663488 00002000:00100000:0.0:1788619929.212539:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79010fdf80 x1875503571663488/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:25/0 lens 488/0 e 0 to 0 dl 1788619940 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619929.212550:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79010fdf80 with x1875503571663488 00000100:00100000:0.0:1788619929.212594:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 877 00000100:00100000:0.0:1788619929.212604:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010fdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2353:x1875503571663488:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619929.212871:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010fdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2353:x1875503571663488:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 266us (484us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619929.212878:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 877 00002000:00100000:0.0:1788619929.212884:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79010fdf80 x1875503571663488/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:25/0 lens 488/4536 e 0 to 0 dl 1788619940 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619929.212892:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79010fdf80 with x1875503571663488 00000100:00100000:3.0:1788619929.215766:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619929.216097:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571663616 00000100:00100000:0.0:1788619929.216108:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571663616 00002000:00100000:0.0:1788619929.216155:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c75f5500 x1875503571663616/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:25/0 lens 488/0 e 0 to 0 dl 1788619940 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619929.216166:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c75f5500 with x1875503571663616 00000100:00100000:0.0:1788619929.216255:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 878 00000100:00100000:0.0:1788619929.216265:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2352:x1875503571663616:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619929.216549:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2352:x1875503571663616:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 285us (793us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619929.216556:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 878 00002000:00100000:0.0:1788619929.216562:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c75f5500 x1875503571663616/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:25/0 lens 488/4536 e 0 to 0 dl 1788619940 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619929.216581:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c75f5500 with x1875503571663616 00000100:00100000:3.0:1788619929.219247:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619929.219436:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571663744 00000100:00100000:0.0:1788619929.219445:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571663744 00002000:00100000:0.0:1788619929.219502:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df387480 x1875503571663744/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:25/0 lens 488/0 e 0 to 0 dl 1788619940 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619929.219514:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df387480 with x1875503571663744 00000100:00100000:0.0:1788619929.219598:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 879 00000100:00100000:0.0:1788619929.219607:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df387480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2353:x1875503571663744:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619929.219862:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df387480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2353:x1875503571663744:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 254us (623us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619929.219869:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 879 00002000:00100000:0.0:1788619929.219876:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df387480 x1875503571663744/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:25/0 lens 488/4536 e 0 to 0 dl 1788619940 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619929.219883:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df387480 with x1875503571663744 00000100:00100000:3.0:1788619929.223274:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619929.223530:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571663872 00000100:00100000:1.0:1788619929.223540:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571663872 00002000:00100000:1.0:1788619929.223605:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df385180 x1875503571663872/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:25/0 lens 488/0 e 0 to 0 dl 1788619940 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619929.223617:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df385180 with x1875503571663872 00000100:00100000:1.0:1788619929.223656:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 880 00000100:00100000:1.0:1788619929.223666:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df385180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2352:x1875503571663872:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619929.223998:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df385180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2352:x1875503571663872:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 332us (732us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619929.224005:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 880 00002000:00100000:1.0:1788619929.224013:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df385180 x1875503571663872/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:25/0 lens 488/4536 e 0 to 0 dl 1788619940 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619929.224027:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df385180 with x1875503571663872 00000100:00100000:3.0:1788619929.227231:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619929.227424:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571664000 00000100:00100000:2.0:1788619929.227434:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571664000 00002000:00100000:2.0:1788619929.227486:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df387800 x1875503571664000/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:25/0 lens 488/0 e 0 to 0 dl 1788619940 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619929.227508:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df387800 with x1875503571664000 00000100:00100000:2.0:1788619929.227545:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 881 00000100:00100000:2.0:1788619929.227554:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df387800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2353:x1875503571664000:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619929.227831:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df387800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2353:x1875503571664000:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 277us (611us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619929.227850:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 881 00002000:00100000:2.0:1788619929.227858:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df387800 x1875503571664000/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:25/0 lens 488/4536 e 0 to 0 dl 1788619940 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619929.227867:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df387800 with x1875503571664000 00000100:00100000:3.0:1788619929.231279:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619929.231379:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571664128 00000100:00100000:0.0:1788619929.231388:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571664128 00002000:00100000:0.0:1788619929.231444:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be5f80 x1875503571664128/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:25/0 lens 488/0 e 0 to 0 dl 1788619940 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619929.231456:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be5f80 with x1875503571664128 00000100:00100000:0.0:1788619929.231493:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 882 00000100:00100000:0.0:1788619929.231502:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2352:x1875503571664128:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619929.231770:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2352:x1875503571664128:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 268us (501us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619929.231778:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 882 00002000:00100000:0.0:1788619929.231786:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be5f80 x1875503571664128/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:25/0 lens 488/4536 e 0 to 0 dl 1788619940 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619929.231793:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be5f80 with x1875503571664128 00000100:00100000:3.0:1788619929.234727:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619929.234834:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571664256 00000100:00100000:1.0:1788619929.234853:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571664256 00002000:00100000:1.0:1788619929.234902:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be4e00 x1875503571664256/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:25/0 lens 488/0 e 0 to 0 dl 1788619940 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619929.234912:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be4e00 with x1875503571664256 00000100:00100000:1.0:1788619929.234953:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 883 00000100:00100000:1.0:1788619929.234963:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2353:x1875503571664256:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619929.235224:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2353:x1875503571664256:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 262us (506us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619929.235231:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 883 00002000:00100000:1.0:1788619929.235238:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4be4e00 x1875503571664256/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:25/0 lens 488/4536 e 0 to 0 dl 1788619940 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619929.235245:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4be4e00 with x1875503571664256 00000100:00100000:3.0:1788619929.238278:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619929.238636:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571664384 00000100:00100000:2.0:1788619929.238646:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571664384 00002000:00100000:2.0:1788619929.238695:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5024700 x1875503571664384/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:25/0 lens 488/0 e 0 to 0 dl 1788619940 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619929.238706:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5024700 with x1875503571664384 00000100:00100000:2.0:1788619929.238745:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 884 00000100:00100000:2.0:1788619929.238756:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5024700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571664384:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619929.239024:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5024700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571664384:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 269us (756us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619929.239032:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 884 00002000:00100000:2.0:1788619929.239038:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5024700 x1875503571664384/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:25/0 lens 488/4536 e 0 to 0 dl 1788619940 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619929.239046:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5024700 with x1875503571664384 00000100:00100000:3.0:1788619929.241911:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619929.242060:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571664512 00000100:00100000:0.0:1788619929.242069:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571664512 00002000:00100000:0.0:1788619929.242115:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c63f8700 x1875503571664512/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:25/0 lens 488/0 e 0 to 0 dl 1788619940 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619929.242127:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c63f8700 with x1875503571664512 00000100:00100000:0.0:1788619929.242167:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 885 00000100:00100000:0.0:1788619929.242177:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63f8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2352:x1875503571664512:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619929.242483:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63f8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2352:x1875503571664512:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 306us (580us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619929.242491:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 885 00002000:00100000:0.0:1788619929.242497:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c63f8700 x1875503571664512/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:25/0 lens 488/4536 e 0 to 0 dl 1788619940 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619929.242505:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c63f8700 with x1875503571664512 00000100:00100000:3.0:1788619929.245694:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619929.245793:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571664640 00000100:00100000:1.0:1788619929.245802:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571664640 00002000:00100000:1.0:1788619929.245855:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c63fb100 x1875503571664640/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:25/0 lens 488/0 e 0 to 0 dl 1788619940 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619929.245867:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c63fb100 with x1875503571664640 00000100:00100000:1.0:1788619929.245904:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 886 00000100:00100000:1.0:1788619929.245912:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63fb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2353:x1875503571664640:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619929.246176:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63fb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2353:x1875503571664640:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 263us (491us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619929.246183:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 886 00002000:00100000:1.0:1788619929.246190:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c63fb100 x1875503571664640/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:25/0 lens 488/4536 e 0 to 0 dl 1788619940 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619929.246199:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c63fb100 with x1875503571664640 00000100:00100000:3.0:1788619929.249108:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619929.249206:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571664768 00000100:00100000:2.0:1788619929.249215:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571664768 00002000:00100000:2.0:1788619929.249265:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5f27100 x1875503571664768/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:25/0 lens 488/0 e 0 to 0 dl 1788619940 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619929.249276:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5f27100 with x1875503571664768 00000100:00100000:2.0:1788619929.249312:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 887 00000100:00100000:2.0:1788619929.249322:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f27100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2352:x1875503571664768:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619929.249627:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f27100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2352:x1875503571664768:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 306us (529us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619929.249634:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 887 00002000:00100000:2.0:1788619929.249642:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5f27100 x1875503571664768/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:25/0 lens 488/4536 e 0 to 0 dl 1788619940 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619929.249650:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5f27100 with x1875503571664768 00000100:00100000:3.0:1788619929.252955:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619929.253392:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571664896 00000100:00100000:0.0:1788619929.253509:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571664896 00002000:00100000:0.0:1788619929.253563:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5f24700 x1875503571664896/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:25/0 lens 488/0 e 0 to 0 dl 1788619940 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619929.253584:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5f24700 with x1875503571664896 00000100:00100000:0.0:1788619929.253623:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 888 00000100:00100000:0.0:1788619929.253633:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f24700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2353:x1875503571664896:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619929.254002:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f24700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2353:x1875503571664896:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 369us (1055us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619929.254010:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 888 00002000:00100000:0.0:1788619929.254018:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5f24700 x1875503571664896/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:25/0 lens 488/4536 e 0 to 0 dl 1788619940 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619929.254032:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5f24700 with x1875503571664896 00000100:00100000:3.0:1788619929.256782:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619929.256869:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571665024 00000100:00100000:1.0:1788619929.256878:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571665024 00000100:00100000:1.0:1788619929.256934:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1442 00000100:00100000:1.0:1788619929.256943:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f24000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+13:2352:x1875503571665024:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619929.256964:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619929.256983:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe53687364ba lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d5d7 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.257019:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe53687364ba lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d5d7 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788619929.257032:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 978388 00010000:00010000:1.0:1788619929.257039:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe53687364ba lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d5d7 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.257052:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619929.257054:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619929.257058:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c800, rc 0) 00000100:00100000:1.0:1788619929.257258:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f24000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+14:2352:x1875503571665024:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 315us (482us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619929.257266:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1442 00000100:00100000:3.0:1788619929.260202:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619929.260301:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571665152 00000100:00100000:2.0:1788619929.260310:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571665152 00000100:00100000:2.0:1788619929.260402:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1182 00000100:00100000:2.0:1788619929.260412:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571665152:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619929.260429:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619929.260445:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0a71b34/0x1cd7fe53687364c1 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d5de expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619929.260505:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fe300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589230848:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619929.260590:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164540 going to sleep for 11 seconds 00000100:00100000:0.0:1788619929.265677:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fe300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589230848:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619929.265718:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e0a71b34/0x1cd7fe53687364c1 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d5de expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.265744:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0a71b34/0x1cd7fe53687364c1 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d5de expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.265756:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d7000, rc 0) 00000100:00100000:0.0:1788619929.265817:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571665152:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5407us (5624us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619929.265826:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1182 00000100:00100000:2.0:1788619929.268422:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619929.268542:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571665280 00000100:00100000:3.0:1788619929.268551:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571665280 00000100:00100000:3.0:1788619929.268614:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1183 00000100:00100000:3.0:1788619929.268624:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010fea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571665280:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619929.268642:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619929.268660:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1c6e115/0x1cd7fe53687364c8 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d5e5 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619929.268718:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589230976:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619929.268806:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619929.271419:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589230976:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619929.271456:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b1c6e115/0x1cd7fe53687364c8 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d5e5 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.271480:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1c6e115/0x1cd7fe53687364c8 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d5e5 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.271493:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f70200, rc 0) 00000100:00100000:3.0:1788619929.271546:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010fea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571665280:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2924us (3134us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619929.271555:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1183 00000100:00100000:0.0:1788619929.283725:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619929.283887:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571665408 00000100:00100000:1.0:1788619929.283897:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571665408 00002000:00100000:1.0:1788619929.283956:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5ff4e00 x1875503571665408/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:25/0 lens 504/0 e 0 to 0 dl 1788619940 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619929.283974:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5ff4e00 with x1875503571665408 00000100:00100000:1.0:1788619929.284014:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 889 00000100:00100000:1.0:1788619929.284024:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2352:x1875503571665408:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619929.307534:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2352:x1875503571665408:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 23500us (23808us total) trans 0 rc 917504/917504 00000100:00100000:1.0:1788619929.307549:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 889 00002000:00100000:1.0:1788619929.307562:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5ff4e00 x1875503571665408/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:25/0 lens 504/440 e 0 to 0 dl 1788619940 ref 1 fl Complete:/600/0 rc 917504/917504 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619929.307582:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5ff4e00 with x1875503571665408 00000100:00100000:2.0:1788619929.310851:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619929.310939:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571665536 00000100:00100000:3.0:1788619929.310950:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571665536 00000100:00100000:3.0:1788619929.311010:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1184 00000100:00100000:3.0:1788619929.311020:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010fc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571665536:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619929.311036:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619929.311053:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1c6e115/0x1cd7fe53687364cf lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d5ec expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619929.311107:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589231104:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619929.311196:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619929.313589:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589231104:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619929.313624:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b1c6e115/0x1cd7fe53687364cf lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d5ec expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.313654:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1c6e115/0x1cd7fe53687364cf lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d5ec expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.313667:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f70200, rc 0) 00000100:00100000:3.0:1788619929.313719:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010fc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571665536:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2699us (2889us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619929.313727:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1184 00000100:00100000:1.0:1788619929.316962:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619929.317071:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571665664 00000100:00100000:0.0:1788619929.317082:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571665664 00000100:00100000:0.0:1788619929.317134:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1185 00000100:00100000:0.0:1788619929.317144:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571665664:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619929.317164:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619929.317178:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000df72df5a/0x1cd7fe53687364d6 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d5f3 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619929.317227:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5ff5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589231232:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619929.317300:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590c00 going to sleep for 11 seconds 00000100:00100000:0.0:1788619929.319220:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5ff5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589231232:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619929.319257:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000df72df5a/0x1cd7fe53687364d6 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d5f3 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.319279:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000df72df5a/0x1cd7fe53687364d6 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d5f3 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.319292:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d5200, rc 0) 00000100:00100000:0.0:1788619929.319393:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571665664:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2250us (2451us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619929.319402:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1185 00000100:00100000:2.0:1788619929.322739:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619929.322824:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571665792 00000100:00100000:0.0:1788619929.322835:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571665792 00000100:00100000:0.0:1788619929.322895:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1186 00000100:00100000:0.0:1788619929.323048:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010fc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571665792:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619929.323066:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619929.323086:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045d2447d/0x1cd7fe53687364dd lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d5fa expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619929.323142:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5ff5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589231360:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619929.323227:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590c00 going to sleep for 11 seconds 00000100:00100000:0.0:1788619929.326166:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5ff5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589231360:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619929.326204:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000045d2447d/0x1cd7fe53687364dd lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d5fa expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.326227:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045d2447d/0x1cd7fe53687364dd lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d5fa expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.326241:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d4800, rc 0) 00000100:00100000:0.0:1788619929.326672:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010fc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571665792:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3767us (3940us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619929.326682:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1186 00000100:00100000:2.0:1788619929.330154:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619929.330236:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571665920 00000100:00100000:3.0:1788619929.330246:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571665920 00000100:00100000:3.0:1788619929.330301:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1187 00000100:00100000:3.0:1788619929.330311:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010fe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571665920:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619929.330364:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619929.330383:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1c6e115/0x1cd7fe53687364e4 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d601 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619929.330452:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589231488:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619929.330534:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619929.332787:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589231488:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619929.332822:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b1c6e115/0x1cd7fe53687364e4 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d601 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.332856:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1c6e115/0x1cd7fe53687364e4 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d601 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.332875:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f70200, rc 0) 00000100:00100000:3.0:1788619929.332929:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010fe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571665920:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2618us (2784us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619929.332938:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1187 00000100:00100000:2.0:1788619929.336112:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619929.336192:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571666048 00000100:00100000:3.0:1788619929.336203:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571666048 00000100:00100000:3.0:1788619929.336266:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1188 00000100:00100000:3.0:1788619929.336276:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dcad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571666048:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619929.336293:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619929.336311:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071401568/0x1cd7fe53687364eb lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d608 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619929.336406:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589231616:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619929.336484:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619929.338767:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589231616:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619929.338811:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000071401568/0x1cd7fe53687364eb lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d608 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.338836:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071401568/0x1cd7fe53687364eb lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d608 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.338854:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f70a00, rc 0) 00000100:00100000:3.0:1788619929.338903:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dcad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571666048:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2630us (2803us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619929.338910:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1188 00000100:00100000:2.0:1788619929.342085:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619929.342178:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571666176 00000100:00100000:0.0:1788619929.342187:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571666176 00000100:00100000:0.0:1788619929.342241:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1189 00000100:00100000:0.0:1788619929.342251:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571666176:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619929.342267:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619929.342284:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045d2447d/0x1cd7fe53687364f2 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d60f expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619929.342374:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5ff5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589231744:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619929.342464:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590c00 going to sleep for 11 seconds 00000100:00100000:0.0:1788619929.345104:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5ff5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589231744:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619929.345144:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000045d2447d/0x1cd7fe53687364f2 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d60f expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.345168:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045d2447d/0x1cd7fe53687364f2 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d60f expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.345181:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d4800, rc 0) 00000100:00100000:0.0:1788619929.345241:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571666176:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2990us (3164us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619929.345251:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1189 00000100:00100000:0.0:1788619929.352944:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619929.353162:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571666304 00000100:00100000:2.0:1788619929.353171:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571666304 00000100:00100000:2.0:1788619929.353225:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1190 00000100:00100000:2.0:1788619929.353234:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571666304:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619929.353253:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619929.353281:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011e2507b/0x1cd7fe53687364f9 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d616 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619929.353369:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fe000000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589231872:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619929.353453:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164c00 going to sleep for 11 seconds 00000100:00100000:0.0:1788619929.356025:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe000000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589231872:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619929.356060:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000011e2507b/0x1cd7fe53687364f9 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d616 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.356083:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011e2507b/0x1cd7fe53687364f9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d616 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.356095:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f73800, rc 0) 00000100:00100000:0.0:1788619929.356189:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571666304:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2955us (3254us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619929.356197:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1190 00000100:00100000:1.0:1788619929.362494:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619929.362858:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571666432 00000100:00100000:3.0:1788619929.362869:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571666432 00000100:00100000:3.0:1788619929.362944:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1191 00000100:00100000:3.0:1788619929.362955:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa40e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571666432:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619929.362975:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619929.362993:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071401568/0x1cd7fe5368736500 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d61d expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619929.363060:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589232000:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619929.363143:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619929.370027:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589232000:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619929.370057:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000071401568/0x1cd7fe5368736500 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d61d expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.370077:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071401568/0x1cd7fe5368736500 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d61d expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.370087:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f70a00, rc 0) 00000100:00100000:3.0:1788619929.370138:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa40e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571666432:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7185us (7749us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619929.370144:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1191 00000100:00100000:2.0:1788619929.374817:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619929.374979:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571666560 00000100:00100000:3.0:1788619929.374990:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571666560 00000100:00100000:3.0:1788619929.375099:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1192 00000100:00100000:3.0:1788619929.375109:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe001180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571666560:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619929.375127:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619929.375145:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1c6e115/0x1cd7fe5368736507 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d624 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619929.375201:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589232128:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619929.375312:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c0c0 going to sleep for 11 seconds 00010000:00010000:3.0:1788619929.378565:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788619929.380181:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589232128:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619929.380219:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b1c6e115/0x1cd7fe5368736507 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d624 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.380243:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1c6e115/0x1cd7fe5368736507 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d624 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.380256:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f70200, rc 0) 00000100:00100000:3.0:1788619929.380317:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe001180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571666560:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5208us (5510us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619929.380355:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1192 00000100:00100000:2.0:1788619929.383671:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619929.383999:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571666688 00000100:00100000:0.0:1788619929.384008:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571666688 00000100:00100000:0.0:1788619929.384071:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1193 00000100:00100000:0.0:1788619929.384082:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe000000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571666688:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619929.384104:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619929.384126:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045d2447d/0x1cd7fe536873650e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d62b expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619929.384191:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5ff5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589232256:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619929.384280:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590c00 going to sleep for 11 seconds 00000100:00100000:0.0:1788619929.386688:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5ff5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589232256:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619929.386725:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000045d2447d/0x1cd7fe536873650e lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d62b expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.386751:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045d2447d/0x1cd7fe536873650e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d62b expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.386765:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d4800, rc 0) 00000100:00100000:0.0:1788619929.386825:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe000000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571666688:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2745us (3163us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619929.386833:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1193 00000100:00100000:1.0:1788619929.391705:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619929.392005:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571666816 00000100:00100000:3.0:1788619929.392019:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571666816 00000100:00100000:3.0:1788619929.392088:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1443 00000100:00100000:3.0:1788619929.392098:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df966300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:14908:x1875503571666816:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619929.392121:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619929.392125:0:5844:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1788619929.392176:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687364b3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d5d0 expref: 16 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.392196:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619929.392202:0:5844:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788619929.392215:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70200/0x1cd7fe5368736515 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d632 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.392241:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe53687364ba lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d5d7 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.392250:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70200/0x1cd7fe5368736515 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d632 expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.392284:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe53687364ba lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d5d7 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.392297:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe53687364ba lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d5d7 expref: 13 pid: 5845 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619929.392311:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501bb80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589232384:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619929.392619:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619929.397229:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619929.397357:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571666944 00000100:00100000:1.0:1788619929.397368:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571666944 00010000:00010000:1.0:1788619929.397421:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe53687364ba lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d5d7 expref: 14 pid: 5845 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619929.397469:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 479 00000100:00100000:1.0:1788619929.397480:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8595880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+14:2355:x1875503571666944:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619929.397497:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619929.397525:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe53687364ba lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d5d7 expref: 14 pid: 5845 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.397540:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe53687364ba lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d5d7 expref: 14 pid: 5845 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.397563:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619929.397578:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70200/0x1cd7fe5368736515 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d632 expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.397591:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70200/0x1cd7fe5368736515 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d632 expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.397601:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70200/0x1cd7fe5368736515 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d632 expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.397648:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70200/0x1cd7fe5368736515 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d632 expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619929.397663:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df964700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589232512:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619929.397747:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebcc0 going to sleep for 11 seconds 00010000:00010000:3.0:1788619929.400251:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe53687364ba lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d5d7 expref: 14 pid: 5845 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619929.400276:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501bb80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589232384:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619929.400306:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70200/0x1cd7fe5368736515 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d632 expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.400321:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619929.400348:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619929.400352:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f70200, rc 0) 00000100:00100000:3.0:1788619929.400417:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df966300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:14908:x1875503571666816:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8320us (8722us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619929.400426:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1443 00000100:00100000:3.0:1788619929.400954:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df964700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589232512:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619929.400969:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619929.401010:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8595880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+13:2355:x1875503571666944:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3531us (3792us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619929.401019:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 479 00000100:00100000:0.0:1788619929.404512:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619929.404740:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571667072 00000100:00100000:2.0:1788619929.404752:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571667072 00002000:00100000:2.0:1788619929.404810:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5ff5180 x1875503571667072/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:25/0 lens 488/0 e 0 to 0 dl 1788619940 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619929.404821:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5ff5180 with x1875503571667072 00000100:00100000:2.0:1788619929.404867:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 890 00000100:00100000:2.0:1788619929.404876:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2352:x1875503571667072:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619929.405157:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2352:x1875503571667072:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 282us (657us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619929.405165:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 890 00002000:00100000:2.0:1788619929.405172:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5ff5180 x1875503571667072/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:25/0 lens 488/4536 e 0 to 0 dl 1788619940 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619929.405180:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5ff5180 with x1875503571667072 00000100:00100000:2.0:1788619929.409933:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619929.410008:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571667200 00000100:00100000:3.0:1788619929.410017:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571667200 00000100:00100000:3.0:1788619929.410075:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1444 00000100:00100000:3.0:1788619929.410084:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8597800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571667200:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619929.410104:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619929.410124:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71600/0x1cd7fe536873651c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d639 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.410159:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70200/0x1cd7fe5368736515 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d632 expref: 15 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619929.410198:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501bb80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589232640:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619929.410287:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619929.413504:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000004:00010000:1.0:1788619929.413679:0:11397:0:(mdt_lvb.c:177:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size: 978388 -> 1002001 00000004:00010000:1.0:1788619929.413684:0:11397:0:(mdt_lvb.c:183:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime: 1788619928 -> 1788619929 00000004:00010000:1.0:1788619929.413687:0:11397:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619928 -> 1788619929 00000100:00100000:1.0:1788619929.413708:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501bb80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589232640:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619929.413728:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1002001 00010000:00010000:1.0:1788619929.413745:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71600/0x1cd7fe536873651c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d639 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.413764:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619929.413767:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619929.413772:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71600/0x1cd7fe536873651c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d639 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.413787:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f71600, rc 0) 00000100:00100000:1.0:1788619929.413860:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8597800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571667200:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3775us (3934us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619929.413869:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1444 00000100:00100000:0.0:1788619929.414071:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571667328 00000100:00100000:0.0:1788619929.414079:0:5836:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571667328 00000100:00100000:0.0:1788619929.414128:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 295 00000100:00100000:0.0:1788619929.414137:0:5836:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5caf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:597450a2-e90c-408c-b401-a0ce48eed204+13:2353:x1875503571667328:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:0.0:1788619929.414173:0:5836:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5caf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:597450a2-e90c-408c-b401-a0ce48eed204+13:2353:x1875503571667328:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 38us (678us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619929.414178:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 295 00000100:00100000:2.0:1788619929.418082:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619929.418161:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571667456 00000100:00100000:1.0:1788619929.418171:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571667456 00000100:00100000:1.0:1788619929.418235:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1445 00000100:00100000:1.0:1788619929.418246:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8594380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+13:2352:x1875503571667456:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619929.418268:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619929.418289:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368736523 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d640 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.418376:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70200/0x1cd7fe5368736515 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d632 expref: 15 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619929.418422:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df965880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589232768:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619929.418501:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb000 going to sleep for 11 seconds 00000100:00100000:1.0:1788619929.421680:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df965880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589232768:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619929.421708:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1002001 00010000:00010000:1.0:1788619929.421729:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368736523 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d640 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.421750:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619929.421754:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619929.421760:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368736523 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d640 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.421775:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683cc00, rc 0) 00000100:00100000:1.0:1788619929.421834:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8594380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+13:2352:x1875503571667456:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3591us (3760us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619929.421843:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1445 00000100:00100000:3.0:1788619929.425314:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619929.425474:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571667584 00000100:00100000:1.0:1788619929.425484:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571667584 00000100:00100000:1.0:1788619929.425533:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1446 00000100:00100000:1.0:1788619929.425542:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5018e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571667584:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619929.425563:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619929.425595:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe536873652a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d647 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.425644:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70200/0x1cd7fe5368736515 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d632 expref: 15 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619929.425688:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df965880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589232896:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619929.425779:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb000 going to sleep for 11 seconds 00000100:00100000:1.0:1788619929.429065:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df965880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589232896:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619929.429095:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1002001 00010000:00010000:1.0:1788619929.429116:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe536873652a lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d647 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.429138:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619929.429142:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619929.429149:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe536873652a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d647 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.429166:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d000, rc 0) 00000100:00100000:1.0:1788619929.429235:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5018e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571667584:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3692us (3929us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619929.429244:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1446 00000100:00100000:2.0:1788619929.432107:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619929.432189:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571667712 00000100:00100000:1.0:1788619929.432198:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571667712 00000100:00100000:1.0:1788619929.432264:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1447 00000100:00100000:1.0:1788619929.432276:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8595c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+13:2355:x1875503571667712:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619929.432297:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619929.432320:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368736531 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d64e expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.432387:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70200/0x1cd7fe5368736515 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d632 expref: 15 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619929.432440:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df965880 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589233024:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619929.432519:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb000 going to sleep for 11 seconds 00000100:00100000:1.0:1788619929.435129:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df965880 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589233024:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619929.435155:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1002001 00010000:00010000:1.0:1788619929.435177:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368736531 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d64e expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.435200:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619929.435203:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619929.435209:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368736531 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d64e expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.435222:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683cc00, rc 0) 00000100:00100000:1.0:1788619929.435408:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8595c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+13:2355:x1875503571667712:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3132us (3307us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619929.435417:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1447 00000100:00100000:2.0:1788619929.439092:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619929.439167:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571667840 00000100:00100000:1.0:1788619929.439178:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571667840 00000100:00100000:1.0:1788619929.439238:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1448 00000100:00100000:1.0:1788619929.439247:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8594a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+13:2355:x1875503571667840:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619929.439269:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619929.439291:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368736538 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d655 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.439371:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70200/0x1cd7fe5368736515 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d632 expref: 15 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619929.439421:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df965880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589233152:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619929.439498:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb000 going to sleep for 11 seconds 00000100:00100000:1.0:1788619929.442422:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df965880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589233152:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619929.442448:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1002001 00010000:00010000:1.0:1788619929.442468:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368736538 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d655 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.442489:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619929.442493:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619929.442499:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368736538 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d655 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.442514:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d000, rc 0) 00000100:00100000:1.0:1788619929.442584:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8594a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+13:2355:x1875503571667840:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3337us (3501us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619929.442593:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1448 00000100:00100000:3.0:1788619929.445783:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619929.446252:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571667968 00000100:00100000:1.0:1788619929.446263:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571667968 00000100:00100000:1.0:1788619929.446316:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1449 00000100:00100000:1.0:1788619929.446354:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+13:2353:x1875503571667968:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619929.446376:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619929.446407:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe536873653f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d65c expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.446446:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70200/0x1cd7fe5368736515 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d632 expref: 15 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619929.446485:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df965880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589233280:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619929.446561:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb000 going to sleep for 11 seconds 00000100:00100000:1.0:1788619929.449554:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df965880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589233280:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619929.449592:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1002001 00010000:00010000:1.0:1788619929.449610:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe536873653f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d65c expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.449630:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619929.449642:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619929.449649:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe536873653f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d65c expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.449664:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683cc00, rc 0) 00000100:00100000:1.0:1788619929.449720:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+13:2353:x1875503571667968:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3394us (3948us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619929.449728:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1449 00000100:00100000:2.0:1788619929.453317:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619929.453527:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571668096 00000100:00100000:1.0:1788619929.453537:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571668096 00000100:00100000:1.0:1788619929.453614:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 891 00000100:00100000:1.0:1788619929.453624:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8594700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2352:x1875503571668096:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:1.0:1788619929.453866:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1002001 -> 978388 00000004:00010000:1.0:1788619929.453870:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619929 -> 1788619929 00000004:00010000:1.0:1788619929.453873:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619929 -> 1788619928 00000004:00010000:1.0:1788619929.453875:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619929 -> 1788619929 00000004:00010000:1.0:1788619929.453877:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1158 -> 1158 00000100:00100000:1.0:1788619929.453955:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8594700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2352:x1875503571668096:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 333us (648us total) trans 4294967804 rc 0/0 00000100:00100000:1.0:1788619929.453962:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 891 00000100:00100000:3.0:1788619929.457316:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619929.457694:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571668224 00000100:00100000:1.0:1788619929.457706:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571668224 00000100:00100000:1.0:1788619929.457805:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1450 00000100:00100000:1.0:1788619929.457815:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6003100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+13:2352:x1875503571668224:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619929.457844:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619929.457865:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368736546 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d663 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.457923:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70200/0x1cd7fe5368736515 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d632 expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619929.457964:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df965880 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589233408:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619929.458053:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb000 going to sleep for 11 seconds 00000004:00010000:1.0:1788619929.460493:0:11397:0:(mdt_lvb.c:177:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size: 978388 -> 1002001 00000004:00010000:1.0:1788619929.460503:0:11397:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619928 -> 1788619929 00000100:00100000:1.0:1788619929.460527:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df965880 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589233408:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619929.460550:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1002001 00010000:00010000:1.0:1788619929.460577:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368736546 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d663 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.460598:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619929.460601:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619929.460606:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368736546 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d663 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.460622:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683cc00, rc 0) 00000100:00100000:1.0:1788619929.460753:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6003100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+13:2352:x1875503571668224:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2938us (3446us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619929.460762:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1450 00000100:00100000:2.0:1788619929.463847:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619929.463987:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571668352 00000100:00100000:3.0:1788619929.463997:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571668352 00000100:00100000:3.0:1788619929.464058:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1451 00000100:00100000:3.0:1788619929.464069:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e6478380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+13:2352:x1875503571668352:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619929.464089:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619929.464109:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70a00/0x1cd7fe536873654d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d66a expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.464146:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70200/0x1cd7fe5368736515 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d632 expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619929.464187:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6000000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589233536:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619929.464272:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48ce40 going to sleep for 11 seconds 00000100:00100000:1.0:1788619929.467554:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6000000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589233536:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619929.467594:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1002001 00010000:00010000:1.0:1788619929.467615:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70a00/0x1cd7fe536873654d lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d66a expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.467653:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619929.467656:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619929.467662:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70a00/0x1cd7fe536873654d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d66a expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.467678:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f70a00, rc 0) 00000100:00100000:1.0:1788619929.467791:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e6478380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+13:2352:x1875503571668352:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3723us (3963us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619929.467799:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1451 00000100:00100000:2.0:1788619929.473259:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619929.473521:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571668480 00000100:00100000:3.0:1788619929.473534:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571668480 00000100:00100000:3.0:1788619929.473603:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1452 00000100:00100000:3.0:1788619929.473613:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e6479880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+13:2352:x1875503571668480:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619929.473969:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619929.473992:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70e00/0x1cd7fe5368736554 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d671 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.474034:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70200/0x1cd7fe5368736515 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d632 expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619929.474073:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6000e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589233664:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619929.474160:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c240 going to sleep for 11 seconds 00000100:00100000:3.0:1788619929.477028:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6000e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589233664:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619929.477055:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1002001 00010000:00010000:3.0:1788619929.477074:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70e00/0x1cd7fe5368736554 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d671 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.477097:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619929.477101:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619929.477106:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70e00/0x1cd7fe5368736554 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d671 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.477122:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f70e00, rc 0) 00000100:00100000:3.0:1788619929.477185:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e6479880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+13:2352:x1875503571668480:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3572us (3939us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619929.477193:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1452 00000100:00100000:0.0:1788619929.481505:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619929.481791:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571668608 00000100:00100000:2.0:1788619929.481803:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571668608 00000100:00100000:2.0:1788619929.481929:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1453 00000100:00100000:2.0:1788619929.481940:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+13:2352:x1875503571668608:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619929.481965:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619929.481988:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72000/0x1cd7fe536873655b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d678 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.482028:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70200/0x1cd7fe5368736515 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d632 expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619929.482071:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e6479180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589233792:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619929.482164:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164540 going to sleep for 11 seconds 00000100:00100000:0.0:1788619929.484583:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e6479180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589233792:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619929.484613:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1002001 00010000:00010000:0.0:1788619929.484641:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72000/0x1cd7fe536873655b lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d678 expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.484661:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619929.484665:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619929.484672:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72000/0x1cd7fe536873655b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d678 expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.484688:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f72000, rc 0) 00000100:00100000:0.0:1788619929.484755:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+13:2352:x1875503571668608:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2817us (3260us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619929.484765:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1453 00000100:00100000:0.0:1788619929.488720:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619929.488860:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571668864 00000100:00100000:1.0:1788619929.488870:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571668864 00000100:00100000:1.0:1788619929.488937:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1454 00000100:00100000:1.0:1788619929.488947:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+17:14908:x1875503571668864:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:1.0:1788619929.489009:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619929.489036:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7c00/0x1cd7fe5368736420 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d544 expref: 17 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.489055:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619929.489060:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788619929.489266:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368736562 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.489281:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70600/0x1cd7fe5368736482 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d59f expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.489290:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368736562 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.489350:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70600/0x1cd7fe5368736482 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d59f expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.489365:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70600/0x1cd7fe5368736482 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d59f expref: 12 pid: 9417 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619929.489381:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df965880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589233920:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619929.489475:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb000 going to sleep for 11 seconds 00000100:00100000:1.0:1788619929.492832:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df965880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589233920:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619929.492871:0:5845:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368736562 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619929.494316:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619929.494527:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571668992 00000100:00100000:2.0:1788619929.494537:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571668992 00010000:00010000:2.0:1788619929.494591:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70600/0x1cd7fe5368736482 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d59f expref: 13 pid: 9417 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619929.494641:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 480 00000100:00100000:2.0:1788619929.494651:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcafc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+13:2355:x1875503571668992:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619929.494665:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619929.494672:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70600/0x1cd7fe5368736482 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d59f expref: 13 pid: 9417 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.494685:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70600/0x1cd7fe5368736482 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d59f expref: 13 pid: 9417 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.494708:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70600/0x1cd7fe5368736482 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d59f expref: 13 pid: 9417 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.494724:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619929.494729:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368736562 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.494739:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368736562 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.494747:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368736562 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.494775:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788619929.495037:0:5845:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368736562 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.495046:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368736562 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788619929.495153:0:5845:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967805 00000004:00010000:1.0:1788619929.495200:0:5845:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1002001 -> 978388 00000004:00010000:1.0:1788619929.495203:0:5845:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619929 -> 1788619929 00000004:00010000:1.0:1788619929.495206:0:5845:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619929 -> 1788619928 00000004:00010000:1.0:1788619929.495208:0:5845:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619929 -> 1788619929 00000004:00010000:1.0:1788619929.495211:0:5845:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1158 -> 1158 00000004:00010000:1.0:1788619929.495217:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000a9ae5381 reply state 00000000a0c1c0d4 transno 4294967806 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368736562 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619929.495232:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788619929.495300:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+19:14908:x1875503571668864:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 6354us (6589us total) trans 4294967806 rc 0/0 00000100:00100000:1.0:1788619929.495308:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1454 00000020:00080000:1.0:1788619929.495313:0:5845:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1788619929.495401:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcafc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571668992:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 182us (524us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619929.495413:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 480 00010000:00010000:0.0:1788619929.497830:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368736562 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.497856:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368736562 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.497864:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368736562 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.497870:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c683cc00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788619929.497874:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368736562 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.497881:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368736562 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.497896:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619929.497899:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788619929.497914:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368736562 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.497925:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368736562 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619929.499864:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619929.499948:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571669120 00000100:00100000:2.0:1788619929.499958:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571669120 00000100:00100000:2.0:1788619929.500020:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1194 00000100:00100000:2.0:1788619929.500032:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:14908:x1875503571669120:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619929.500051:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619929.500069:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c0482c0/0x1cd7fe5368736569 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d67f expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.500101:0:6698:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000614d4133/0x1cd7fe5368736412 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d53d expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.500111:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=1s v=(3 12) 00010000:00010000:2.0:1788619929.500117:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(4 12) 00010000:00010000:2.0:1788619929.500122:0:6698:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c0482c0/0x1cd7fe5368736569 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d67f expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.500162:0:6698:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000614d4133/0x1cd7fe5368736412 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d53d expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.500177:0:6698:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000614d4133/0x1cd7fe5368736412 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d53d expref: 6 pid: 7654 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619929.500191:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e647b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589234048:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619929.500279:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164840 going to sleep for 11 seconds 00000100:00100000:3.0:1788619929.503788:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619929.503909:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e647b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589234048:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619929.503947:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005c0482c0/0x1cd7fe5368736569 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d67f expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.503969:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f73000, rc 0) 00000100:00100000:2.0:1788619929.504035:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:14908:x1875503571669120:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4006us (4181us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619929.504044:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1194 00000100:00100000:0.0:1788619929.505505:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571669248 00000100:00100000:0.0:1788619929.505519:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571669248 00010000:00010000:0.0:1788619929.505561:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000614d4133/0x1cd7fe5368736412 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d53d expref: 7 pid: 7654 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619929.505616:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 481 00000100:00100000:0.0:1788619929.505626:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6000e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571669248:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619929.505650:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619929.505676:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000614d4133/0x1cd7fe5368736412 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d53d expref: 7 pid: 7654 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.505691:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000614d4133/0x1cd7fe5368736412 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d53d expref: 7 pid: 7654 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.505712:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000614d4133/0x1cd7fe5368736412 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d53d expref: 7 pid: 7654 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.505726:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(4 12) 00010000:00010000:0.0:1788619929.505730:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(4 12) 00010000:00010000:0.0:1788619929.505738:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000005c0482c0/0x1cd7fe5368736569 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d67f expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.505753:0:13330:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c0482c0/0x1cd7fe5368736569 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d67f expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.505789:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000005c0482c0/0x1cd7fe5368736569 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d67f expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619929.505803:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5ff7480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589234176:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619929.506149:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590b40 going to sleep for 11 seconds 00000100:00100000:1.0:1788619929.510944:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619929.511150:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5ff7480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589234176:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619929.511176:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619929.511242:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6000e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571669248:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 5617us (7464us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619929.511254:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 481 00000100:00100000:3.0:1788619929.512322:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571669376 00000100:00100000:3.0:1788619929.512521:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571669376 00002000:00010000:2.0:1788619929.512640:0:6701:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1571283->18446744073709551615) 00002000:00100000:2.0:1788619929.512646:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df965880 with x1875503571669376 ext(1571283->18446744073709551615) 00002000:00010000:2.0:1788619929.512657:0:6701:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df965880. 00000100:00100000:2.0:1788619929.512717:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 683 00000100:00100000:2.0:1788619929.512727:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df965880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571669376:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:2.0:1788619929.512957:0:6701:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 0 -> 1571283 00002000:00010000:2.0:1788619929.512962:0:6701:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619928 -> 1788619929 00002000:00010000:2.0:1788619929.512965:0:6701:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619928 -> 1788619927 00002000:00010000:2.0:1788619929.512967:0:6701:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619929 -> 1788619929 00002000:00010000:2.0:1788619929.512969:0:6701:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 391 -> 391 00000100:00100000:2.0:1788619929.513027:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df965880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571669376:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 301us (2092us total) trans 4294967498 rc 0/0 00000100:00100000:2.0:1788619929.513034:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 683 00002000:00010000:2.0:1788619929.513038:0:6701:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1571283->18446744073709551615) 00002000:00100000:2.0:1788619929.513040:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df965880 with x1875503571669376 ext(1571283->18446744073709551615) 00002000:00010000:2.0:1788619929.513045:0:6701:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df965880. 00000100:00100000:1.0:1788619929.518416:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619929.518470:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619929.518629:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571669504 00000100:00100000:2.0:1788619929.518639:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571669504 00002000:00100000:2.0:1788619929.518691:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1572863) req@ffff8e78df964e00 x1875503571669504/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:25/0 lens 488/0 e 0 to 0 dl 1788619940 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619929.518703:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df964e00 with x1875503571669504 ext(1048576->1572863) 00002000:00010000:2.0:1788619929.518709:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df964e00 00000100:00100000:2.0:1788619929.518744:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 684 00000100:00100000:2.0:1788619929.518755:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df964e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2352:x1875503571669504:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619929.520871:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571669632 00000100:00100000:0.0:1788619929.520887:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571669632 00002000:00100000:0.0:1788619929.520940:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df964a80 x1875503571669632/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:25/0 lens 488/0 e 0 to 0 dl 1788619940 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619929.520952:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df964a80 with x1875503571669632 00000100:00100000:0.0:1788619929.520995:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 892 00000100:00100000:0.0:1788619929.521005:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df964a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2352:x1875503571669632:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619929.526798:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df964e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2352:x1875503571669504:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 8032us (8315us total) trans 0 rc 524288/524288 00000100:00100000:2.0:1788619929.526813:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 684 00002000:00100000:2.0:1788619929.526825:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1572863) req@ffff8e78df964e00 x1875503571669504/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:25/0 lens 488/440 e 0 to 0 dl 1788619940 ref 1 fl Complete:/600/0 rc 524288/524288 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619929.526846:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df964e00 with x1875503571669504 ext(1048576->1572863) 00002000:00010000:2.0:1788619929.526853:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df964e00 00000100:00100000:0.0:1788619929.527175:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df964a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2352:x1875503571669632:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 6170us (8778us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619929.527184:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 892 00002000:00100000:0.0:1788619929.527192:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df964a80 x1875503571669632/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:25/0 lens 488/440 e 0 to 0 dl 1788619940 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619929.527201:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df964a80 with x1875503571669632 00000100:00100000:0.0:1788619929.533059:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619929.533167:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571669760 00000100:00100000:1.0:1788619929.533178:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571669760 00002000:00100000:1.0:1788619929.533229:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1548288->1571282) req@ffff8e77c5ff7480 x1875503571669760/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:25/0 lens 488/0 e 0 to 0 dl 1788619940 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619929.533242:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5ff7480 with x1875503571669760 ext(1548288->1571282) 00002000:00010000:1.0:1788619929.533248:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5ff7480 00000100:00100000:1.0:1788619929.533280:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 685 00000100:00100000:1.0:1788619929.533290:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571669760:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:1.0:1788619929.537352:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571669760:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 4036us (4276us total) trans 4294967499 rc 0/0 00000100:00100000:1.0:1788619929.537366:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 685 00002000:00100000:1.0:1788619929.537381:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1548288->1571282) req@ffff8e77c5ff7480 x1875503571669760/t4294967499(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:25/0 lens 488/448 e 0 to 0 dl 1788619940 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619929.537404:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5ff7480 with x1875503571669760 ext(1548288->1571282) 00002000:00010000:1.0:1788619929.537411:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5ff7480 00000100:00100000:0.0:1788619929.541272:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619929.541355:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571669888 00000100:00100000:3.0:1788619929.541367:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571669888 00000100:00100000:3.0:1788619929.541436:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1195 00000100:00100000:3.0:1788619929.541446:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571669888:12345-192.168.201.5@tcp:16:fsx.0 00000100:00100000:2.0:1788619929.565938:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571669888:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 24483us (24666us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619929.565952:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1195 00000001:00080000:1.0:1788619929.566028:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967499 is committed 00000001:00080000:1.0:1788619929.566046:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967497 is committed 00000100:00100000:1.0:1788619929.568917:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619929.569015:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571670016 00000100:00100000:3.0:1788619929.569024:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571670016 00000100:00100000:3.0:1788619929.569117:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1455 00000100:00100000:3.0:1788619929.569127:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df966a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:14908:x1875503571670016:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619929.569148:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619929.569169:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70e00/0x1cd7fe5368736570 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d686 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.569257:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70000/0x1cd7fe5368736577 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.569274:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70000/0x1cd7fe5368736577 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.569285:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70000/0x1cd7fe5368736577 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788619929.569311:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1571283 00000004:00200000:3.0:1788619929.569316:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788619929.569357:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70000/0x1cd7fe5368736577 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.569400:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70e00/0x1cd7fe5368736570 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d686 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.569424:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70000/0x1cd7fe5368736577 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d686 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.569434:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619929.569437:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788619929.569442:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f70000, rc 0) 00000100:00100000:3.0:1788619929.569520:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df966a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+13:14908:x1875503571670016:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 394us (614us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619929.569527:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1455 00000100:00100000:0.0:1788619929.573037:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619929.573126:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619929.573618:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571670272 00000100:00100000:3.0:1788619929.573630:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571670272 00000100:00100000:3.0:1788619929.573696:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1196 00000100:00100000:3.0:1788619929.573706:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b16d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571670272:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619929.573724:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619929.573741:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fe91493a/0x1cd7fe536873657e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d694 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619929.573798:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6000000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589234304:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619929.574002:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c240 going to sleep for 11 seconds 00000100:00100000:1.0:1788619929.574800:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571670144 00000100:00100000:1.0:1788619929.574810:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571670144 00000100:00100000:1.0:1788619929.574866:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1456 00000100:00100000:1.0:1788619929.574875:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+13:2353:x1875503571670144:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619929.574890:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619929.574904:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368736585 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d68d expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.574937:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70200/0x1cd7fe5368736515 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d632 expref: 17 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619929.574965:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df967800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589234432:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619929.575020:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb0c0 going to sleep for 11 seconds 00002000:00010000:3.0:1788619929.577636:0:6696:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619927 -> 1788619929 00000100:00100000:3.0:1788619929.577667:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6000000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589234304:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619929.577700:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000fe91493a/0x1cd7fe536873657e lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d694 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.577724:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fe91493a/0x1cd7fe536873657e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d694 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.577738:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f71800, rc 0) 00000004:00010000:1.0:1788619929.578061:0:9417:0:(mdt_lvb.c:177:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size: 978388 -> 1002001 00000004:00010000:1.0:1788619929.578066:0:9417:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619928 -> 1788619929 00000100:00100000:1.0:1788619929.578083:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df967800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589234432:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619929.578097:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1002001 00010000:00010000:1.0:1788619929.578108:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368736585 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d68d expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.578121:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619929.578124:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619929.578129:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368736585 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d68d expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.578140:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c400, rc 0) 00000100:00100000:3.0:1788619929.578181:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b16d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571670272:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4475us (5053us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619929.578190:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1196 00000100:00100000:1.0:1788619929.578517:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+13:2353:x1875503571670144:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3642us (5490us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619929.578526:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1456 00000100:00100000:2.0:1788619929.583095:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619929.583141:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619929.583223:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571670400 00000100:00100000:1.0:1788619929.583232:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571670400 00000100:00100000:1.0:1788619929.583294:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1457 00000100:00100000:1.0:1788619929.583304:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e647aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+13:2352:x1875503571670400:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619929.583341:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619929.583363:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe536873658c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d69b expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.583411:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70200/0x1cd7fe5368736515 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d632 expref: 17 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619929.583456:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df967800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589234560:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619929.583536:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619929.585221:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571670528 00000100:00100000:0.0:1788619929.585231:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571670528 00000100:00100000:0.0:1788619929.585290:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1197 00000100:00100000:0.0:1788619929.585301:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e647b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571670528:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619929.585315:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619929.585359:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bdacd512/0x1cd7fe5368736593 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d6a2 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619929.585423:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b14a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589234688:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619929.585502:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:1.0:1788619929.587970:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df967800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589234560:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619929.587996:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1002001 00010000:00010000:1.0:1788619929.588015:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe536873658c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d69b expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.588034:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619929.588037:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619929.588043:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe536873658c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d69b expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.588057:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683dc00, rc 0) 00000100:00100000:1.0:1788619929.588111:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e647aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+13:2352:x1875503571670400:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4808us (4968us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619929.588120:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1457 00000100:00100000:3.0:1788619929.589260:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b14a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589234688:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619929.589294:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bdacd512/0x1cd7fe5368736593 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6a2 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.589321:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bdacd512/0x1cd7fe5368736593 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6a2 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.589353:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d4c00, rc 0) 00000100:00100000:3.0:1788619929.589414:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e647b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571670528:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4115us (6328us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619929.589422:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1197 00000100:00100000:0.0:1788619929.591657:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619929.591734:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571670656 00000100:00100000:3.0:1788619929.591743:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571670656 00000100:00100000:3.0:1788619929.591800:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1458 00000100:00100000:3.0:1788619929.591810:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b15500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+18:14908:x1875503571670656:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619929.591833:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619929.591854:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70e00/0x1cd7fe536873659a lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d6a9 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.592008:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71c00/0x1cd7fe53687365a1 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.592026:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71c00/0x1cd7fe53687365a1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.592038:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71c00/0x1cd7fe53687365a1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788619929.592066:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1571283 00000004:00200000:3.0:1788619929.592071:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788619929.592089:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71c00/0x1cd7fe53687365a1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.592114:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70e00/0x1cd7fe536873659a lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6a9 expref: 20 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.592131:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71c00/0x1cd7fe53687365a1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6a9 expref: 19 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.592140:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619929.592143:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788619929.592146:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f71c00, rc 0) 00000100:00100000:3.0:1788619929.592241:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b15500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+19:14908:x1875503571670656:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 432us (595us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619929.592247:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1458 00000100:00100000:1.0:1788619929.599554:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619929.599644:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571670784 00000100:00100000:0.0:1788619929.599654:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571670784 00000100:00100000:0.0:1788619929.599719:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1459 00000100:00100000:0.0:1788619929.599730:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df967800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+13:14908:x1875503571670784:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619929.599750:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619929.599773:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5400/0x1cd7fe53687365a8 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d6b0 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.599802:0:9417:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70200/0x1cd7fe5368736515 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d632 expref: 18 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.599814:0:9417:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5400/0x1cd7fe53687365a8 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6b0 expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.599860:0:9417:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70200/0x1cd7fe5368736515 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d632 expref: 18 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.599874:0:9417:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70200/0x1cd7fe5368736515 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d632 expref: 18 pid: 5844 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619929.599887:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b14e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589234816:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619929.599972:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:0.0:1788619929.602095:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619929.602405:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571670912 00000100:00100000:3.0:1788619929.602413:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571670912 00002000:00100000:3.0:1788619929.602461:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b15880 x1875503571670912/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:25/0 lens 488/0 e 0 to 0 dl 1788619940 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619929.602473:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b15880 with x1875503571670912 00010000:00010000:3.0:1788619929.602481:0:5857:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70200/0x1cd7fe5368736515 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d632 expref: 19 pid: 5844 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.602496:0:5857:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70200/0x1cd7fe5368736515 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d632 expref: 19 pid: 5844 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788619929.602505:0:5857:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e77c6b15880 00000100:00100000:3.0:1788619929.602530:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 37 00000100:00100000:3.0:1788619929.602540:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b15880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2354:x1875503571670912:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:1.0:1788619929.603121:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b14e00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589234816:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619929.603154:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5400/0x1cd7fe53687365a8 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6b0 expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.603171:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619929.603174:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619929.603179:0:9417:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5400/0x1cd7fe53687365a8 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6b0 expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.603191:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d5400, rc 0) 00000100:00100000:1.0:1788619929.603227:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df967800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+14:14908:x1875503571670784:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3499us (3683us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619929.603235:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1459 00000004:00010000:2.0:1788619929.607728:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1002001 -> 1002001 00000004:00010000:2.0:1788619929.607738:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619929 -> 1788619929 00000004:00010000:2.0:1788619929.607741:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619929 -> 1788619928 00000004:00010000:2.0:1788619929.607743:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619929 -> 1788619929 00000004:00010000:2.0:1788619929.607746:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1158 -> 1158 00000100:00100000:2.0:1788619929.607813:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b15880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2354:x1875503571670912:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 5269us (5723us total) trans 4294967807 rc 0/0 00000100:00100000:2.0:1788619929.607824:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 37 00002000:00100000:2.0:1788619929.607849:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b15880 x1875503571670912/t4294967807(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:25/0 lens 488/448 e 0 to 0 dl 1788619940 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619929.607859:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b15880 with x1875503571670912 00010000:00010000:2.0:1788619929.607869:0:5857:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70200/0x1cd7fe5368736515 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d632 expref: 21 pid: 5844 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.607885:0:5857:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70200/0x1cd7fe5368736515 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d632 expref: 21 pid: 5844 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788619929.607893:0:5857:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e77c6b15880 00000100:00100000:3.0:1788619929.617185:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619929.617296:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571671040 00000100:00100000:2.0:1788619929.617307:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571671040 00010000:00010000:0.0:1788619929.625914:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70200/0x1cd7fe5368736515 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d632 expref: 21 pid: 5844 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619929.625974:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 482 00000100:00100000:0.0:1788619929.625988:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6000000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2352:x1875503571671040:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619929.626004:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619929.626027:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70200/0x1cd7fe5368736515 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d632 expref: 21 pid: 5844 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.626040:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70200/0x1cd7fe5368736515 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d632 expref: 21 pid: 5844 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.626059:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70200/0x1cd7fe5368736515 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d632 expref: 21 pid: 5844 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.626078:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619929.626084:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5400/0x1cd7fe53687365a8 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6b0 expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.626095:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5400/0x1cd7fe53687365a8 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6b0 expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.626104:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5400/0x1cd7fe53687365a8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6b0 expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.626141:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5400/0x1cd7fe53687365a8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6b0 expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619929.626153:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b17480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589234944:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619929.626241:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619929.631702:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b17480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589234944:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619929.631741:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619929.631803:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6000000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2352:x1875503571671040:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 5819us (14625us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619929.631815:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 482 00000100:00100000:3.0:1788619929.632025:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619929.632211:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571671168 00000100:00100000:1.0:1788619929.632219:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571671168 00000100:00100000:1.0:1788619929.632278:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 893 00000100:00100000:1.0:1788619929.632289:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+14:2354:x1875503571671168:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:1.0:1788619929.632615:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1002001 -> 1002001 00000004:00010000:1.0:1788619929.632620:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619929 -> 1788619929 00000004:00010000:1.0:1788619929.632623:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619928 -> 1788619928 00000004:00010000:1.0:1788619929.632625:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619929 -> 1788619929 00000004:00010000:1.0:1788619929.632627:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1158 -> 1158 00000100:00100000:1.0:1788619929.632672:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+15:2354:x1875503571671168:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 384us (646us total) trans 4294967808 rc 0/0 00000100:00100000:1.0:1788619929.632679:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 893 00000100:00100000:3.0:1788619929.642428:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619929.645867:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571671296 00000100:00100000:1.0:1788619929.645885:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571671296 00000100:00100000:1.0:1788619929.645969:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1460 00000100:00100000:1.0:1788619929.645980:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2355:x1875503571671296:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619929.646005:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619929.646027:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe53687365af lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d6b7 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.646069:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5400/0x1cd7fe53687365a8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6b0 expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619929.646112:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df965f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589235072:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619929.646214:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebb40 going to sleep for 11 seconds 00000004:00010000:1.0:1788619929.649646:0:5845:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619928 -> 1788619929 00000100:00100000:1.0:1788619929.649682:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df965f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589235072:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619929.649708:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1002001 00010000:00010000:1.0:1788619929.649736:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe53687365af lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6b7 expref: 21 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.649757:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619929.649760:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619929.649766:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe53687365af lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6b7 expref: 21 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.649785:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c400, rc 0) 00000100:00100000:1.0:1788619929.649845:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2355:x1875503571671296:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3866us (7427us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619929.649854:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1460 00000100:00100000:2.0:1788619929.653665:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619929.653947:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571671424 00000100:00100000:0.0:1788619929.653957:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571671424 00000100:00100000:0.0:1788619929.654019:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1461 00000100:00100000:0.0:1788619929.654030:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e6479c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2354:x1875503571671424:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619929.654050:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619929.654071:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4200/0x1cd7fe53687365b6 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d6be expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.654111:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5400/0x1cd7fe53687365a8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6b0 expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619929.654157:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b17480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589235200:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619929.654248:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619929.667886:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b17480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589235200:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619929.667922:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1002001 00010000:00010000:2.0:1788619929.667941:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4200/0x1cd7fe53687365b6 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6be expref: 21 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.667967:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619929.667971:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619929.667976:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4200/0x1cd7fe53687365b6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6be expref: 21 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.667993:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d4200, rc 0) 00000100:00100000:2.0:1788619929.668069:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e6479c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2354:x1875503571671424:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 14040us (14414us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619929.668077:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1461 00000100:00100000:3.0:1788619929.679979:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619929.680270:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571671680 00000100:00100000:1.0:1788619929.680282:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571671680 00000100:00100000:1.0:1788619929.680385:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1462 00000100:00100000:1.0:1788619929.680395:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+20:14908:x1875503571671680:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:1.0:1788619929.680454:0:9417:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619929.680480:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f71c00/0x1cd7fe53687365a1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6a9 expref: 20 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.680498:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619929.680503:0:9417:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788619929.680723:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687365bd lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.680738:0:9417:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70000/0x1cd7fe5368736577 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d686 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.680748:0:9417:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687365bd lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.680780:0:9417:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70000/0x1cd7fe5368736577 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d686 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.680795:0:9417:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70000/0x1cd7fe5368736577 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d686 expref: 14 pid: 5844 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619929.680811:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df965f80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589235328:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619929.680895:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebb40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619929.682917:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df965f80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589235328:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619929.682947:0:9417:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687365bd lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619929.684041:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619929.684429:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571671808 00000100:00100000:1.0:1788619929.684438:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571671808 00010000:00010000:1.0:1788619929.684480:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70000/0x1cd7fe5368736577 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d686 expref: 15 pid: 5844 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619929.684512:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 483 00000100:00100000:1.0:1788619929.684521:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f6680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+15:2355:x1875503571671808:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619929.684540:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619929.684548:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70000/0x1cd7fe5368736577 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d686 expref: 15 pid: 5844 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.684570:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70000/0x1cd7fe5368736577 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d686 expref: 15 pid: 5844 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.684593:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f70000/0x1cd7fe5368736577 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d686 expref: 15 pid: 5844 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.684609:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619929.684615:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687365bd lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.684627:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687365bd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.684650:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687365bd lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.684690:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619929.685060:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f6680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+14:2355:x1875503571671808:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 538us (1024us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619929.685067:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 483 00010000:00010000:2.0:1788619929.685284:0:9417:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687365bd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.685304:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687365bd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788619929.685476:0:9417:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967809 00000004:00010000:2.0:1788619929.685539:0:9417:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1002001 -> 1002001 00000004:00010000:2.0:1788619929.685542:0:9417:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619929 -> 1788619929 00000004:00010000:2.0:1788619929.685544:0:9417:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619929 -> 1788619928 00000004:00010000:2.0:1788619929.685546:0:9417:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619929 -> 1788619929 00000004:00010000:2.0:1788619929.685549:0:9417:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1158 -> 1158 00000004:00010000:2.0:1788619929.685556:0:9417:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000005512d6c8 reply state 000000006a121e36 transno 4294967810 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687365bd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619929.685575:0:9417:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788619929.685664:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+22:14908:x1875503571671680:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 5269us (5695us total) trans 4294967810 rc 0/0 00000100:00100000:2.0:1788619929.685671:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1462 00000020:00080000:2.0:1788619929.685674:0:9417:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788619929.693130:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687365bd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.693147:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687365bd lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.693155:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687365bd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.693161:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c683c000 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788619929.693166:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687365bd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.693174:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687365bd lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.693194:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619929.693197:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788619929.693204:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687365bd lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.693210:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687365bd lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619929.693635:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619929.694558:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571671936 00000100:00100000:1.0:1788619929.694580:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571671936 00000100:00100000:1.0:1788619929.694640:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1463 00000100:00100000:1.0:1788619929.694650:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+21:14908:x1875503571671936:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619929.694673:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619929.694691:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687365c4 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d6c5 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.694732:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5400/0x1cd7fe53687365a8 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6b0 expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.694742:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687365c4 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6c5 expref: 22 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.694773:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5400/0x1cd7fe53687365a8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6b0 expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.694786:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5400/0x1cd7fe53687365a8 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6b0 expref: 13 pid: 9417 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619929.694806:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df966680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589235456:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619929.694880:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb540 going to sleep for 11 seconds 00000100:00100000:1.0:1788619929.699677:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df966680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589235456:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619929.699726:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687365c4 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6c5 expref: 22 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.699745:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619929.699748:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619929.699753:0:5845:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687365c4 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6c5 expref: 22 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.699764:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d600, rc 0) 00000100:00100000:1.0:1788619929.699819:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+22:14908:x1875503571671936:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5169us (6187us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619929.699828:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1463 00000100:00100000:3.0:1788619929.700962:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619929.701109:0:3316:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1788619929.701127:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b15180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589235584:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1788619929.701170:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619929.701293:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571672064 00000100:00100000:1.0:1788619929.701300:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571672064 00010000:00010000:1.0:1788619929.701354:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5400/0x1cd7fe53687365a8 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6b0 expref: 14 pid: 9417 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619929.701520:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 484 00000100:00100000:1.0:1788619929.701530:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f4000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+14:2354:x1875503571672064:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619929.701543:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619929.701577:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5400/0x1cd7fe53687365a8 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6b0 expref: 14 pid: 9417 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.701591:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5400/0x1cd7fe53687365a8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6b0 expref: 14 pid: 9417 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.701612:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5400/0x1cd7fe53687365a8 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6b0 expref: 14 pid: 9417 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.701625:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619929.701698:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687365c4 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6c5 expref: 21 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.701709:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687365c4 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6c5 expref: 21 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.702156:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687365c4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6c5 expref: 21 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.702192:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687365c4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6c5 expref: 21 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619929.702202:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df966680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589235712:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619929.702902:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb540 going to sleep for 11 seconds 00000100:00100000:1.0:1788619929.702926:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589235584 00000100:00100000:1.0:1788619929.702933:0:6699:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589235584 00000100:00100000:1.0:1788619929.702970:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 92 00000100:00100000:1.0:1788619929.702978:0:6699:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b15f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503589235584:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1788619929.703008:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3856662528 avail 3854565376 00000100:00100000:1.0:1788619929.703070:0:6699:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b15f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503589235584:12345-0@lo:13:osp-pre-1-0.0 Request processed in 93us (1899us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619929.703076:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 92 00000100:00100000:1.0:1788619929.703119:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b15180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589235584:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1788619929.714884:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df966680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589235712:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619929.714912:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619929.714966:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f4000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571672064:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 13437us (14009us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619929.714977:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 484 00000100:00100000:2.0:1788619929.719321:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619929.719485:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571672192 00000100:00100000:3.0:1788619929.719494:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571672192 00002000:00010000:3.0:1788619929.719530:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1222383->18446744073709551615) 00002000:00100000:3.0:1788619929.719534:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78e6479180 with x1875503571672192 ext(1222383->18446744073709551615) 00002000:00010000:3.0:1788619929.719545:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78e6479180. 00000100:00100000:3.0:1788619929.719585:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 686 00000100:00100000:3.0:1788619929.719595:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e6479180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571672192:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:3.0:1788619929.719927:0:6703:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1571283 -> 1222383 00002000:00010000:3.0:1788619929.719931:0:6703:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619929 -> 1788619929 00002000:00010000:3.0:1788619929.719933:0:6703:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619929 -> 1788619928 00002000:00010000:3.0:1788619929.719934:0:6703:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619929 -> 1788619929 00002000:00010000:3.0:1788619929.719936:0:6703:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 391 -> 135 00000100:00100000:3.0:1788619929.719986:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e6479180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571672192:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 392us (675us total) trans 4294967500 rc 0/0 00000100:00100000:3.0:1788619929.719993:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 686 00002000:00010000:3.0:1788619929.719997:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1222383->18446744073709551615) 00002000:00100000:3.0:1788619929.719999:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78e6479180 with x1875503571672192 ext(1222383->18446744073709551615) 00002000:00010000:3.0:1788619929.720004:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78e6479180. 00000100:00100000:3.0:1788619929.722864:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619929.722925:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571672320 00000100:00100000:2.0:1788619929.722935:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571672320 00000100:00100000:2.0:1788619929.722994:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1464 00000100:00100000:2.0:1788619929.723005:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+22:14908:x1875503571672320:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619929.723024:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619929.723044:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73800/0x1cd7fe53687365cb lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d6cc expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.723125:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73200/0x1cd7fe53687365d2 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.723140:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73200/0x1cd7fe53687365d2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.723151:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73200/0x1cd7fe53687365d2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788619929.723178:0:5846:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1222383 00000004:00200000:2.0:1788619929.723182:0:5846:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788619929.723198:0:5846:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73200/0x1cd7fe53687365d2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.723224:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73800/0x1cd7fe53687365cb lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6cc expref: 24 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.723242:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73200/0x1cd7fe53687365d2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6cc expref: 23 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.723250:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619929.723253:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788619929.723257:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f73200, rc 0) 00000100:00100000:2.0:1788619929.723315:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+23:14908:x1875503571672320:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 312us (458us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619929.723322:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1464 00000100:00100000:0.0:1788619929.726047:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619929.726127:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571672448 00000100:00100000:2.0:1788619929.726137:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571672448 00002000:00100000:2.0:1788619929.726190:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b15c00 x1875503571672448/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:25/0 lens 488/0 e 0 to 0 dl 1788619940 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619929.726201:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b15c00 with x1875503571672448 00000100:00100000:2.0:1788619929.726238:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 894 00000100:00100000:2.0:1788619929.726248:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b15c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+23:2355:x1875503571672448:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619929.726478:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b15c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+23:2355:x1875503571672448:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 230us (442us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619929.726486:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 894 00002000:00100000:2.0:1788619929.726494:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b15c00 x1875503571672448/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:25/0 lens 488/440 e 0 to 0 dl 1788619940 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619929.726501:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b15c00 with x1875503571672448 00000100:00100000:3.0:1788619929.728592:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619929.728648:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571672576 00000100:00100000:0.0:1788619929.728658:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571672576 00002000:00100000:0.0:1788619929.728702:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c74f5c00 x1875503571672576/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:25/0 lens 488/0 e 0 to 0 dl 1788619940 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619929.728721:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c74f5c00 with x1875503571672576 00000100:00100000:0.0:1788619929.728754:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 895 00000100:00100000:0.0:1788619929.728764:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+23:2352:x1875503571672576:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619929.728961:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+23:2352:x1875503571672576:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 198us (376us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619929.728970:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 895 00002000:00100000:0.0:1788619929.728977:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c74f5c00 x1875503571672576/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:25/0 lens 488/440 e 0 to 0 dl 1788619940 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619929.728984:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c74f5c00 with x1875503571672576 00000100:00100000:3.0:1788619929.732880:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619929.732952:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571672704 00000100:00100000:2.0:1788619929.732963:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571672704 00000100:00100000:2.0:1788619929.733030:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1465 00000100:00100000:2.0:1788619929.733041:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+13:14908:x1875503571672704:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619929.733065:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619929.733087:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73800/0x1cd7fe53687365d9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d6d3 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.733180:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73e00/0x1cd7fe53687365e0 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.733195:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73e00/0x1cd7fe53687365e0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.733207:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73e00/0x1cd7fe53687365e0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788619929.733236:0:9417:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1222383 00000004:00200000:2.0:1788619929.733240:0:9417:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788619929.733260:0:9417:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73e00/0x1cd7fe53687365e0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.733289:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73800/0x1cd7fe53687365d9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6d3 expref: 15 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.733323:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73e00/0x1cd7fe53687365e0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6d3 expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.733380:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619929.733383:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788619929.733386:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f73e00, rc 0) 00000100:00100000:2.0:1788619929.733448:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+14:14908:x1875503571672704:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 408us (575us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619929.733454:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1465 00000100:00100000:3.0:1788619929.736122:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619929.736150:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619929.736214:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571672960 00000100:00100000:1.0:1788619929.736215:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571672832 00000100:00100000:1.0:1788619929.736224:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571672832 00000100:00100000:0.0:1788619929.736224:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571672960 00000100:00100000:0.0:1788619929.736270:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1198 00000100:00100000:1.0:1788619929.736272:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1466 00000100:00100000:1.0:1788619929.736279:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+14:2355:x1875503571672832:12345-192.168.201.5@tcp:101:fsx.0 00000100:00100000:0.0:1788619929.736279:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571672960:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619929.736296:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619929.736303:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619929.736312:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000010a73d3/0x1cd7fe53687365ee lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d6e1 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.736313:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687365e7 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d6da expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.736367:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687365c4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6c5 expref: 22 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619929.736389:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df966680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589235968:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619929.736389:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b14380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589235840:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619929.736686:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb540 going to sleep for 11 seconds 00000100:00100000:0.0:1788619929.736686:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa0c0 going to sleep for 11 seconds 00000004:00010000:0.0:1788619929.739264:0:5845:0:(mdt_lvb.c:177:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size: 1002001 -> 1039390 00000100:00100000:2.0:1788619929.739291:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b14380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589235840:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619929.739346:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000010a73d3/0x1cd7fe53687365ee lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6e1 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619929.739367:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df966680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589235968:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619929.739372:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000010a73d3/0x1cd7fe53687365ee lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6e1 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788619929.739381:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1039390 00010000:00010000:0.0:1788619929.739393:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687365e7 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6da expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.739394:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d4000, rc 0) 00010000:00010000:0.0:1788619929.739406:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619929.739409:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619929.739413:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687365e7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6da expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.739422:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d800, rc 0) 00000100:00100000:2.0:1788619929.741963:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571672960:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5676us (5801us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619929.741963:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+14:2355:x1875503571672832:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5671us (5836us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619929.741975:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1198 00000100:00100000:0.0:1788619929.741975:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1466 00000100:00100000:3.0:1788619929.744309:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619929.744623:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619929.744957:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571673216 00000100:00100000:0.0:1788619929.744968:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571673216 00000100:00100000:0.0:1788619929.745066:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1199 00000100:00100000:0.0:1788619929.745078:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571673216:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619929.745097:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619929.745115:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff481b1/0x1cd7fe53687365f5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d6ef expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619929.745175:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b17b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589236096:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619929.745213:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571673088 00000100:00100000:2.0:1788619929.745218:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571673088 00000100:00100000:2.0:1788619929.745251:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1467 00000100:00100000:0.0:1788619929.745257:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:2.0:1788619929.745258:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+14:2352:x1875503571673088:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619929.745268:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619929.745281:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72800/0x1cd7fe53687365fc lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d6e8 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.745310:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687365c4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6c5 expref: 22 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619929.746678:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df387b80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589236224:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619929.746767:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81649c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619929.747522:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b17b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589236096:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619929.747558:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000dff481b1/0x1cd7fe53687365f5 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6ef expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619929.748265:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df387b80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589236224:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619929.748280:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1039390 00010000:00010000:2.0:1788619929.748292:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72800/0x1cd7fe53687365fc lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6e8 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.748311:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619929.748314:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619929.748320:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72800/0x1cd7fe53687365fc lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6e8 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.748363:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f72800, rc 0) 00000100:00100000:2.0:1788619929.748410:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+14:2352:x1875503571673088:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3153us (4120us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619929.748416:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1467 00010000:00010000:0.0:1788619929.748892:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff481b1/0x1cd7fe53687365f5 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6ef expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.748991:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d5e00, rc 0) 00000100:00100000:0.0:1788619929.749036:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571673216:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3961us (4413us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619929.749044:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1199 00000100:00100000:1.0:1788619929.752027:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619929.752070:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619929.752206:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571673344 00000100:00100000:2.0:1788619929.752215:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571673344 00000100:00100000:2.0:1788619929.752272:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1468 00000100:00100000:2.0:1788619929.752282:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df966d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+14:2352:x1875503571673344:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619929.752302:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619929.752349:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73800/0x1cd7fe5368736603 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d6f6 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.752387:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687365c4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6c5 expref: 22 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619929.752428:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df387b80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589236352:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619929.752515:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81649c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619929.752599:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571673472 00000100:00100000:3.0:1788619929.752603:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571673472 00000100:00100000:3.0:1788619929.752633:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1200 00000100:00100000:3.0:1788619929.752639:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df967b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571673472:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619929.752648:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619929.752659:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d234829/0x1cd7fe536873660a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d6fd expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619929.752696:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589236480:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619929.752735:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c000 going to sleep for 11 seconds 00000100:00100000:2.0:1788619929.756190:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df387b80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589236352:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619929.756216:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1039390 00010000:00010000:2.0:1788619929.756236:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73800/0x1cd7fe5368736603 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6f6 expref: 15 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.756257:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619929.756260:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619929.756266:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73800/0x1cd7fe5368736603 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6f6 expref: 15 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.756282:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f73800, rc 0) 00000100:00100000:2.0:1788619929.756391:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df966d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+14:2352:x1875503571673344:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4103us (4368us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619929.756400:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1468 00000100:00100000:3.0:1788619929.759036:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589236480:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619929.759076:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009d234829/0x1cd7fe536873660a lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6fd expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.759098:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d234829/0x1cd7fe536873660a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6fd expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.759110:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f70800, rc 0) 00000100:00100000:3.0:1788619929.759212:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df967b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571673472:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6572us (7139us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619929.759221:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1200 00000100:00100000:1.0:1788619929.764951:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619929.764996:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619929.765175:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571673600 00000100:00100000:0.0:1788619929.765185:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571673600 00000100:00100000:3.0:1788619929.765304:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571673728 00000100:00100000:3.0:1788619929.765309:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571673728 00000100:00100000:3.0:1788619929.765526:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1201 00000100:00100000:3.0:1788619929.765538:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df965c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571673728:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619929.765571:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619929.765592:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fe91493a/0x1cd7fe5368736611 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d70b expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619929.765671:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589236608:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619929.765783:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c000 going to sleep for 11 seconds 00000100:00100000:2.0:1788619929.768746:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589236608:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619929.768786:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000fe91493a/0x1cd7fe5368736611 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d70b expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.768811:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fe91493a/0x1cd7fe5368736611 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d70b expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.768835:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f71800, rc 0) 00000100:00100000:2.0:1788619929.768891:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df965c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571673728:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3356us (3893us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619929.768900:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1201 00000100:00100000:0.0:1788619929.770191:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1469 00000100:00100000:0.0:1788619929.770213:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df967100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+14:2355:x1875503571673600:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619929.770237:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619929.770259:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5200/0x1cd7fe5368736618 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d704 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.770298:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687365c4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6c5 expref: 22 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619929.770363:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b17b80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589236736:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619929.770454:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:0.0:1788619929.773793:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b17b80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589236736:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619929.773822:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1039390 00010000:00010000:0.0:1788619929.773852:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5200/0x1cd7fe5368736618 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d704 expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.773874:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619929.773877:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619929.773884:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5200/0x1cd7fe5368736618 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d704 expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.773900:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d5200, rc 0) 00000100:00100000:0.0:1788619929.774207:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df967100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+14:2355:x1875503571673600:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4003us (9273us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619929.774215:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1469 00000100:00100000:0.0:1788619929.785117:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619929.785263:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571673856 00000100:00100000:2.0:1788619929.785274:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571673856 00002000:00100000:2.0:1788619929.785456:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b17b80 x1875503571673856/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:25/0 lens 504/0 e 0 to 0 dl 1788619940 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619929.785475:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b17b80 with x1875503571673856 00000100:00100000:2.0:1788619929.785570:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 896 00000100:00100000:2.0:1788619929.785590:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b17b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+23:2352:x1875503571673856:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619929.800710:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b17b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+23:2352:x1875503571673856:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 15111us (15584us total) trans 0 rc 1003520/1003520 00000100:00100000:2.0:1788619929.800725:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 896 00002000:00100000:2.0:1788619929.800740:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b17b80 x1875503571673856/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:25/0 lens 504/440 e 0 to 0 dl 1788619940 ref 1 fl Complete:/600/0 rc 1003520/1003520 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619929.800750:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b17b80 with x1875503571673856 00000100:00100000:3.0:1788619929.804585:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619929.804633:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619929.804753:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571674112 00000100:00100000:0.0:1788619929.804763:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571674112 00000100:00100000:0.0:1788619929.804821:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1202 00000100:00100000:0.0:1788619929.804841:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571674112:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619929.804858:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619929.804877:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000df72df5a/0x1cd7fe536873661f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d719 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619929.804959:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b17800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589236864:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619929.805048:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:2.0:1788619929.805658:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571673984 00000100:00100000:2.0:1788619929.805664:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571673984 00000100:00100000:2.0:1788619929.805709:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1470 00000100:00100000:2.0:1788619929.805717:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+14:2352:x1875503571673984:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619929.805728:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619929.805740:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73800/0x1cd7fe5368736626 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d712 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619929.805767:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687365c4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6c5 expref: 22 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619929.805791:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df387b80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589236992:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619929.805908:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81649c0 going to sleep for 11 seconds 00002000:00010000:0.0:1788619929.810882:0:6698:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619928 -> 1788619929 00000100:00100000:0.0:1788619929.810917:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b17800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589236864:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619929.810980:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000df72df5a/0x1cd7fe536873661f lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d719 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.811010:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000df72df5a/0x1cd7fe536873661f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d719 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.811024:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d5200, rc 0) 00000100:00100000:0.0:1788619929.811105:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571674112:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6275us (6470us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619929.811114:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1202 00000004:00010000:3.0:1788619929.812873:0:5846:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619928 -> 1788619929 00000100:00100000:3.0:1788619929.812901:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df387b80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589236992:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619929.812923:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1039390 00010000:00010000:3.0:1788619929.813059:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73800/0x1cd7fe5368736626 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d712 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.813079:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619929.813083:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619929.813088:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73800/0x1cd7fe5368736626 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d712 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.813101:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f73800, rc 0) 00000100:00100000:3.0:1788619929.813159:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+14:2352:x1875503571673984:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7443us (8585us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619929.813167:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1470 00000100:00100000:2.0:1788619929.817794:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619929.817853:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619929.818098:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571674240 00000100:00100000:0.0:1788619929.818109:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571674240 00000100:00100000:0.0:1788619929.818169:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1471 00000100:00100000:0.0:1788619929.818179:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df387b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+14:2352:x1875503571674240:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619929.818201:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619929.818222:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5200/0x1cd7fe536873662d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d720 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.818263:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687365c4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6c5 expref: 22 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619929.818311:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b17800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589237120:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619929.819055:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:1.0:1788619929.819668:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571674368 00000100:00100000:1.0:1788619929.819677:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571674368 00000100:00100000:1.0:1788619929.819739:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1203 00000100:00100000:1.0:1788619929.819748:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571674368:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619929.819762:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619929.819777:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe5368736634 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d727 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619929.819824:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df964380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589237248:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619929.819892:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebc00 going to sleep for 11 seconds 00000100:00100000:1.0:1788619929.824418:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df964380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589237248:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619929.824467:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe5368736634 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d727 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.824494:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe5368736634 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d727 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.824509:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683dc00, rc 0) 00000100:00100000:1.0:1788619929.824575:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571674368:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4828us (6720us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619929.824583:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1203 00000100:00100000:0.0:1788619929.825594:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b17800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589237120:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619929.825614:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1039390 00010000:00010000:0.0:1788619929.825630:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5200/0x1cd7fe536873662d lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d720 expref: 15 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.825646:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619929.825649:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619929.825654:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5200/0x1cd7fe536873662d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d720 expref: 15 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.825665:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d5200, rc 0) 00000100:00100000:0.0:1788619929.825783:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df387b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+14:2352:x1875503571674240:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7605us (8000us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619929.825791:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1471 00000100:00100000:2.0:1788619929.834766:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619929.834827:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619929.835026:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571674624 00000100:00100000:3.0:1788619929.835037:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571674624 00000100:00100000:3.0:1788619929.835103:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1204 00000100:00100000:3.0:1788619929.835114:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571674624:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619929.835132:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619929.835149:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e4266ce/0x1cd7fe536873663b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d735 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619929.835201:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589237376:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619929.835282:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48ccc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619929.836100:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571674496 00000100:00100000:0.0:1788619929.836106:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571674496 00000100:00100000:0.0:1788619929.836143:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1472 00000100:00100000:0.0:1788619929.836151:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8595880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+14:2352:x1875503571674496:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619929.836164:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619929.836180:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5200/0x1cd7fe5368736642 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d72e expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.836212:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687365c4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6c5 expref: 22 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619929.836244:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b17800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589237504:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619929.836305:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:1.0:1788619929.840066:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b17800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589237504:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619929.840096:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1039390 00010000:00010000:1.0:1788619929.840116:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5200/0x1cd7fe5368736642 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d72e expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.840143:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619929.840146:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619929.840152:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5200/0x1cd7fe5368736642 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d72e expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.840168:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d5200, rc 0) 00000100:00100000:1.0:1788619929.840227:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8595880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+14:2352:x1875503571674496:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4076us (5469us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619929.840236:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1472 00002000:00010000:0.0:1788619929.840556:0:6696:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size: 1222383 -> 2006628 00000100:00100000:0.0:1788619929.840584:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589237376:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619929.840607:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003e4266ce/0x1cd7fe536873663b lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d735 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.840624:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e4266ce/0x1cd7fe536873663b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d735 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.840633:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d6200, rc 0) 00000100:00100000:0.0:1788619929.840659:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571674624:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5548us (5832us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619929.840666:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1204 00000100:00100000:2.0:1788619929.845180:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619929.845211:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619929.845228:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571674752 00000100:00100000:3.0:1788619929.845238:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571674752 00000100:00100000:3.0:1788619929.845348:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1473 00000100:00100000:3.0:1788619929.845358:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a58000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+14:2355:x1875503571674752:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619929.845379:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619929.845408:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7c00/0x1cd7fe5368736649 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d73c expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.845447:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687365c4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6c5 expref: 22 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619929.845486:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f1c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589237632:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619929.845579:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48ccc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619929.847792:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f1c00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589237632:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619929.847823:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1039390 00010000:00010000:3.0:1788619929.847856:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7c00/0x1cd7fe5368736649 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d73c expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.847878:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619929.847881:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619929.847887:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7c00/0x1cd7fe5368736649 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d73c expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.847911:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d7c00, rc 0) 00000100:00100000:3.0:1788619929.847965:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a58000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+14:2355:x1875503571674752:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2607us (2791us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619929.847973:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1473 00000100:00100000:0.0:1788619929.849628:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571674880 00000100:00100000:0.0:1788619929.849644:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571674880 00000100:00100000:0.0:1788619929.849707:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1205 00000100:00100000:0.0:1788619929.850049:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a59180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571674880:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619929.850066:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619929.850080:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c161f442/0x1cd7fe5368736650 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d743 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619929.850130:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b17800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589237760:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619929.850204:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:0.0:1788619929.853590:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b17800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589237760:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619929.853636:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c161f442/0x1cd7fe5368736650 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d743 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.853663:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c161f442/0x1cd7fe5368736650 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d743 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.853675:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d4400, rc 0) 00000100:00100000:0.0:1788619929.853930:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a59180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571674880:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4214us (8716us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619929.853940:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1205 00000100:00100000:2.0:1788619929.857670:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619929.857754:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571675136 00000100:00100000:0.0:1788619929.857763:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571675136 00000100:00100000:0.0:1788619929.857826:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1474 00000100:00100000:0.0:1788619929.857848:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+14:14908:x1875503571675136:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:0.0:1788619929.857901:0:9417:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619929.857944:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73e00/0x1cd7fe53687365e0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6d3 expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.857963:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619929.857969:0:9417:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788619929.858180:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4e00/0x1cd7fe5368736657 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.858195:0:9417:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73200/0x1cd7fe53687365d2 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6cc expref: 22 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.858204:0:9417:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4e00/0x1cd7fe5368736657 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.858235:0:9417:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73200/0x1cd7fe53687365d2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6cc expref: 22 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.858249:0:9417:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73200/0x1cd7fe53687365d2 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6cc expref: 22 pid: 5846 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619929.858263:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b17800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589237888:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619929.858373:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:2.0:1788619929.861728:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619929.861816:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b17800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589237888:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619929.861854:0:9417:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4e00/0x1cd7fe5368736657 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619929.861921:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571675264 00000100:00100000:3.0:1788619929.861928:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571675264 00010000:00010000:3.0:1788619929.861964:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73200/0x1cd7fe53687365d2 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6cc expref: 23 pid: 5846 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619929.862005:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 485 00000100:00100000:3.0:1788619929.862014:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a59880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+23:2355:x1875503571675264:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619929.862030:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619929.862037:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73200/0x1cd7fe53687365d2 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6cc expref: 23 pid: 5846 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.862050:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73200/0x1cd7fe53687365d2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6cc expref: 23 pid: 5846 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.862071:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f73200/0x1cd7fe53687365d2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6cc expref: 23 pid: 5846 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.862087:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619929.862092:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4e00/0x1cd7fe5368736657 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.862102:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4e00/0x1cd7fe5368736657 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.862109:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4e00/0x1cd7fe5368736657 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.862137:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619929.862194:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a59880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2355:x1875503571675264:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 180us (477us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619929.862201:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 485 00010000:00010000:0.0:1788619929.862254:0:9417:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4e00/0x1cd7fe5368736657 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.862262:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4e00/0x1cd7fe5368736657 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788619929.862425:0:9417:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967811 00000004:00010000:0.0:1788619929.862475:0:9417:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1039390 -> 1002001 00000004:00010000:0.0:1788619929.862479:0:9417:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619929 -> 1788619929 00000004:00010000:0.0:1788619929.862481:0:9417:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619929 -> 1788619928 00000004:00010000:0.0:1788619929.862484:0:9417:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619929 -> 1788619929 00000004:00010000:0.0:1788619929.862486:0:9417:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1158 -> 1158 00000004:00010000:0.0:1788619929.862495:0:9417:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000c352e0c8 reply state 0000000008e28c38 transno 4294967812 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4e00/0x1cd7fe5368736657 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619929.862503:0:9417:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788619929.862583:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+16:14908:x1875503571675136:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 4736us (4922us total) trans 4294967812 rc 0/0 00000100:00100000:0.0:1788619929.862590:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1474 00000020:00080000:0.0:1788619929.862593:0:9417:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788619929.864739:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4e00/0x1cd7fe5368736657 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.864754:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4e00/0x1cd7fe5368736657 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.864762:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4e00/0x1cd7fe5368736657 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.864768:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c64d4e00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788619929.864773:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4e00/0x1cd7fe5368736657 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.864781:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4e00/0x1cd7fe5368736657 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.864797:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619929.864800:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788619929.864805:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4e00/0x1cd7fe5368736657 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.864810:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4e00/0x1cd7fe5368736657 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619929.866462:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619929.866605:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571675392 00000100:00100000:1.0:1788619929.866616:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571675392 00000100:00100000:1.0:1788619929.866671:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1475 00000100:00100000:1.0:1788619929.866681:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+15:14908:x1875503571675392:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619929.866716:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619929.866736:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe536873665e lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d74a expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.866762:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687365c4 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6c5 expref: 21 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.866772:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe536873665e lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d74a expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.866802:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687365c4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6c5 expref: 21 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.866819:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687365c4 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6c5 expref: 21 pid: 5845 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619929.866838:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df964380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589238016:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619929.866923:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeba80 going to sleep for 11 seconds 00000100:00100000:1.0:1788619929.871115:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df964380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589238016:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619929.871151:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe536873665e lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d74a expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.871170:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619929.871173:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619929.871177:0:5845:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe536873665e lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d74a expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.871187:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683dc00, rc 0) 00000100:00100000:1.0:1788619929.871245:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+16:14908:x1875503571675392:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4565us (4792us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619929.871253:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1475 00000100:00100000:2.0:1788619929.873196:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619929.873279:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571675520 00000100:00100000:0.0:1788619929.873288:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571675520 00002000:00100000:0.0:1788619929.873369:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6a5a300 x1875503571675520/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:25/0 lens 488/0 e 0 to 0 dl 1788619940 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619929.873380:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6a5a300 with x1875503571675520 00010000:00010000:0.0:1788619929.873398:0:5858:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687365c4 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6c5 expref: 22 pid: 5845 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.873412:0:5858:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687365c4 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6c5 expref: 22 pid: 5845 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788619929.873420:0:5858:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e77c6a5a300 00000100:00100000:0.0:1788619929.873443:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 38 00000100:00100000:0.0:1788619929.873452:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2355:x1875503571675520:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:0.0:1788619929.877481:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1002001 -> 1039390 00000004:00010000:0.0:1788619929.877491:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619929 -> 1788619929 00000004:00010000:0.0:1788619929.877494:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619928 -> 1788619928 00000004:00010000:0.0:1788619929.877496:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619929 -> 1788619929 00000004:00010000:0.0:1788619929.877498:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1158 -> 1158 00000100:00100000:0.0:1788619929.878046:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2355:x1875503571675520:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 4588us (4853us total) trans 4294967813 rc 0/0 00000100:00100000:0.0:1788619929.878058:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 38 00002000:00100000:0.0:1788619929.878069:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6a5a300 x1875503571675520/t4294967813(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:25/0 lens 488/448 e 0 to 0 dl 1788619940 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619929.878085:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6a5a300 with x1875503571675520 00010000:00010000:0.0:1788619929.878095:0:5858:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687365c4 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6c5 expref: 24 pid: 5845 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.878112:0:5858:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687365c4 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6c5 expref: 24 pid: 5845 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788619929.878121:0:5858:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e77c6a5a300 00000100:00100000:2.0:1788619929.884795:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619929.885267:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571675648 00000100:00100000:0.0:1788619929.885278:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571675648 00010000:00010000:0.0:1788619929.885320:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687365c4 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6c5 expref: 24 pid: 5845 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619929.885572:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 486 00000100:00100000:0.0:1788619929.885583:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2352:x1875503571675648:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619929.885598:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619929.885625:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687365c4 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6c5 expref: 24 pid: 5845 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.885641:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687365c4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6c5 expref: 24 pid: 5845 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.885664:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687365c4 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d6c5 expref: 24 pid: 5845 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.885681:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619929.885689:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe536873665e lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d74a expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.885717:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe536873665e lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d74a expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.885726:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe536873665e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d74a expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.885763:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe536873665e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d74a expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619929.885776:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b17800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589238144:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619929.885876:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5eaf600 going to sleep for 11 seconds 00000100:00100000:2.0:1788619929.889377:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619929.889683:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571675776 00000100:00100000:3.0:1788619929.889693:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571675776 00000100:00100000:3.0:1788619929.889759:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1206 00000100:00100000:3.0:1788619929.889769:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:14908:x1875503571675776:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619929.889789:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619929.889806:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e195d52/0x1cd7fe5368736665 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d751 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.889849:0:6698:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000005c0482c0/0x1cd7fe5368736569 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d67f expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.889858:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(4 12) 00010000:00010000:3.0:1788619929.889863:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(5 12) 00010000:00010000:3.0:1788619929.889869:0:6698:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e195d52/0x1cd7fe5368736665 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d751 expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.890051:0:6698:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000005c0482c0/0x1cd7fe5368736569 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d67f expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.890068:0:6698:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000005c0482c0/0x1cd7fe5368736569 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d67f expref: 6 pid: 6698 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619929.890084:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f1c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589238272:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619929.890168:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48ccc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619929.891236:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b17800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589238144:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619929.891257:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619929.891291:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+23:2352:x1875503571675648:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 5711us (6506us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619929.894218:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f1c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589238272:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619929.894256:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009e195d52/0x1cd7fe5368736665 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d751 expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.894279:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d7c00, rc 0) 00000100:00100000:1.0:1788619929.894375:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:14908:x1875503571675776:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4606us (5008us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619929.894394:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1206 00000100:00100000:0.0:1788619929.895586:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 486 00000100:00100000:3.0:1788619929.897842:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619929.898110:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571675904 00000100:00100000:1.0:1788619929.898121:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571675904 00002000:00100000:1.0:1788619929.898170:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1982464->2006627) req@ffff8e78e67f0e00 x1875503571675904/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:25/0 lens 488/0 e 0 to 0 dl 1788619940 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619929.898181:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78e67f0e00 with x1875503571675904 ext(1982464->2006627) 00010000:00010000:1.0:1788619929.898190:0:6701:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000005c0482c0/0x1cd7fe5368736569 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d67f expref: 7 pid: 6698 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.898207:0:6701:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000005c0482c0/0x1cd7fe5368736569 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d67f expref: 7 pid: 6698 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788619929.898214:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff8e78e67f0e00 00000100:00100000:1.0:1788619929.898243:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 23 00000100:00100000:1.0:1788619929.898252:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571675904:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:1.0:1788619929.902167:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2355:x1875503571675904:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 3906us (4334us total) trans 4294967501 rc 0/0 00000100:00100000:1.0:1788619929.902181:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 23 00002000:00100000:1.0:1788619929.902193:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1982464->2006627) req@ffff8e78e67f0e00 x1875503571675904/t4294967501(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:25/0 lens 488/448 e 0 to 0 dl 1788619940 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619929.902204:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78e67f0e00 with x1875503571675904 ext(1982464->2006627) 00010000:00010000:1.0:1788619929.902215:0:6701:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000005c0482c0/0x1cd7fe5368736569 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d67f expref: 9 pid: 6698 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.902242:0:6701:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000005c0482c0/0x1cd7fe5368736569 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d67f expref: 9 pid: 6698 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788619929.902250:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff8e78e67f0e00 00000100:00100000:2.0:1788619929.906473:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619929.906570:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571676032 00000100:00100000:3.0:1788619929.906581:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571676032 00010000:00010000:3.0:1788619929.906622:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000005c0482c0/0x1cd7fe5368736569 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d67f expref: 9 pid: 6698 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619929.906672:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 487 00000100:00100000:3.0:1788619929.906682:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcac450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2353:x1875503571676032:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619929.906698:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619929.906734:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000005c0482c0/0x1cd7fe5368736569 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d67f expref: 9 pid: 6698 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.906749:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000005c0482c0/0x1cd7fe5368736569 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d67f expref: 9 pid: 6698 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.906779:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c0482c0/0x1cd7fe5368736569 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d67f expref: 9 pid: 6698 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.906798:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(5 12) 00010000:00010000:3.0:1788619929.906803:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(5 12) 00010000:00010000:3.0:1788619929.906813:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000009e195d52/0x1cd7fe5368736665 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d751 expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.906824:0:13330:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e195d52/0x1cd7fe5368736665 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d751 expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.906878:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000009e195d52/0x1cd7fe5368736665 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d751 expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619929.906894:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f3b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589238400:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619929.906983:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c000 going to sleep for 11 seconds 00000100:00100000:0.0:1788619929.910482:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f3b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589238400:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619929.910511:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619929.910576:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcac450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571676032:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3894us (4112us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619929.910586:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 487 00000100:00100000:2.0:1788619929.912212:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619929.912270:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571676160 00000100:00100000:3.0:1788619929.912278:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571676160 00002000:00010000:3.0:1788619929.912314:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1998533->18446744073709551615) 00002000:00100000:3.0:1788619929.912318:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a59880 with x1875503571676160 ext(1998533->18446744073709551615) 00002000:00010000:3.0:1788619929.912363:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a59880. 00000100:00100000:3.0:1788619929.912394:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 687 00000100:00100000:3.0:1788619929.912409:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a59880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571676160:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:3.0:1788619929.912679:0:6703:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 2006628 -> 1998533 00002000:00010000:3.0:1788619929.912683:0:6703:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619929 -> 1788619929 00002000:00010000:3.0:1788619929.912686:0:6703:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619929 -> 1788619928 00002000:00010000:3.0:1788619929.912688:0:6703:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619929 -> 1788619929 00002000:00010000:3.0:1788619929.912691:0:6703:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 135 -> 135 00000100:00100000:3.0:1788619929.912747:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a59880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571676160:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 345us (541us total) trans 4294967502 rc 0/0 00000100:00100000:3.0:1788619929.912753:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 687 00002000:00010000:3.0:1788619929.912757:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1998533->18446744073709551615) 00002000:00100000:3.0:1788619929.912760:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a59880 with x1875503571676160 ext(1998533->18446744073709551615) 00002000:00010000:3.0:1788619929.912764:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a59880. 00000100:00100000:2.0:1788619929.916817:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619929.917009:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571676288 00000100:00100000:3.0:1788619929.917020:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571676288 00000100:00100000:3.0:1788619929.917079:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1476 00000100:00100000:3.0:1788619929.917089:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+16:14908:x1875503571676288:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619929.917111:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619929.917132:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7400/0x1cd7fe536873666c lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d758 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.917225:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6c00/0x1cd7fe5368736673 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.917243:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6c00/0x1cd7fe5368736673 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.917255:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6c00/0x1cd7fe5368736673 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788619929.917287:0:5846:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1998533 00000004:00200000:3.0:1788619929.917291:0:5846:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788619929.917311:0:5846:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6c00/0x1cd7fe5368736673 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.917380:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7400/0x1cd7fe536873666c lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d758 expref: 18 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.917399:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6c00/0x1cd7fe5368736673 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d758 expref: 17 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.917530:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619929.917533:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788619929.917537:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d6c00, rc 0) 00000100:00100000:3.0:1788619929.917609:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+17:14908:x1875503571676288:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 521us (805us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619929.917617:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1476 00000100:00100000:2.0:1788619929.922080:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619929.922173:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571676416 00000100:00100000:0.0:1788619929.922182:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571676416 00000100:00100000:0.0:1788619929.922239:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1477 00000100:00100000:0.0:1788619929.922247:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+23:14908:x1875503571676416:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619929.922272:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619929.922294:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4e00/0x1cd7fe536873667a lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d75f expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.922436:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4400/0x1cd7fe5368736681 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.922451:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4400/0x1cd7fe5368736681 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.922463:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4400/0x1cd7fe5368736681 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619929.922491:0:9417:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1998533 00000004:00200000:0.0:1788619929.922495:0:9417:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619929.922514:0:9417:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4400/0x1cd7fe5368736681 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.922543:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4e00/0x1cd7fe536873667a lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d75f expref: 25 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.922572:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4400/0x1cd7fe5368736681 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d75f expref: 24 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.922581:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619929.922583:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619929.922587:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d4400, rc 0) 00000100:00100000:0.0:1788619929.922644:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+24:14908:x1875503571676416:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 398us (573us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619929.922651:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1477 00000100:00100000:2.0:1788619929.926288:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619929.928032:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619929.928104:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571676672 00000100:00100000:0.0:1788619929.928115:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571676672 00000100:00100000:0.0:1788619929.928173:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1207 00000100:00100000:0.0:1788619929.928183:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571676672:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619929.928201:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619929.928220:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096deb5d6/0x1cd7fe5368736688 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d76d expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619929.928280:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b17800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589238528:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619929.928395:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5eafe40 going to sleep for 11 seconds 00002000:00010000:0.0:1788619929.930602:0:6697:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619928 -> 1788619929 00000100:00100000:0.0:1788619929.930634:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b17800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589238528:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619929.930669:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000096deb5d6/0x1cd7fe5368736688 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d76d expref: 9 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.930692:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096deb5d6/0x1cd7fe5368736688 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d76d expref: 9 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619929.930779:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d5c00, rc 0) 00000100:00100000:0.0:1788619929.930938:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571676672:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2645us (2801us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619929.930950:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1207 00000100:00100000:1.0:1788619929.932658:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571676544 00000100:00100000:1.0:1788619929.932670:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571676544 00000100:00100000:1.0:1788619929.932859:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1478 00000100:00100000:1.0:1788619929.932870:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2352:x1875503571676544:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619929.932891:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619929.932924:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe536873668f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d766 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.932962:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe536873665e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d74a expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619929.933006:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df964380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589238656:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619929.933095:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeba80 going to sleep for 11 seconds 00000004:00010000:1.0:1788619929.936272:0:5845:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619928 -> 1788619929 00000100:00100000:1.0:1788619929.936305:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df964380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589238656:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619929.936357:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1039390 00010000:00010000:1.0:1788619929.936379:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe536873668f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d766 expref: 25 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.936400:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619929.936414:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619929.936421:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe536873668f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d766 expref: 25 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.936436:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683cc00, rc 0) 00000100:00100000:1.0:1788619929.936493:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2352:x1875503571676544:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3625us (10214us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619929.936502:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1478 00000100:00100000:2.0:1788619929.940980:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619929.941023:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619929.941094:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571676928 00000100:00100000:3.0:1788619929.941103:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571676928 00000100:00100000:3.0:1788619929.941168:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1208 00000100:00100000:3.0:1788619929.941179:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571676928:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619929.941197:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619929.941214:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029d0975e/0x1cd7fe5368736696 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d77b expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619929.941275:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589238784:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619929.941396:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c240 going to sleep for 11 seconds 00000100:00100000:1.0:1788619929.941735:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571676800 00000100:00100000:1.0:1788619929.941740:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571676800 00000100:00100000:1.0:1788619929.941782:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1479 00000100:00100000:1.0:1788619929.941792:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2352:x1875503571676800:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619929.941802:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619929.941815:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe536873669d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d774 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.941843:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe536873665e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d74a expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619929.941869:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df964380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589238912:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619929.941914:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeba80 going to sleep for 11 seconds 00000100:00100000:3.0:1788619929.948293:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589238784:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619929.948377:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000029d0975e/0x1cd7fe5368736696 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d77b expref: 9 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.948541:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029d0975e/0x1cd7fe5368736696 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d77b expref: 9 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619929.948555:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d7400, rc 0) 00000100:00100000:2.0:1788619929.948851:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571676928:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7672us (7825us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619929.948861:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1208 00000100:00100000:1.0:1788619929.951791:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df964380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589238912:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619929.951819:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1039390 00010000:00010000:1.0:1788619929.951851:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe536873669d lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d774 expref: 25 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.951871:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619929.951875:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619929.951882:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe536873669d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d774 expref: 25 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619929.951896:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c800, rc 0) 00000100:00100000:1.0:1788619929.951964:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+24:2352:x1875503571676800:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 10174us (10993us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619929.951973:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1479 00000100:00100000:0.0:1788619929.958540:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619929.958933:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571677056 00000100:00100000:2.0:1788619929.958942:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571677056 00002000:00100000:2.0:1788619929.958996:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b17800 x1875503571677056/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:25/0 lens 488/0 e 0 to 0 dl 1788619940 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619929.959009:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b17800 with x1875503571677056 00000100:00100000:2.0:1788619929.959051:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 897 00000100:00100000:2.0:1788619929.959061:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b17800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+17:2353:x1875503571677056:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619929.959287:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b17800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+17:2353:x1875503571677056:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 226us (755us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619929.959294:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 897 00002000:00100000:2.0:1788619929.959301:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b17800 x1875503571677056/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:25/0 lens 488/4536 e 0 to 0 dl 1788619940 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619929.959309:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b17800 with x1875503571677056 00000100:00100000:2.0:1788619929.962794:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619929.962995:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571677184 00000100:00100000:0.0:1788619929.963005:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571677184 00002000:00100000:0.0:1788619929.963055:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df19ea00 x1875503571677184/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:25/0 lens 488/0 e 0 to 0 dl 1788619940 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619929.963066:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df19ea00 with x1875503571677184 00000100:00100000:0.0:1788619929.963106:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 898 00000100:00100000:0.0:1788619929.963116:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+17:2353:x1875503571677184:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619929.963365:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+17:2353:x1875503571677184:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 249us (580us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619929.963373:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 898 00002000:00100000:0.0:1788619929.963380:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df19ea00 x1875503571677184/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:25/0 lens 488/4536 e 0 to 0 dl 1788619940 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619929.963388:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df19ea00 with x1875503571677184 00000100:00100000:3.0:1788619929.967567:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619929.967680:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571677312 00000100:00100000:1.0:1788619929.967689:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571677312 00002000:00100000:1.0:1788619929.967919:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e67f1f80 x1875503571677312/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:25/0 lens 488/0 e 0 to 0 dl 1788619940 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619929.967931:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e67f1f80 with x1875503571677312 00000100:00100000:1.0:1788619929.967969:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 899 00000100:00100000:1.0:1788619929.967979:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+17:2353:x1875503571677312:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619929.968194:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+17:2353:x1875503571677312:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 216us (646us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619929.968203:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 899 00002000:00100000:1.0:1788619929.968210:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e67f1f80 x1875503571677312/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:25/0 lens 488/4536 e 0 to 0 dl 1788619940 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619929.968218:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e67f1f80 with x1875503571677312 00000100:00100000:1.0:1788619929.974798:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619929.975147:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571677440 00000100:00100000:2.0:1788619929.975158:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571677440 00002000:00100000:2.0:1788619929.975218:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df964380 x1875503571677440/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:25/0 lens 488/0 e 0 to 0 dl 1788619940 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619929.975228:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df964380 with x1875503571677440 00000100:00100000:2.0:1788619929.975273:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 900 00000100:00100000:2.0:1788619929.975284:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df964380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+17:2352:x1875503571677440:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619929.975578:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df964380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+17:2352:x1875503571677440:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 295us (791us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619929.975586:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 900 00002000:00100000:2.0:1788619929.975592:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df964380 x1875503571677440/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:25/0 lens 488/4536 e 0 to 0 dl 1788619940 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619929.975600:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df964380 with x1875503571677440 00000100:00100000:1.0:1788619929.982023:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619929.982265:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571677568 00000100:00100000:0.0:1788619929.982276:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571677568 00002000:00100000:0.0:1788619929.982364:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df965180 x1875503571677568/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:25/0 lens 16872/0 e 0 to 0 dl 1788619940 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619929.982388:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df965180 with x1875503571677568 00000100:00100000:0.0:1788619929.982427:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 901 00000100:00100000:0.0:1788619929.982437:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df965180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+17:2354:x1875503571677568:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:0.0:1788619929.983019:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1039390 -> 1039390 00000004:00010000:0.0:1788619929.983024:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619929 -> 1788619929 00000004:00010000:0.0:1788619929.983026:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619929 -> 1788619928 00000004:00010000:0.0:1788619929.983028:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619929 -> 1788619929 00000004:00010000:0.0:1788619929.983031:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1158 -> 1158 00000100:00100000:0.0:1788619929.983141:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df965180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+19:2354:x1875503571677568:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 704us (1126us total) trans 4294967814 rc 0/0 00000100:00100000:0.0:1788619929.983149:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 901 00002000:00100000:0.0:1788619929.983158:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df965180 x1875503571677568/t4294967814(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:25/0 lens 16872/448 e 0 to 0 dl 1788619940 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619929.983167:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df965180 with x1875503571677568 00000100:00100000:1.0:1788619929.987163:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619929.987376:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571677696 00000100:00100000:3.0:1788619929.987484:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571677696 00000100:00100000:3.0:1788619929.987540:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1480 00000100:00100000:3.0:1788619929.987551:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df966680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+19:2355:x1875503571677696:12345-192.168.201.5@tcp:16:fsx.0 00000001:00080000:2.0:1788619930.030022:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967814 is committed 00000001:00080000:2.0:1788619930.030044:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967813 is committed 00000100:00100000:0.0:1788619930.030265:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df966680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571677696:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 42710us (43107us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619930.030278:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1480 00000100:00100000:2.0:1788619930.033775:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619930.033817:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619930.033931:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571677824 00000100:00100000:1.0:1788619930.033950:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571677824 00000100:00080000:3.0:1788619930.034056:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1788619930.034201:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1481 00000100:00100000:1.0:1788619930.034219:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571677824:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619930.034250:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1788619930.034267:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571677952 00010000:00010000:1.0:1788619930.034281:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687366a4 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d782 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619930.034281:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571677952 00010000:00010000:1.0:1788619930.034378:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe536873665e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d74a expref: 9 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619930.034439:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df965f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589239040:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619930.034576:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeba80 going to sleep for 11 seconds 00000100:00100000:0.0:1788619930.034589:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1209 00000100:00100000:0.0:1788619930.034605:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571677952:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619930.034623:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619930.034642:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002919f738/0x1cd7fe53687366ab lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d789 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619930.036047:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b14380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589239168:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619930.036146:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5eaf300 going to sleep for 11 seconds 00000100:00080000:3.0:1788619930.036345:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000004:00010000:3.0:1788619930.038089:0:5845:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619928 -> 1788619929 00000004:00010000:3.0:1788619930.038097:0:5845:0:(mdt_lvb.c:201:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks: 1158 -> 1672 00000100:00100000:3.0:1788619930.038122:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df965f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589239040:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619930.038144:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1039390 00010000:00010000:3.0:1788619930.038162:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687366a4 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d782 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.038182:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619930.038185:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619930.038191:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687366a4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d782 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.038205:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683da00, rc 0) 00000100:00100000:3.0:1788619930.038299:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571677824:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4086us (4532us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619930.038308:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1481 00000100:00100000:0.0:1788619930.040485:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b14380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589239168:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619930.040526:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002919f738/0x1cd7fe53687366ab lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d789 expref: 9 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.040550:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002919f738/0x1cd7fe53687366ab lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d789 expref: 9 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.040578:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d4600, rc 0) 00000100:00100000:0.0:1788619930.040689:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571677952:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6090us (6870us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619930.040708:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1209 00000100:00100000:2.0:1788619930.044279:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619930.044530:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619930.044614:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571678208 00000100:00100000:3.0:1788619930.044624:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571678208 00000100:00100000:3.0:1788619930.044680:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1210 00000100:00100000:3.0:1788619930.044689:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571678208:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619930.044814:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619930.044842:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029d0975e/0x1cd7fe53687366b2 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d797 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619930.044901:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f2300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589239296:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619930.045025:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619930.045264:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571678080 00000100:00100000:1.0:1788619930.045269:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571678080 00000100:00100000:1.0:1788619930.045385:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1482 00000100:00100000:1.0:1788619930.045395:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571678080:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619930.045406:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619930.045420:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe53687366b9 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d790 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.045451:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe536873665e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d74a expref: 9 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619930.045478:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df964700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589239424:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619930.045514:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb540 going to sleep for 11 seconds 00000100:00100000:3.0:1788619930.050035:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f2300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589239296:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619930.050074:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000029d0975e/0x1cd7fe53687366b2 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d797 expref: 9 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.050100:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029d0975e/0x1cd7fe53687366b2 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d797 expref: 9 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.050114:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d7400, rc 0) 00000100:00100000:3.0:1788619930.050179:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571678208:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5490us (5647us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619930.050191:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1210 00000100:00100000:1.0:1788619930.051812:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df964700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589239424:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619930.051891:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1039390 00010000:00010000:1.0:1788619930.052020:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe53687366b9 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d790 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.052046:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619930.052050:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619930.052055:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe53687366b9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d790 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.052068:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c800, rc 0) 00000100:00100000:1.0:1788619930.052159:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571678080:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6765us (7888us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619930.052167:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1482 00000100:00100000:2.0:1788619930.055627:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619930.055717:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619930.055844:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571678336 00000100:00100000:3.0:1788619930.055853:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571678336 00000100:00100000:1.0:1788619930.056051:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1483 00000100:00100000:1.0:1788619930.056065:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed60380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571678336:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619930.056086:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619930.056108:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687366c0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d79e expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.056148:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe536873665e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d74a expref: 9 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619930.056190:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df964700 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589239552:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619930.056274:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb540 going to sleep for 11 seconds 00000100:00100000:1.0:1788619930.058641:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df964700 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589239552:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619930.058667:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1039390 00010000:00010000:1.0:1788619930.058685:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687366c0 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d79e expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.058720:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619930.058723:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619930.058728:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687366c0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d79e expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.058742:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d600, rc 0) 00000100:00100000:1.0:1788619930.058785:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed60380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571678336:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2725us (3068us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619930.058793:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1483 00000100:00100000:0.0:1788619930.062491:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571678464 00000100:00100000:0.0:1788619930.062506:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571678464 00000100:00100000:1.0:1788619930.062672:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1211 00000100:00100000:1.0:1788619930.062683:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed63800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571678464:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619930.062769:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619930.062789:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a803caae/0x1cd7fe53687366c7 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d7a5 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619930.063001:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df964700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589239680:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619930.063194:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb540 going to sleep for 11 seconds 00000100:00100000:2.0:1788619930.066316:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df964700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589239680:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619930.066602:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a803caae/0x1cd7fe53687366c7 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7a5 expref: 9 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.066628:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a803caae/0x1cd7fe53687366c7 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7a5 expref: 9 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.066641:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d000, rc 0) 00000100:00100000:2.0:1788619930.066798:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed63800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571678464:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4115us (11179us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619930.066808:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1211 00000100:00100000:3.0:1788619930.069974:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619930.070047:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619930.070162:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571678720 00000100:00100000:0.0:1788619930.070172:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571678720 00000100:00100000:0.0:1788619930.070224:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1212 00000100:00100000:0.0:1788619930.070233:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571678720:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619930.070249:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619930.070265:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d6eb124/0x1cd7fe53687366ce lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d7b3 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619930.070318:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b14380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589239808:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619930.070425:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5eaf300 going to sleep for 11 seconds 00000100:00100000:1.0:1788619930.072870:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571678592 00000100:00100000:1.0:1788619930.072885:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571678592 00000100:00100000:1.0:1788619930.073178:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1484 00000100:00100000:1.0:1788619930.073190:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571678592:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619930.073210:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619930.073231:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe53687366d5 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d7ac expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.073275:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe536873665e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d74a expref: 9 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619930.073315:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df964700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589239936:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619930.073601:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619930.075967:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df964700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589239936:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619930.075991:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1039390 00010000:00010000:1.0:1788619930.076008:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe53687366d5 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7ac expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.076028:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619930.076031:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619930.076036:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe53687366d5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7ac expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.076049:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c800, rc 0) 00000100:00100000:1.0:1788619930.076105:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571678592:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2917us (6138us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619930.076113:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1484 00000100:00100000:2.0:1788619930.077633:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b14380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589239808:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619930.077675:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009d6eb124/0x1cd7fe53687366ce lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7b3 expref: 9 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.077771:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d6eb124/0x1cd7fe53687366ce lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7b3 expref: 9 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.077786:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d5000, rc 0) 00000100:00100000:2.0:1788619930.078110:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571678720:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7876us (8060us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619930.078118:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1212 00000100:00100000:2.0:1788619930.082754:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619930.082927:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571678848 00000100:00100000:1.0:1788619930.082938:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571678848 00000100:00100000:1.0:1788619930.083027:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 902 00000100:00100000:1.0:1788619930.083037:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed62a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571678848:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:1.0:1788619930.083373:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1039390 -> 1039390 00000004:00010000:1.0:1788619930.083386:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619929 -> 1788619929 00000004:00010000:1.0:1788619930.083389:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619929 -> 1788619928 00000004:00010000:1.0:1788619930.083392:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619929 -> 1788619929 00000004:00010000:1.0:1788619930.083394:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1672 -> 1672 00000100:00100000:1.0:1788619930.083584:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed62a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571678848:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 546us (838us total) trans 4294967815 rc 0/0 00000100:00100000:1.0:1788619930.083592:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 902 00000100:00080000:2.0:1788619930.083834:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1788619930.088452:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619930.088521:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571678976 00000100:00100000:1.0:1788619930.088531:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571678976 00000100:00100000:1.0:1788619930.088627:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 688 00000100:00100000:1.0:1788619930.088638:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b15180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571678976:12345-192.168.201.5@tcp:22:fsx.0 00002000:00010000:1.0:1788619930.088895:0:6701:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1998533 -> 1998533 00002000:00010000:1.0:1788619930.088899:0:6701:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619929 -> 1788619929 00002000:00010000:1.0:1788619930.088911:0:6701:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619929 -> 1788619928 00002000:00010000:1.0:1788619930.088913:0:6701:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619929 -> 1788619929 00002000:00010000:1.0:1788619930.088915:0:6701:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 135 -> 135 00000100:00100000:1.0:1788619930.088970:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b15180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+8:2355:x1875503571678976:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 334us (528us total) trans 4294967503 rc 0/0 00000100:00100000:1.0:1788619930.088978:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 688 00000100:00080000:3.0:1788619930.089306:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1788619930.100041:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619930.100127:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571679104 00000100:00100000:3.0:1788619930.100137:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571679104 00000100:00100000:3.0:1788619930.100188:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1485 00000100:00100000:3.0:1788619930.100195:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df964700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571679104:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619930.100211:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619930.100228:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7400/0x1cd7fe53687366dc lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d7ba expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.100252:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe536873665e lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d74a expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.100259:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7400/0x1cd7fe53687366dc lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7ba expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.100283:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe536873665e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d74a expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.100292:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe536873665e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d74a expref: 10 pid: 5845 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619930.100302:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f1180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589240064:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619930.100401:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619930.107136:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619930.107385:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f1180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589240064:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619930.107420:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7400/0x1cd7fe53687366dc lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7ba expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.107441:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619930.107444:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619930.107450:0:5844:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7400/0x1cd7fe53687366dc lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7ba expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.107462:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d7400, rc 0) 00000100:00100000:1.0:1788619930.107526:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df964700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571679104:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7329us (7493us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619930.107535:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1485 00000100:00100000:2.0:1788619930.107923:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571679232 00000100:00100000:2.0:1788619930.107931:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571679232 00010000:00010000:2.0:1788619930.107962:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe536873665e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d74a expref: 11 pid: 5845 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619930.107990:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 488 00000100:00100000:2.0:1788619930.107997:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f3100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571679232:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619930.108008:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619930.108029:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe536873665e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d74a expref: 11 pid: 5845 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.108038:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe536873665e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d74a expref: 11 pid: 5845 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.108054:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe536873665e lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d74a expref: 11 pid: 5845 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.108066:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619930.108069:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7400/0x1cd7fe53687366dc lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7ba expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.108075:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7400/0x1cd7fe53687366dc lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7ba expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.108081:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7400/0x1cd7fe53687366dc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7ba expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.108110:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7400/0x1cd7fe53687366dc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7ba expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619930.108119:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed62680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589240192:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619930.108172:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164000 going to sleep for 11 seconds 00000100:00100000:2.0:1788619930.111937:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed62680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589240192:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619930.111969:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619930.112031:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f3100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571679232:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 4032us (4904us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619930.112041:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 488 00000100:00100000:3.0:1788619930.112496:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619930.112551:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571679360 00000100:00100000:0.0:1788619930.112567:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571679360 00002000:00100000:0.0:1788619930.112612:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e67f0700 x1875503571679360/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/0 e 0 to 0 dl 1788619941 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619930.112622:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e67f0700 with x1875503571679360 00000100:00100000:3.0:1788619930.112717:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 903 00000100:00100000:3.0:1788619930.112728:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571679360:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619930.112764:0:5858:0:(service.c:3462:ptlrpc_start_thread()) mdt_io[0] started 3 min 3 max 116 00000100:00100000:0.0:1788619930.112772:0:5858:0:(service.c:3521:ptlrpc_start_thread()) starting thread 'mdt_io00_003' 00000100:00100000:3.0:1788619930.113175:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571679360:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 449us (678us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619930.113184:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 903 00002000:00100000:3.0:1788619930.113191:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e67f0700 x1875503571679360/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/4536 e 0 to 0 dl 1788619941 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619930.113201:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e67f0700 with x1875503571679360 00000100:00100000:0.0:1788619930.118626:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619930.118813:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571679488 00000100:00100000:1.0:1788619930.118823:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571679488 00002000:00100000:1.0:1788619930.118871:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b17480 x1875503571679488/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/0 e 0 to 0 dl 1788619941 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619930.118883:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b17480 with x1875503571679488 00000100:00100000:1.0:1788619930.118918:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 904 00000100:00100000:1.0:1788619930.118927:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b17480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2353:x1875503571679488:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619930.119174:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b17480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2353:x1875503571679488:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 247us (558us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619930.119182:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 904 00002000:00100000:1.0:1788619930.119189:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b17480 x1875503571679488/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/4536 e 0 to 0 dl 1788619941 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619930.119197:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b17480 with x1875503571679488 00000100:00100000:1.0:1788619930.122114:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619930.122180:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571679616 00000100:00100000:0.0:1788619930.122188:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571679616 00000100:00100000:0.0:1788619930.122234:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1213 00000100:00100000:0.0:1788619930.122244:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df965f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571679616:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619930.122261:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619930.122281:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096deb5d6/0x1cd7fe53687366e3 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d7c1 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619930.122675:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b14e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589240320:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619930.122766:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5eaf240 going to sleep for 11 seconds 00002000:00010000:2.0:1788619930.124530:0:7654:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619928 -> 1788619929 00000100:00100000:0.0:1788619930.124608:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b14e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589240320:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619930.124637:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000096deb5d6/0x1cd7fe53687366e3 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7c1 expref: 9 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.124746:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096deb5d6/0x1cd7fe53687366e3 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7c1 expref: 9 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.124756:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d5c00, rc 0) 00000100:00100000:0.0:1788619930.124800:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df965f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571679616:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2557us (2692us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619930.124809:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1213 00000100:00100000:2.0:1788619930.126204:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619930.126262:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571679744 00000100:00100000:3.0:1788619930.126269:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571679744 00000100:00100000:3.0:1788619930.126309:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1214 00000100:00100000:3.0:1788619930.126316:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed62680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571679744:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619930.126348:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619930.126365:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e3d6efb/0x1cd7fe53687366ea lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d7c8 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619930.126418:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589240448:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619930.126480:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c300 going to sleep for 11 seconds 00000100:00100000:3.0:1788619930.128199:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589240448:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619930.128234:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009e3d6efb/0x1cd7fe53687366ea lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7c8 expref: 9 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.128262:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e3d6efb/0x1cd7fe53687366ea lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7c8 expref: 9 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.128275:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d7600, rc 0) 00000100:00100000:3.0:1788619930.128358:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed62680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571679744:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2040us (2158us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619930.128365:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1214 00000100:00100000:2.0:1788619930.132408:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619930.132602:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571679872 00000100:00100000:1.0:1788619930.132612:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571679872 00000100:00100000:1.0:1788619930.132672:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1486 00000100:00100000:1.0:1788619930.132681:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed62300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:14908:x1875503571679872:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619930.132704:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619930.132725:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687366f1 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d7cf expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.132754:0:9417:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7400/0x1cd7fe53687366dc lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7ba expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.132765:0:9417:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687366f1 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7cf expref: 11 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.132799:0:9417:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7400/0x1cd7fe53687366dc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7ba expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.132813:0:9417:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7400/0x1cd7fe53687366dc lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7ba expref: 13 pid: 5844 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619930.132826:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4ba2680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589240576:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619930.132910:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb000 going to sleep for 11 seconds 00000100:00100000:2.0:1788619930.135284:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619930.135431:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4ba2680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589240576:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619930.135466:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687366f1 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7cf expref: 11 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.135485:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619930.135488:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619930.135493:0:9417:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687366f1 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7cf expref: 11 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.135503:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c000, rc 0) 00000100:00100000:1.0:1788619930.135568:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed62300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571679872:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2887us (3172us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619930.135577:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1486 00000100:00100000:0.0:1788619930.135895:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571680000 00000100:00100000:0.0:1788619930.135903:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571680000 00002000:00100000:0.0:1788619930.135949:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fed60700 x1875503571680000/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/0 e 0 to 0 dl 1788619941 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619930.135958:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fed60700 with x1875503571680000 00010000:00010000:0.0:1788619930.135965:0:5858:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7400/0x1cd7fe53687366dc lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7ba expref: 14 pid: 5844 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.135979:0:5858:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7400/0x1cd7fe53687366dc lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7ba expref: 14 pid: 5844 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788619930.135986:0:5858:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e78fed60700 00000100:00100000:0.0:1788619930.136014:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 39 00000100:00100000:0.0:1788619930.136024:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed60700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571680000:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:0.0:1788619930.139466:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1039390 -> 1039390 00000004:00010000:0.0:1788619930.139475:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619929 -> 1788619930 00000004:00010000:0.0:1788619930.139478:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619928 -> 1788619928 00000004:00010000:0.0:1788619930.139480:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619929 -> 1788619930 00000004:00010000:0.0:1788619930.139482:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1672 -> 1672 00000100:00100000:0.0:1788619930.139567:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed60700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571680000:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 3529us (4276us total) trans 4294967816 rc 0/0 00000100:00100000:0.0:1788619930.139579:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 39 00002000:00100000:0.0:1788619930.139591:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fed60700 x1875503571680000/t4294967816(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/448 e 0 to 0 dl 1788619941 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619930.139601:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fed60700 with x1875503571680000 00010000:00010000:0.0:1788619930.139612:0:5858:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7400/0x1cd7fe53687366dc lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7ba expref: 16 pid: 5844 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.139627:0:5858:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7400/0x1cd7fe53687366dc lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7ba expref: 16 pid: 5844 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788619930.139639:0:5858:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e78fed60700 00000100:00100000:2.0:1788619930.142074:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619930.142185:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571680128 00000100:00100000:1.0:1788619930.142194:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571680128 00010000:00010000:1.0:1788619930.142230:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7400/0x1cd7fe53687366dc lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7ba expref: 16 pid: 5844 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619930.142271:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 489 00000100:00100000:1.0:1788619930.142281:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed61180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571680128:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619930.142295:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619930.142317:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7400/0x1cd7fe53687366dc lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7ba expref: 16 pid: 5844 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.142356:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7400/0x1cd7fe53687366dc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7ba expref: 16 pid: 5844 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.142388:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7400/0x1cd7fe53687366dc lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7ba expref: 16 pid: 5844 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.142404:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619930.142411:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687366f1 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7cf expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.142422:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687366f1 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7cf expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.142432:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687366f1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7cf expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.142467:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687366f1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7cf expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619930.142480:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4ba2680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589240704:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619930.142829:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb000 going to sleep for 11 seconds 00000100:00100000:1.0:1788619930.144422:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4ba2680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589240704:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619930.144449:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619930.144497:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed61180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571680128:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 2217us (2431us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619930.144507:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 489 00000100:00100000:2.0:1788619930.146135:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619930.146216:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571680256 00000100:00100000:3.0:1788619930.146225:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571680256 00000100:00100000:3.0:1788619930.146293:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 905 00000100:00100000:3.0:1788619930.146303:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed63480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571680256:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:3.0:1788619930.146664:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1039390 -> 1039390 00000004:00010000:3.0:1788619930.146669:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619930 -> 1788619930 00000004:00010000:3.0:1788619930.146671:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619928 -> 1788619928 00000004:00010000:3.0:1788619930.146674:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619930 -> 1788619930 00000004:00010000:3.0:1788619930.146676:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1672 -> 1672 00000100:00100000:3.0:1788619930.146746:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed63480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571680256:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 444us (619us total) trans 4294967817 rc 0/0 00000100:00100000:3.0:1788619930.146754:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 905 00000100:00100000:2.0:1788619930.149195:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619930.149356:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571680384 00000100:00100000:1.0:1788619930.149366:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571680384 00000100:00100000:1.0:1788619930.149431:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1487 00000100:00100000:1.0:1788619930.149441:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed61f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571680384:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619930.149464:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619930.149483:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe53687366f8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d7d6 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.149519:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687366f1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7cf expref: 11 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619930.149567:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4ba2680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589240832:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619930.149645:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb000 going to sleep for 11 seconds 00000100:00100000:2.0:1788619930.151062:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619930.151124:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571680512 00000100:00100000:1.0:1788619930.151130:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571680512 00000100:00100000:1.0:1788619930.151202:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1215 00000100:00100000:1.0:1788619930.151209:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed60a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571680512:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619930.151220:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619930.151234:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f980cda4/0x1cd7fe53687366ff lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d7dd expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619930.151281:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4ba2300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589240960:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619930.151367:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebcc0 going to sleep for 11 seconds 00000004:00010000:1.0:1788619930.154441:0:5845:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619928 -> 1788619929 00000100:00100000:1.0:1788619930.154474:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4ba2680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589240832:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619930.154499:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1039390 00002000:00010000:2.0:1788619930.154513:0:6697:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime: 1788619929 -> 1788619930 00002000:00010000:2.0:1788619930.154517:0:6697:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime: 1788619929 -> 1788619930 00010000:00010000:1.0:1788619930.154520:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe53687366f8 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7d6 expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619930.154533:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4ba2300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589240960:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619930.154541:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619930.154544:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619930.154549:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe53687366f8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7d6 expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.154551:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f980cda4/0x1cd7fe53687366ff lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7dd expref: 9 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.154574:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683ce00, rc 0) 00010000:00010000:2.0:1788619930.154582:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f980cda4/0x1cd7fe53687366ff lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7dd expref: 9 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.154590:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d200, rc 0) 00000100:00100000:1.0:1788619930.154638:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed61f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571680384:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5198us (5453us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619930.154648:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1487 00000100:00100000:2.0:1788619930.155080:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed60a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571680512:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3868us (4023us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619930.155089:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1215 00000100:00100000:0.0:1788619930.158530:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619930.158987:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619930.159372:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571680768 00000100:00100000:2.0:1788619930.159383:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571680768 00000100:00100000:2.0:1788619930.159444:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1216 00000100:00100000:2.0:1788619930.159454:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900985c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571680768:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619930.159473:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619930.159495:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b040813e/0x1cd7fe5368736706 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d7eb expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619930.159552:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed61880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589241088:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619930.159702:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571680640 00000100:00100000:1.0:1788619930.159708:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571680640 00000100:00100000:1.0:1788619930.159747:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1488 00000100:00100000:1.0:1788619930.159755:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b14e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571680640:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619930.159767:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619930.159781:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe536873670d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d7e4 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.159812:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687366f1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7cf expref: 11 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619930.159982:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4ba2680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589241216:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619930.161489:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb000 going to sleep for 11 seconds 00000100:00100000:2.0:1788619930.162824:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164000 going to sleep for 11 seconds 00000100:00100000:2.0:1788619930.165620:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed61880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589241088:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619930.165693:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4ba2680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589241216:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619930.165711:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1039390 00010000:00010000:1.0:1788619930.165731:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe536873670d lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7e4 expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.165753:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619930.165757:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619930.165763:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe536873670d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7e4 expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.165778:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d400, rc 0) 00010000:00010000:2.0:1788619930.165839:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b040813e/0x1cd7fe5368736706 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7eb expref: 9 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.165859:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b040813e/0x1cd7fe5368736706 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7eb expref: 9 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.165875:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f72600, rc 0) 00000100:00100000:1.0:1788619930.165940:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b14e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571680640:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6185us (7421us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619930.165951:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1488 00000100:00100000:1.0:1788619930.166458:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900985c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571680768:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7003us (7470us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619930.166466:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1216 00000100:00100000:0.0:1788619930.170232:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619930.170323:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571680896 00000100:00100000:3.0:1788619930.170546:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571680896 00000100:00100000:0.0:1788619930.170714:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 689 00000100:00100000:0.0:1788619930.170725:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b14a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+8:2354:x1875503571680896:12345-192.168.201.5@tcp:22:fsx.0 00002000:00010000:0.0:1788619930.170981:0:6702:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1998533 -> 1998533 00002000:00010000:0.0:1788619930.170986:0:6702:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619930 -> 1788619929 00002000:00010000:0.0:1788619930.170988:0:6702:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619929 -> 1788619928 00002000:00010000:0.0:1788619930.170991:0:6702:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619930 -> 1788619929 00002000:00010000:0.0:1788619930.170993:0:6702:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 135 -> 135 00000100:00100000:0.0:1788619930.171052:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b14a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571680896:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 329us (831us total) trans 4294967504 rc 0/0 00000100:00100000:0.0:1788619930.171060:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 689 00000100:00100000:0.0:1788619930.174802:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619930.174855:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619930.174948:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571681152 00000100:00100000:2.0:1788619930.174959:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571681152 00000100:00100000:2.0:1788619930.175008:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1217 00000100:00100000:2.0:1788619930.175018:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b16d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571681152:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619930.175035:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619930.175052:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003b7bbd8d/0x1cd7fe5368736714 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d7f9 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619930.175109:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed61880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589241344:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619930.175189:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164000 going to sleep for 11 seconds 00000100:00100000:1.0:1788619930.176318:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571681024 00000100:00100000:1.0:1788619930.176466:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571681024 00000100:00100000:1.0:1788619930.176518:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1489 00000100:00100000:1.0:1788619930.176527:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b14380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571681024:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619930.176542:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619930.176568:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe536873671b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d7f2 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.176602:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687366f1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7cf expref: 11 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619930.176746:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4ba2680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589241472:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619930.176847:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb000 going to sleep for 11 seconds 00002000:00010000:2.0:1788619930.179807:0:7654:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime: 1788619929 -> 1788619930 00002000:00010000:2.0:1788619930.179816:0:7654:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619928 -> 1788619929 00002000:00010000:2.0:1788619930.179819:0:7654:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime: 1788619929 -> 1788619930 00000100:00100000:2.0:1788619930.179846:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed61880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589241344:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619930.179885:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003b7bbd8d/0x1cd7fe5368736714 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7f9 expref: 9 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.179910:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b7bbd8d/0x1cd7fe5368736714 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7f9 expref: 9 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.179924:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f72c00, rc 0) 00000100:00100000:2.0:1788619930.179986:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b16d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571681152:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4968us (5193us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619930.179996:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1217 00000100:00100000:1.0:1788619930.180323:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4ba2680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589241472:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619930.180366:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1039390 00010000:00010000:1.0:1788619930.180381:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe536873671b lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7f2 expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.180395:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619930.180397:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619930.180402:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe536873671b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7f2 expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.180413:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683ce00, rc 0) 00000100:00100000:1.0:1788619930.180454:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b14380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571681024:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3928us (5598us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619930.180460:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1489 00000100:00100000:0.0:1788619930.187312:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619930.187642:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619930.187733:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571681408 00000100:00100000:3.0:1788619930.187752:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571681408 00000100:00100000:3.0:1788619930.187851:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1218 00000100:00100000:3.0:1788619930.187873:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6000000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571681408:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619930.187898:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619930.187922:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e3d6efb/0x1cd7fe5368736722 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d807 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619930.188005:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589241600:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619930.188120:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c300 going to sleep for 11 seconds 00000100:00100000:1.0:1788619930.188305:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571681280 00000100:00100000:1.0:1788619930.188316:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571681280 00000100:00100000:0.0:1788619930.190070:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1490 00000100:00100000:0.0:1788619930.190087:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6000e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2353:x1875503571681280:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619930.190109:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619930.190132:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5a00/0x1cd7fe5368736729 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d800 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.190185:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687366f1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7cf expref: 11 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619930.190234:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c74f6d80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589241728:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619930.190420:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5eaf240 going to sleep for 11 seconds 00000100:00100000:3.0:1788619930.191381:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589241600:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619930.191423:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009e3d6efb/0x1cd7fe5368736722 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d807 expref: 9 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.191446:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e3d6efb/0x1cd7fe5368736722 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d807 expref: 9 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.191458:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d7600, rc 0) 00000100:00100000:3.0:1788619930.193045:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6000000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571681408:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5032us (5259us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619930.193058:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1218 00000100:00100000:0.0:1788619930.194485:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c74f6d80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589241728:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619930.194880:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1039390 00010000:00010000:0.0:1788619930.194970:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5a00/0x1cd7fe5368736729 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d800 expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.194996:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619930.194999:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619930.195005:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5a00/0x1cd7fe5368736729 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d800 expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.195020:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d5a00, rc 0) 00000100:00100000:0.0:1788619930.195131:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6000e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2353:x1875503571681280:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5048us (7829us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619930.195140:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1490 00000100:00100000:0.0:1788619930.199220:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619930.199293:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571681536 00000100:00100000:3.0:1788619930.199302:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571681536 00002000:00100000:3.0:1788619930.199489:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1900544->1904639) req@ffff8e78c74f6d80 x1875503571681536/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:26/0 lens 488/0 e 0 to 0 dl 1788619941 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619930.199501:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c74f6d80 with x1875503571681536 ext(1900544->1904639) 00002000:00010000:3.0:1788619930.199506:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c74f6d80 00000100:00100000:3.0:1788619930.199572:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 690 00000100:00100000:3.0:1788619930.199584:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571681536:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619930.199849:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571681536:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 266us (634us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619930.199857:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 690 00002000:00100000:3.0:1788619930.199863:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1900544->1904639) req@ffff8e78c74f6d80 x1875503571681536/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:26/0 lens 488/4536 e 0 to 0 dl 1788619941 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619930.199871:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c74f6d80 with x1875503571681536 ext(1900544->1904639) 00002000:00010000:3.0:1788619930.199876:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c74f6d80 00000100:00100000:2.0:1788619930.204448:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619930.204564:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571681664 00000100:00100000:0.0:1788619930.204574:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571681664 00002000:00100000:0.0:1788619930.204626:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1904640->1908735) req@ffff8e78fed61880 x1875503571681664/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:26/0 lens 488/0 e 0 to 0 dl 1788619941 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619930.204637:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fed61880 with x1875503571681664 ext(1904640->1908735) 00002000:00010000:0.0:1788619930.204643:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fed61880 00000100:00100000:0.0:1788619930.204728:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 691 00000100:00100000:0.0:1788619930.204739:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed61880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571681664:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619930.205027:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed61880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571681664:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 289us (645us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619930.205034:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 691 00002000:00100000:0.0:1788619930.205041:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1904640->1908735) req@ffff8e78fed61880 x1875503571681664/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:26/0 lens 488/4536 e 0 to 0 dl 1788619941 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619930.205049:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fed61880 with x1875503571681664 ext(1904640->1908735) 00002000:00010000:0.0:1788619930.205053:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fed61880 00000100:00100000:2.0:1788619930.208292:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619930.208359:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571681792 00000100:00100000:2.0:1788619930.208368:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571681792 00000100:00100000:2.0:1788619930.208436:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1491 00000100:00100000:2.0:1788619930.208446:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed62d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571681792:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619930.208465:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619930.208486:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72a00/0x1cd7fe5368736730 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d80e expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.208521:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687366f1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7cf expref: 11 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619930.208569:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed60e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589241856:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619930.208644:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164000 going to sleep for 11 seconds 00000100:00100000:2.0:1788619930.210131:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619930.210209:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571681920 00000100:00100000:3.0:1788619930.210217:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571681920 00000100:00100000:3.0:1788619930.210267:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1219 00000100:00100000:3.0:1788619930.210276:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed63800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571681920:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619930.210291:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619930.210306:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e3d6efb/0x1cd7fe5368736737 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d815 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619930.210394:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589241984:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619930.210456:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619930.212674:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed60e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589241856:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619930.212701:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1039390 00010000:00010000:1.0:1788619930.212721:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72a00/0x1cd7fe5368736730 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d80e expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.212742:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619930.212745:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619930.212751:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72a00/0x1cd7fe5368736730 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d80e expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.212765:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f72a00, rc 0) 00000100:00100000:1.0:1788619930.212830:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed62d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571681792:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4385us (4547us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619930.212839:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1491 00000100:00100000:3.0:1788619930.213037:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589241984:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619930.213057:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009e3d6efb/0x1cd7fe5368736737 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d815 expref: 9 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.213074:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e3d6efb/0x1cd7fe5368736737 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d815 expref: 9 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.213088:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d7600, rc 0) 00000100:00100000:3.0:1788619930.213109:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed63800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571681920:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2836us (2984us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619930.213115:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1219 00000100:00100000:1.0:1788619930.216085:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619930.216176:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571682176 00000100:00100000:2.0:1788619930.216185:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571682176 00000100:00100000:2.0:1788619930.216244:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1220 00000100:00100000:2.0:1788619930.216254:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571682176:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619930.216272:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619930.216291:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002dd9059c/0x1cd7fe536873673e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d823 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619930.216394:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed62680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589242112:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619930.216472:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619930.219250:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619930.219323:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571682048 00000100:00100000:1.0:1788619930.219364:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571682048 00000100:00100000:1.0:1788619930.219423:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1492 00000100:00100000:1.0:1788619930.219432:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed60a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571682048:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619930.219452:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619930.219472:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368736745 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d81c expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.219514:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687366f1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7cf expref: 11 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619930.219566:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4ba1c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589242240:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619930.219638:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb000 going to sleep for 11 seconds 00000100:00100000:2.0:1788619930.221204:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed62680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589242112:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619930.221248:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002dd9059c/0x1cd7fe536873673e lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d823 expref: 9 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.221272:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002dd9059c/0x1cd7fe536873673e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d823 expref: 9 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.221286:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f73c00, rc 0) 00000100:00100000:2.0:1788619930.221369:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571682176:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5116us (5292us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619930.222410:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1220 00000100:00100000:1.0:1788619930.222974:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4ba1c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589242240:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619930.222994:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1039390 00010000:00010000:1.0:1788619930.223010:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368736745 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d81c expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.223030:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619930.223033:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619930.223039:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368736745 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d81c expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.223052:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683de00, rc 0) 00000100:00100000:1.0:1788619930.223106:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed60a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571682048:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3675us (3865us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619930.223114:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1492 00000100:00100000:3.0:1788619930.227527:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619930.227584:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619930.227689:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571682432 00000100:00100000:2.0:1788619930.227698:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571682432 00000100:00100000:2.0:1788619930.227754:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1221 00000100:00100000:2.0:1788619930.227764:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571682432:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619930.227781:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619930.227797:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a4727b52/0x1cd7fe536873674c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d831 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619930.227865:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed62680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589242368:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619930.227959:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619930.228485:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571682304 00000100:00100000:0.0:1788619930.228492:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571682304 00000100:00100000:0.0:1788619930.228531:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1493 00000100:00100000:0.0:1788619930.228539:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2353:x1875503571682304:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619930.228561:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619930.228575:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5a00/0x1cd7fe5368736753 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d82a expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.228608:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687366f1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7cf expref: 11 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619930.228635:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c74f7800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589242496:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619930.228679:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5eafc00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619930.231033:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed62680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589242368:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619930.231072:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a4727b52/0x1cd7fe536873674c lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d831 expref: 9 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.231094:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a4727b52/0x1cd7fe536873674c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d831 expref: 9 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.231108:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f73a00, rc 0) 00000100:00100000:2.0:1788619930.231170:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571682432:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3408us (3585us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619930.231178:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1221 00000100:00100000:3.0:1788619930.234005:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c74f7800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589242496:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619930.234034:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1039390 00010000:00010000:3.0:1788619930.234055:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5a00/0x1cd7fe5368736753 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d82a expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.234076:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619930.234080:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619930.234086:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5a00/0x1cd7fe5368736753 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d82a expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.234100:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d5a00, rc 0) 00000100:00100000:3.0:1788619930.234163:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2353:x1875503571682304:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5624us (6645us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619930.234171:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1493 00000100:00100000:0.0:1788619930.237101:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619930.237185:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571682560 00000100:00100000:3.0:1788619930.237195:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571682560 00000100:00100000:3.0:1788619930.237253:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1222 00000100:00100000:3.0:1788619930.237263:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:14908:x1875503571682560:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619930.237284:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619930.237304:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e3d6efb/0x1cd7fe536873675a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d838 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.237368:0:6698:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000009e195d52/0x1cd7fe5368736665 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d751 expref: 8 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.237386:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(5 12) 00010000:00010000:3.0:1788619930.237391:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(6 12) 00010000:00010000:3.0:1788619930.237396:0:6698:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e3d6efb/0x1cd7fe536873675a lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1126400->1130495] (req 1126400->1130495) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d838 expref: 9 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.237430:0:6698:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000009e195d52/0x1cd7fe5368736665 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d751 expref: 8 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.237445:0:6698:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000009e195d52/0x1cd7fe5368736665 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d751 expref: 8 pid: 6698 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619930.237459:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589242624:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619930.237547:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619930.240843:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589242624:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619930.240884:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009e3d6efb/0x1cd7fe536873675a lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1126400->1130495] (req 1126400->1130495) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d838 expref: 9 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.240915:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d7600, rc 0) 00000100:00100000:3.0:1788619930.240983:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+9:14908:x1875503571682560:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3721us (3891us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619930.240995:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1222 00000100:00100000:1.0:1788619930.242869:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619930.243089:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571682688 00000100:00100000:3.0:1788619930.243098:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571682688 00002000:00100000:3.0:1788619930.243144:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1900544->1908735) req@ffff8e77c4ba1c00 x1875503571682688/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:26/0 lens 8680/0 e 0 to 0 dl 1788619941 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619930.243154:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4ba1c00 with x1875503571682688 ext(1900544->1908735) 00010000:00010000:3.0:1788619930.243163:0:6703:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000009e195d52/0x1cd7fe5368736665 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d751 expref: 9 pid: 6698 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.243178:0:6703:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000009e195d52/0x1cd7fe5368736665 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d751 expref: 9 pid: 6698 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788619930.243186:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff8e77c4ba1c00 00000100:00100000:3.0:1788619930.243256:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 24 00000100:00100000:3.0:1788619930.243265:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571682688:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:3.0:1788619930.243770:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571682688:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 505us (907us total) trans 4294967505 rc 0/0 00000100:00100000:3.0:1788619930.243780:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 24 00002000:00100000:3.0:1788619930.243788:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1900544->1908735) req@ffff8e77c4ba1c00 x1875503571682688/t4294967505(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:26/0 lens 8680/448 e 0 to 0 dl 1788619941 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619930.243796:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4ba1c00 with x1875503571682688 ext(1900544->1908735) 00010000:00010000:3.0:1788619930.243802:0:6703:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000009e195d52/0x1cd7fe5368736665 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d751 expref: 11 pid: 6698 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.245039:0:6703:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000009e195d52/0x1cd7fe5368736665 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d751 expref: 11 pid: 6698 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788619930.245052:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff8e77c4ba1c00 00000100:00100000:0.0:1788619930.246892:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619930.247146:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571682816 00000100:00100000:2.0:1788619930.247155:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571682816 00010000:00010000:2.0:1788619930.247194:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000009e195d52/0x1cd7fe5368736665 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d751 expref: 11 pid: 6698 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619930.247274:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 490 00000100:00100000:2.0:1788619930.247284:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f4000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571682816:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619930.247299:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619930.247320:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000009e195d52/0x1cd7fe5368736665 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d751 expref: 11 pid: 6698 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.247351:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000009e195d52/0x1cd7fe5368736665 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d751 expref: 11 pid: 6698 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.247375:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e195d52/0x1cd7fe5368736665 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d751 expref: 11 pid: 6698 timeout: 522 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.247406:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(6 12) 00010000:00010000:2.0:1788619930.247410:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(6 12) 00010000:00010000:2.0:1788619930.247418:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000009e3d6efb/0x1cd7fe536873675a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1126400->1130495) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d838 expref: 8 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.247429:0:5829:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e3d6efb/0x1cd7fe536873675a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1126400->1130495) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d838 expref: 8 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.247464:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000009e3d6efb/0x1cd7fe536873675a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1126400->1130495) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d838 expref: 8 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619930.247478:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fed62680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1875503589242752:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619930.247567:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619930.252578:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619930.252682:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fed62680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1875503589242752:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619930.252712:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619930.252775:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f4000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571682816:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 5492us (5889us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619930.252786:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 490 00000100:00100000:0.0:1788619930.254176:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571682944 00000100:00100000:0.0:1788619930.254189:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571682944 00002000:00100000:0.0:1788619930.254242:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1126400->1998847) req@ffff8e78fed60e00 x1875503571682944/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 504/0 e 0 to 0 dl 1788619941 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619930.254255:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fed60e00 with x1875503571682944 ext(1126400->1998847) 00002000:00010000:0.0:1788619930.254260:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fed60e00 00000100:00100000:0.0:1788619930.254356:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 692 00000100:00100000:0.0:1788619930.254366:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed60e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2355:x1875503571682944:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619930.274030:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed60e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2355:x1875503571682944:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 19654us (21450us total) trans 0 rc 778240/778240 00000100:00100000:0.0:1788619930.274044:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 692 00002000:00100000:0.0:1788619930.274057:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1126400->1998847) req@ffff8e78fed60e00 x1875503571682944/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 504/440 e 0 to 0 dl 1788619941 ref 1 fl Complete:/600/0 rc 778240/778240 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619930.274067:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fed60e00 with x1875503571682944 ext(1126400->1998847) 00002000:00010000:0.0:1788619930.274073:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fed60e00 00000100:00100000:2.0:1788619930.277626:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619930.277717:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571683072 00000100:00100000:3.0:1788619930.277728:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571683072 00002000:00100000:3.0:1788619930.277780:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1130496->1134591) req@ffff8e78fed62680 x1875503571683072/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/0 e 0 to 0 dl 1788619941 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619930.277791:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fed62680 with x1875503571683072 ext(1130496->1134591) 00002000:00010000:3.0:1788619930.277797:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fed62680 00000100:00100000:3.0:1788619930.277840:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 693 00000100:00100000:3.0:1788619930.277850:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed62680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2355:x1875503571683072:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619930.278108:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed62680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2355:x1875503571683072:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 258us (492us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619930.278116:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 693 00002000:00100000:3.0:1788619930.278123:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1130496->1134591) req@ffff8e78fed62680 x1875503571683072/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/4536 e 0 to 0 dl 1788619941 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619930.278131:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78fed62680 with x1875503571683072 ext(1130496->1134591) 00002000:00010000:3.0:1788619930.278135:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78fed62680 00000100:00100000:2.0:1788619930.286040:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619930.286260:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571683200 00000100:00100000:0.0:1788619930.286270:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571683200 00002000:00100000:0.0:1788619930.286323:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1134592->1138687) req@ffff8e78df96b480 x1875503571683200/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/0 e 0 to 0 dl 1788619941 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619930.286372:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df96b480 with x1875503571683200 ext(1134592->1138687) 00002000:00010000:0.0:1788619930.286388:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df96b480 00000100:00100000:0.0:1788619930.286421:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 694 00000100:00100000:0.0:1788619930.286433:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df96b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571683200:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619930.286944:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df96b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571683200:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 510us (911us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619930.286953:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 694 00002000:00100000:0.0:1788619930.286960:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1134592->1138687) req@ffff8e78df96b480 x1875503571683200/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/4536 e 0 to 0 dl 1788619941 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619930.286969:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df96b480 with x1875503571683200 ext(1134592->1138687) 00002000:00010000:0.0:1788619930.286973:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df96b480 00000100:00100000:2.0:1788619930.291214:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619930.291314:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571683328 00000100:00100000:1.0:1788619930.291459:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571683328 00002000:00100000:0.0:1788619930.291600:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1138688->1142783) req@ffff8e78df968a80 x1875503571683328/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/0 e 0 to 0 dl 1788619941 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619930.291613:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df968a80 with x1875503571683328 ext(1138688->1142783) 00002000:00010000:0.0:1788619930.291619:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df968a80 00000100:00100000:0.0:1788619930.291692:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 695 00000100:00100000:0.0:1788619930.291703:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df968a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571683328:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619930.291978:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df968a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571683328:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 275us (772us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619930.291986:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 695 00002000:00100000:0.0:1788619930.291992:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1138688->1142783) req@ffff8e78df968a80 x1875503571683328/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/4536 e 0 to 0 dl 1788619941 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619930.292000:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df968a80 with x1875503571683328 ext(1138688->1142783) 00002000:00010000:0.0:1788619930.292004:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df968a80 00000100:00100000:1.0:1788619930.295286:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619930.295424:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571683456 00000100:00100000:0.0:1788619930.295434:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571683456 00002000:00100000:0.0:1788619930.295482:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1142784->1146879) req@ffff8e77c4ba2d80 x1875503571683456/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/0 e 0 to 0 dl 1788619941 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619930.295494:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4ba2d80 with x1875503571683456 ext(1142784->1146879) 00002000:00010000:0.0:1788619930.295500:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4ba2d80 00000100:00100000:0.0:1788619930.295527:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 696 00000100:00100000:0.0:1788619930.295536:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571683456:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619930.296047:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571683456:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 509us (769us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619930.296057:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 696 00002000:00100000:0.0:1788619930.296064:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1142784->1146879) req@ffff8e77c4ba2d80 x1875503571683456/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/4536 e 0 to 0 dl 1788619941 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619930.296072:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4ba2d80 with x1875503571683456 ext(1142784->1146879) 00002000:00010000:0.0:1788619930.296075:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4ba2d80 00000100:00100000:2.0:1788619930.299876:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619930.300012:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571683584 00000100:00100000:0.0:1788619930.300023:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571683584 00002000:00100000:0.0:1788619930.300073:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1126400->1146879) req@ffff8e78df969f80 x1875503571683584/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/0 e 0 to 0 dl 1788619941 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619930.300084:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df969f80 with x1875503571683584 ext(1126400->1146879) 00002000:00010000:0.0:1788619930.300089:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df969f80 00000100:00100000:0.0:1788619930.300124:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 697 00000100:00100000:0.0:1788619930.300134:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df969f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2353:x1875503571683584:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:0.0:1788619930.303177:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df969f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+11:2353:x1875503571683584:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 3035us (3301us total) trans 4294967506 rc 0/0 00000100:00100000:0.0:1788619930.303193:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 697 00002000:00100000:0.0:1788619930.303206:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1126400->1146879) req@ffff8e78df969f80 x1875503571683584/t4294967506(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/448 e 0 to 0 dl 1788619941 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619930.303216:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df969f80 with x1875503571683584 ext(1126400->1146879) 00002000:00010000:0.0:1788619930.303224:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df969f80 00000100:00100000:2.0:1788619930.306842:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619930.306943:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571683712 00000100:00100000:1.0:1788619930.306954:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571683712 00000100:00100000:1.0:1788619930.307012:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1223 00000100:00100000:1.0:1788619930.307024:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df968e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+11:2352:x1875503571683712:12345-192.168.201.5@tcp:16:fsx.0 00000001:00080000:3.0:1788619930.334671:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967506 is committed 00000001:00080000:3.0:1788619930.334692:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967504 is committed 00000001:00080000:3.0:1788619930.334708:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967505 is committed 00000100:00100000:1.0:1788619930.334949:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df968e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571683712:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 27921us (28121us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619930.334962:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1223 00000100:00100000:2.0:1788619930.337971:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619930.338044:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571683840 00000100:00100000:3.0:1788619930.338054:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571683840 00000100:00100000:3.0:1788619930.338115:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1224 00000100:00100000:3.0:1788619930.338126:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df96bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571683840:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619930.338145:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619930.338165:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d20f1966/0x1cd7fe5368736761 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d83f expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619930.338225:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589242880:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619930.340695:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c240 going to sleep for 11 seconds 00002000:00010000:0.0:1788619930.341136:0:6696:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619929 -> 1788619930 00002000:00010000:0.0:1788619930.341141:0:6696:0:(ofd_lvb.c:284:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks: 135 -> 391 00000100:00100000:0.0:1788619930.341166:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589242880:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619930.341205:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d20f1966/0x1cd7fe5368736761 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d83f expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.341229:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d20f1966/0x1cd7fe5368736761 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d83f expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.341242:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d6000, rc 0) 00000100:00100000:0.0:1788619930.341508:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df96bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571683840:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3382us (3545us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619930.341517:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1224 00000100:00100000:0.0:1788619930.344180:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619930.344269:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571683968 00000100:00100000:2.0:1788619930.344278:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571683968 00000100:00100000:2.0:1788619930.344507:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1225 00000100:00100000:2.0:1788619930.344518:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571683968:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619930.344538:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619930.344567:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c1bed124/0x1cd7fe5368736768 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d846 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619930.344624:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df96b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589243008:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619930.344705:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164540 going to sleep for 11 seconds 00000100:00100000:0.0:1788619930.346778:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df96b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589243008:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619930.346812:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c1bed124/0x1cd7fe5368736768 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d846 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.346855:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c1bed124/0x1cd7fe5368736768 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d846 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.346867:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f50a00, rc 0) 00000100:00100000:0.0:1788619930.346928:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571683968:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2411us (2755us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619930.346937:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1225 00000100:00100000:3.0:1788619930.349865:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619930.350121:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571684096 00000100:00100000:1.0:1788619930.350131:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571684096 00000100:00100000:1.0:1788619930.350190:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1494 00000100:00100000:1.0:1788619930.350200:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2353:x1875503571684096:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619930.350220:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619930.350240:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe536873676f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d84d expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.350278:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687366f1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7cf expref: 11 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619930.350318:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4ba3100 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589243136:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619930.350435:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590a80 going to sleep for 11 seconds 00000004:00010000:0.0:1788619930.352882:0:11397:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619929 -> 1788619930 00000100:00100000:0.0:1788619930.352914:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4ba3100 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589243136:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619930.352936:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1039390 00010000:00010000:0.0:1788619930.352954:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe536873676f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d84d expref: 16 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.352972:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619930.352975:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619930.352980:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe536873676f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d84d expref: 16 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.352995:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683de00, rc 0) 00000100:00100000:0.0:1788619930.353050:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2353:x1875503571684096:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2851us (3194us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619930.353060:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1494 00000100:00100000:2.0:1788619930.356174:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619930.356280:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571684224 00000100:00100000:1.0:1788619930.356290:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571684224 00000100:00100000:1.0:1788619930.356408:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1495 00000100:00100000:1.0:1788619930.356419:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df968000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2352:x1875503571684224:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619930.356440:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619930.356464:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe5368736776 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d854 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.356502:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687366f1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7cf expref: 11 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619930.356544:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4ba1500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589243264:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619930.356630:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c85903c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619930.359058:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4ba1500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589243264:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619930.359084:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1039390 00010000:00010000:1.0:1788619930.359103:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe5368736776 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d854 expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.359124:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619930.359128:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619930.359133:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe5368736776 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d854 expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.359148:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d800, rc 0) 00000100:00100000:1.0:1788619930.359209:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df968000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2352:x1875503571684224:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2791us (3049us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619930.359218:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1495 00000100:00100000:0.0:1788619930.362852:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619930.362941:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571684352 00000100:00100000:3.0:1788619930.362950:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571684352 00000100:00100000:3.0:1788619930.363006:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1226 00000100:00100000:3.0:1788619930.363017:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571684352:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619930.363035:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619930.363053:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003566e5e5/0x1cd7fe536873677d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d85b expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619930.363110:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f1180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589243392:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619930.363190:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48ccc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619930.368389:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f1180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589243392:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619930.368428:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003566e5e5/0x1cd7fe536873677d lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d85b expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.368453:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003566e5e5/0x1cd7fe536873677d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d85b expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.368471:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d6600, rc 0) 00000100:00100000:3.0:1788619930.368529:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571684352:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5514us (5686us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619930.368538:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1226 00000100:00100000:2.0:1788619930.373923:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619930.374000:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571684480 00000100:00100000:1.0:1788619930.374008:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571684480 00000100:00100000:1.0:1788619930.374061:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1227 00000100:00100000:1.0:1788619930.374070:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df969880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571684480:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619930.374086:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619930.374103:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001fc792ba/0x1cd7fe5368736784 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d862 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619930.374162:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4ba1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589243520:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619930.374236:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c85903c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619930.379587:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4ba1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589243520:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619930.379628:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001fc792ba/0x1cd7fe5368736784 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d862 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.379652:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001fc792ba/0x1cd7fe5368736784 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d862 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.379667:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d800, rc 0) 00000100:00100000:1.0:1788619930.379730:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df969880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571684480:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5661us (5815us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619930.379739:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1227 00000100:00100000:2.0:1788619930.382646:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619930.382755:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571684608 00000100:00100000:0.0:1788619930.382765:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571684608 00000100:00100000:0.0:1788619930.382914:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1228 00000100:00100000:0.0:1788619930.382924:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df96a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:14908:x1875503571684608:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619930.382942:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619930.382959:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fd10f4fd/0x1cd7fe536873678b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d869 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.382989:0:6696:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000009e3d6efb/0x1cd7fe536873675a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1126400->1130495) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d838 expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.382997:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=1s v=(6 12) 00010000:00010000:0.0:1788619930.383001:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(7 12) 00010000:00010000:0.0:1788619930.383006:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd10f4fd/0x1cd7fe536873678b lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->1056767] (req 1048576->1056767) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d869 expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.383037:0:6696:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000009e3d6efb/0x1cd7fe536873675a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1126400->1130495) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d838 expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.383050:0:6696:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000009e3d6efb/0x1cd7fe536873675a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1126400->1130495) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d838 expref: 5 pid: 6698 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619930.383062:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589243648:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619930.383168:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5eaf300 going to sleep for 11 seconds 00000100:00100000:2.0:1788619930.388199:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619930.388303:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589243648:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619930.388366:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000fd10f4fd/0x1cd7fe536873678b lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->1056767] (req 1048576->1056767) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d869 expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.388501:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d5800, rc 0) 00000100:00100000:0.0:1788619930.388570:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df96a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:14908:x1875503571684608:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5647us (5933us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619930.388580:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1228 00000100:00100000:3.0:1788619930.388657:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571684736 00000100:00100000:3.0:1788619930.388664:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571684736 00010000:00010000:3.0:1788619930.388698:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000009e3d6efb/0x1cd7fe536873675a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1126400->1130495) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d838 expref: 6 pid: 6698 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619930.388729:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 491 00000100:00100000:3.0:1788619930.388739:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df96a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571684736:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619930.388752:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619930.388775:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000009e3d6efb/0x1cd7fe536873675a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1126400->1130495) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d838 expref: 6 pid: 6698 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.388788:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000009e3d6efb/0x1cd7fe536873675a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1126400->1130495) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d838 expref: 6 pid: 6698 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.388812:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e3d6efb/0x1cd7fe536873675a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1126400->1130495) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d838 expref: 6 pid: 6698 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.388971:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(7 12) 00010000:00010000:3.0:1788619930.388976:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(7 12) 00010000:00010000:3.0:1788619930.388985:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000fd10f4fd/0x1cd7fe536873678b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1056767) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d869 expref: 5 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.388996:0:13330:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fd10f4fd/0x1cd7fe536873678b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1056767) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d869 expref: 5 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.389031:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000fd10f4fd/0x1cd7fe536873678b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1056767) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d869 expref: 5 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619930.389044:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f1180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589243776:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619930.389094:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48ccc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619930.392855:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619930.392894:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619930.393008:0:13346:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571684864 00000100:00100000:1.0:1788619930.393017:0:13346:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571684864 00000100:00100000:1.0:1788619930.393091:0:13346:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 906 00000100:00100000:1.0:1788619930.393101:0:13346:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df96b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571684864:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:1.0:1788619930.393351:0:13346:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1039390 -> 1039390 00000004:00010000:1.0:1788619930.393355:0:13346:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619930 -> 1788619930 00000004:00010000:1.0:1788619930.393358:0:13346:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619930 -> 1788619928 00000004:00010000:1.0:1788619930.393360:0:13346:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619930 -> 1788619930 00000004:00010000:1.0:1788619930.393362:0:13346:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1672 -> 1672 00000100:00100000:1.0:1788619930.393436:0:13346:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df96b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571684864:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 336us (589us total) trans 4294967818 rc 0/0 00000100:00100000:1.0:1788619930.393444:0:13346:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 906 00000100:00100000:0.0:1788619930.393581:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571684992 00000100:00100000:0.0:1788619930.393585:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571684992 00000100:00100000:0.0:1788619930.393624:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 698 00000100:00100000:0.0:1788619930.393632:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df969500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571684992:12345-192.168.201.5@tcp:22:fsx.0 00002000:00010000:0.0:1788619930.393842:0:6702:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1998533 -> 1998533 00002000:00010000:0.0:1788619930.393846:0:6702:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619930 -> 1788619930 00002000:00010000:0.0:1788619930.393848:0:6702:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619930 -> 1788619929 00002000:00010000:0.0:1788619930.393850:0:6702:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619930 -> 1788619930 00002000:00010000:0.0:1788619930.393852:0:6702:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 391 -> 391 00000100:00100000:0.0:1788619930.393876:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df969500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571684992:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 245us (981us total) trans 4294967507 rc 0/0 00000100:00100000:0.0:1788619930.393883:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 698 00000100:00100000:3.0:1788619930.394394:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f1180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589243776:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619930.394415:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619930.394445:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df96a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571684736:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 5708us (6256us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619930.394453:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 491 00000100:00100000:2.0:1788619930.398803:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619930.399013:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619930.399104:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571685120 00000100:00100000:3.0:1788619930.399115:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571685120 00000100:00100000:3.0:1788619930.399177:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1496 00000100:00100000:3.0:1788619930.399187:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df96aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2353:x1875503571685120:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619930.399206:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619930.399225:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6600/0x1cd7fe5368736792 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d870 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.399262:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687366f1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7cf expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619930.399304:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f1180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589243904:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619930.399542:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48ccc0 going to sleep for 11 seconds 00000004:00010000:3.0:1788619930.401658:0:5845:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619928 -> 1788619930 00000100:00100000:3.0:1788619930.401687:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f1180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589243904:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619930.401709:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1039390 00010000:00010000:3.0:1788619930.401726:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6600/0x1cd7fe5368736792 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d870 expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.401746:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619930.401749:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619930.401755:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6600/0x1cd7fe5368736792 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d870 expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.401769:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d6600, rc 0) 00000100:00100000:0.0:1788619930.401915:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571685248 00000100:00100000:0.0:1788619930.401922:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571685248 00000100:00100000:3.0:1788619930.402045:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df96aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2353:x1875503571685120:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2858us (3250us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619930.402054:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1496 00000100:00100000:0.0:1788619930.402111:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1229 00000100:00100000:0.0:1788619930.402121:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df969c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571685248:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619930.402135:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619930.402153:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025706efa/0x1cd7fe5368736799 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d877 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619930.402208:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589244032:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619930.402258:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5eaf300 going to sleep for 11 seconds 00010000:00010000:2.0:1788619930.403298:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:0.0:1788619930.405273:0:7654:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619929 -> 1788619930 00000100:00100000:0.0:1788619930.405580:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589244032:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619930.405614:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000025706efa/0x1cd7fe5368736799 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d877 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.405639:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025706efa/0x1cd7fe5368736799 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d877 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.405654:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d5400, rc 0) 00000100:00100000:0.0:1788619930.405702:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df969c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571685248:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3582us (6688us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619930.405710:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1229 00000100:00100000:2.0:1788619930.408234:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619930.408273:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619930.408424:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571685504 00000100:00100000:3.0:1788619930.408434:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571685504 00000100:00100000:3.0:1788619930.408492:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1230 00000100:00100000:3.0:1788619930.408503:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df969180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571685504:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619930.408520:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619930.408537:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e3d6efb/0x1cd7fe53687367a0 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d885 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619930.408611:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f1180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589244160:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619930.408685:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48ccc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619930.410653:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f1180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589244160:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619930.410690:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009e3d6efb/0x1cd7fe53687367a0 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d885 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.410717:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e3d6efb/0x1cd7fe53687367a0 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d885 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.410730:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d7600, rc 0) 00000100:00100000:3.0:1788619930.410787:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df969180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571685504:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2285us (2561us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619930.410796:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1230 00000100:00100000:0.0:1788619930.412515:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571685376 00000100:00100000:0.0:1788619930.412529:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571685376 00000100:00100000:0.0:1788619930.412636:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1497 00000100:00100000:0.0:1788619930.412646:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df968700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2352:x1875503571685376:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619930.412667:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619930.412688:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5600/0x1cd7fe53687367a7 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d87e expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.412726:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687366f1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7cf expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619930.412772:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c5180 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589244288:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619930.412857:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5eaf300 going to sleep for 11 seconds 00000100:00100000:0.0:1788619930.415796:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c5180 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589244288:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619930.415884:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1039390 00010000:00010000:0.0:1788619930.416018:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5600/0x1cd7fe53687367a7 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d87e expref: 16 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.416041:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619930.416044:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619930.416050:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5600/0x1cd7fe53687367a7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d87e expref: 16 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.416065:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d5600, rc 0) 00000100:00100000:0.0:1788619930.416164:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df968700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2352:x1875503571685376:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3518us (7888us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619930.416173:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1497 00000100:00100000:1.0:1788619930.419529:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619930.419582:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619930.419649:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571685632 00000100:00100000:0.0:1788619930.419659:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571685632 00000100:00100000:0.0:1788619930.419717:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1498 00000100:00100000:0.0:1788619930.419728:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2353:x1875503571685632:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619930.419749:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619930.419769:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5400/0x1cd7fe53687367ae lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d88c expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.419807:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687366f1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7cf expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619930.419859:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c5180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589244416:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619930.419952:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5eaf300 going to sleep for 11 seconds 00000100:00100000:3.0:1788619930.420520:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571685760 00000100:00100000:3.0:1788619930.420527:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571685760 00000100:00100000:3.0:1788619930.420575:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1231 00000100:00100000:3.0:1788619930.420584:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571685760:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619930.420594:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619930.420605:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003566e5e5/0x1cd7fe53687367b5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d893 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619930.420643:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f1180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589244544:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619930.420683:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48ccc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619930.423706:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c5180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589244416:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619930.423732:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1039390 00010000:00010000:0.0:1788619930.423751:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5400/0x1cd7fe53687367ae lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d88c expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.423772:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619930.423775:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619930.423781:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5400/0x1cd7fe53687367ae lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d88c expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.423797:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d5400, rc 0) 00000100:00100000:3.0:1788619930.423942:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f1180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589244544:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619930.423959:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003566e5e5/0x1cd7fe53687367b5 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d893 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.423974:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003566e5e5/0x1cd7fe53687367b5 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d893 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.423982:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d6600, rc 0) 00000100:00100000:0.0:1788619930.424021:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2353:x1875503571685632:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4295us (4501us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619930.424030:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1498 00000100:00100000:3.0:1788619930.424592:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571685760:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4008us (5008us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619930.424600:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1231 00000100:00100000:2.0:1788619930.427594:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619930.427693:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571685888 00000100:00100000:0.0:1788619930.427704:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571685888 00000100:00100000:0.0:1788619930.427759:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1232 00000100:00100000:0.0:1788619930.427769:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df96ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:14908:x1875503571685888:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619930.427786:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619930.427803:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000854192d0/0x1cd7fe53687367bc lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d89a expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.427986:0:6696:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000fd10f4fd/0x1cd7fe536873678b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1056767) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d869 expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.427997:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(7 12) 00010000:00010000:0.0:1788619930.428001:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(8 12) 00010000:00010000:0.0:1788619930.428007:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000854192d0/0x1cd7fe53687367bc lrc: 4/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1466368->1470463] (req 1466368->1470463) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d89a expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.428042:0:6696:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000fd10f4fd/0x1cd7fe536873678b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1056767) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d869 expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.428055:0:6696:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000fd10f4fd/0x1cd7fe536873678b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1056767) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d869 expref: 6 pid: 6696 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619930.428071:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589244672:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619930.428204:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5eaf300 going to sleep for 11 seconds 00000100:00100000:2.0:1788619930.430827:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619930.431094:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571686016 00000100:00100000:1.0:1788619930.431103:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571686016 00010000:00010000:1.0:1788619930.431140:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000fd10f4fd/0x1cd7fe536873678b lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1056767) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d869 expref: 7 pid: 6696 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619930.431184:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 492 00000100:00100000:1.0:1788619930.431194:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df96b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571686016:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619930.431208:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619930.431229:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000fd10f4fd/0x1cd7fe536873678b lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1056767) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d869 expref: 7 pid: 6696 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.431243:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000fd10f4fd/0x1cd7fe536873678b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1056767) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d869 expref: 7 pid: 6696 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.431266:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(8 12) 00010000:00010000:1.0:1788619930.431270:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(8 12) 00010000:00010000:1.0:1788619930.431279:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000854192d0/0x1cd7fe53687367bc lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1466368->1470463) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d89a expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.431289:0:5831:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000854192d0/0x1cd7fe53687367bc lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1466368->1470463) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d89a expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.431322:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000854192d0/0x1cd7fe53687367bc lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1466368->1470463) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d89a expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619930.431507:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4ba3480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5831:1875503589244800:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619930.431587:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c85903c0 going to sleep for 11 seconds 00010000:00010000:0.0:1788619930.431794:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fd10f4fd/0x1cd7fe536873678b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1056767) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d869 expref: 7 pid: 6696 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619930.431811:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589244672:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619930.431841:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000854192d0/0x1cd7fe53687367bc lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1466368->1470463) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d89a expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.431855:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d5600, rc 0) 00000100:00100000:0.0:1788619930.431895:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df96ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:14908:x1875503571685888:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4127us (4311us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619930.432087:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1232 00000100:00100000:1.0:1788619930.435096:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619930.435220:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571686144 00000100:00100000:0.0:1788619930.435228:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571686144 00002000:00100000:0.0:1788619930.435272:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1466368->1470463) req@ffff8e77c4ba2a00 x1875503571686144/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/0 e 0 to 0 dl 1788619941 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619930.435284:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4ba2a00 with x1875503571686144 ext(1466368->1470463) 00002000:00010000:0.0:1788619930.435290:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4ba2a00 00000100:00100000:0.0:1788619930.435320:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 699 00000100:00100000:0.0:1788619930.435361:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571686144:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619930.435617:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571686144:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 257us (528us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619930.435625:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 699 00002000:00100000:0.0:1788619930.435633:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1466368->1470463) req@ffff8e77c4ba2a00 x1875503571686144/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/4536 e 0 to 0 dl 1788619941 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619930.435641:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4ba2a00 with x1875503571686144 ext(1466368->1470463) 00002000:00010000:0.0:1788619930.435645:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4ba2a00 00000100:00100000:3.0:1788619930.437444:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4ba3480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5831:1875503589244800:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619930.437471:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619930.437525:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df96b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571686016:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 6331us (6716us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619930.437534:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 492 00000100:00100000:1.0:1788619930.442280:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619930.442644:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571686272 00000100:00100000:0.0:1788619930.442654:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571686272 00002000:00100000:0.0:1788619930.442705:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1470464->1474559) req@ffff8e77c4ba3800 x1875503571686272/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/0 e 0 to 0 dl 1788619941 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619930.442716:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4ba3800 with x1875503571686272 ext(1470464->1474559) 00002000:00010000:0.0:1788619930.442723:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4ba3800 00000100:00100000:0.0:1788619930.442751:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 700 00000100:00100000:0.0:1788619930.442761:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571686272:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619930.443066:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571686272:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 305us (796us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619930.443074:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 700 00002000:00100000:0.0:1788619930.443081:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1470464->1474559) req@ffff8e77c4ba3800 x1875503571686272/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/4536 e 0 to 0 dl 1788619941 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619930.443090:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4ba3800 with x1875503571686272 ext(1470464->1474559) 00002000:00010000:0.0:1788619930.443094:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4ba3800 00000100:00100000:0.0:1788619930.447182:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619930.447268:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571686400 00000100:00100000:1.0:1788619930.447276:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571686400 00002000:00100000:1.0:1788619930.447316:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1474560->1478655) req@ffff8e77c50c5180 x1875503571686400/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/0 e 0 to 0 dl 1788619941 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619930.447351:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c50c5180 with x1875503571686400 ext(1474560->1478655) 00002000:00010000:1.0:1788619930.447356:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c50c5180 00000100:00100000:1.0:1788619930.447384:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 701 00000100:00100000:1.0:1788619930.447393:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571686400:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619930.447623:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571686400:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 230us (450us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619930.447631:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 701 00002000:00100000:1.0:1788619930.447637:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1474560->1478655) req@ffff8e77c50c5180 x1875503571686400/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/4536 e 0 to 0 dl 1788619941 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619930.447646:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c50c5180 with x1875503571686400 ext(1474560->1478655) 00002000:00010000:1.0:1788619930.447649:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c50c5180 00000100:00100000:0.0:1788619930.450238:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619930.450360:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571686528 00000100:00100000:1.0:1788619930.450369:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571686528 00002000:00100000:1.0:1788619930.450511:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1478656->1482751) req@ffff8e77c50c5f80 x1875503571686528/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/0 e 0 to 0 dl 1788619941 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619930.450528:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c50c5f80 with x1875503571686528 ext(1478656->1482751) 00002000:00010000:1.0:1788619930.450534:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c50c5f80 00000100:00100000:1.0:1788619930.450572:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 702 00000100:00100000:1.0:1788619930.450581:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571686528:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619930.450832:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571686528:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 251us (604us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619930.450840:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 702 00002000:00100000:1.0:1788619930.450847:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1478656->1482751) req@ffff8e77c50c5f80 x1875503571686528/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/4536 e 0 to 0 dl 1788619941 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619930.450854:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c50c5f80 with x1875503571686528 ext(1478656->1482751) 00002000:00010000:1.0:1788619930.450857:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c50c5f80 00000100:00100000:2.0:1788619930.454133:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619930.454233:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571686656 00000100:00100000:0.0:1788619930.454242:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571686656 00002000:00100000:0.0:1788619930.454292:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1482752->1486847) req@ffff8e78c5c30380 x1875503571686656/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/0 e 0 to 0 dl 1788619941 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619930.454303:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5c30380 with x1875503571686656 ext(1482752->1486847) 00002000:00010000:0.0:1788619930.454309:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5c30380 00000100:00100000:0.0:1788619930.454379:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 703 00000100:00100000:0.0:1788619930.454389:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c30380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571686656:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619930.454647:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c30380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571686656:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 258us (523us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619930.454655:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 703 00002000:00100000:0.0:1788619930.454662:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1482752->1486847) req@ffff8e78c5c30380 x1875503571686656/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/4536 e 0 to 0 dl 1788619941 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619930.454671:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5c30380 with x1875503571686656 ext(1482752->1486847) 00002000:00010000:0.0:1788619930.454675:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5c30380 00000100:00100000:2.0:1788619930.457652:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619930.457800:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571686784 00000100:00100000:0.0:1788619930.457810:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571686784 00002000:00100000:0.0:1788619930.457866:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1486848->1490943) req@ffff8e78c5c32d80 x1875503571686784/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/0 e 0 to 0 dl 1788619941 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619930.457877:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5c32d80 with x1875503571686784 ext(1486848->1490943) 00002000:00010000:0.0:1788619930.457882:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5c32d80 00000100:00100000:0.0:1788619930.457918:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 704 00000100:00100000:0.0:1788619930.457928:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571686784:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619930.458298:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571686784:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 370us (653us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619930.458307:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 704 00002000:00100000:0.0:1788619930.458315:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1486848->1490943) req@ffff8e78c5c32d80 x1875503571686784/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/4536 e 0 to 0 dl 1788619941 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619930.458361:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5c32d80 with x1875503571686784 ext(1486848->1490943) 00002000:00010000:0.0:1788619930.458366:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5c32d80 00000100:00100000:2.0:1788619930.462151:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619930.462246:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571686912 00000100:00100000:3.0:1788619930.462256:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571686912 00000100:00100000:1.0:1788619930.462570:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1499 00000100:00100000:1.0:1788619930.462583:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c32a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2353:x1875503571686912:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619930.462605:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619930.462625:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe53687367c3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d8a1 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.462662:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687366f1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7cf expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619930.462703:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4ba3480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589244928:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619930.462832:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590240 going to sleep for 11 seconds 00000100:00100000:1.0:1788619930.464672:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4ba3480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589244928:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619930.464698:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1039390 00010000:00010000:1.0:1788619930.464717:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe53687367c3 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8a1 expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.464738:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619930.464742:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619930.464747:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe53687367c3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8a1 expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.464764:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683dc00, rc 0) 00000100:00100000:1.0:1788619930.464899:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c32a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2353:x1875503571686912:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2318us (2755us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619930.464908:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1499 00000100:00100000:2.0:1788619930.467448:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619930.467540:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571687040 00000100:00100000:0.0:1788619930.467558:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571687040 00000100:00100000:0.0:1788619930.467646:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1500 00000100:00100000:0.0:1788619930.467657:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c33800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2353:x1875503571687040:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619930.467676:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619930.467694:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5800/0x1cd7fe53687367ca lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d8a8 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.467730:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687366f1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7cf expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619930.467774:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c5c00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589245056:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619930.467854:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5eafe40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619930.469797:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c5c00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589245056:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619930.469832:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1039390 00010000:00010000:0.0:1788619930.469851:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5800/0x1cd7fe53687367ca lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8a8 expref: 16 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.469868:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619930.469871:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619930.469876:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5800/0x1cd7fe53687367ca lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8a8 expref: 16 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.469890:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d5800, rc 0) 00000100:00100000:0.0:1788619930.469954:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c33800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2353:x1875503571687040:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2299us (2628us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619930.469963:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1500 00000100:00100000:2.0:1788619930.472316:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619930.472430:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571687168 00000100:00100000:3.0:1788619930.472439:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571687168 00000100:00100000:3.0:1788619930.472495:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1233 00000100:00100000:3.0:1788619930.472505:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c31c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571687168:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619930.472522:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619930.472539:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003566e5e5/0x1cd7fe53687367d1 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d8af expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.472581:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(8 12) 00010000:00010000:3.0:1788619930.472586:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(8 12) 00010000:00010000:3.0:1788619930.472596:0:7654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003566e5e5/0x1cd7fe53687367d1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8af expref: 7 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.472614:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003566e5e5/0x1cd7fe53687367d1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8af expref: 7 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.472626:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d6600, rc 0) 00000100:00100000:3.0:1788619930.472676:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c31c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+7:2352:x1875503571687168:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 173us (370us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619930.472683:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1233 00000100:00100000:2.0:1788619930.475865:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619930.476141:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571687296 00000100:00100000:1.0:1788619930.476152:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571687296 00000100:00100000:1.0:1788619930.476202:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1501 00000100:00100000:1.0:1788619930.476211:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c33480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2353:x1875503571687296:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619930.476232:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619930.476253:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687367d8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d8b6 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.476289:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687366f1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7cf expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619930.476448:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4ba3480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589245184:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619930.476574:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590240 going to sleep for 11 seconds 00000100:00100000:1.0:1788619930.478989:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4ba3480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589245184:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619930.479017:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1039390 00010000:00010000:1.0:1788619930.479035:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687367d8 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8b6 expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.479055:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619930.479058:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619930.479064:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687367d8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8b6 expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.479080:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d600, rc 0) 00000100:00100000:1.0:1788619930.479179:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c33480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2353:x1875503571687296:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2967us (3322us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619930.479188:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1501 00000100:00100000:2.0:1788619930.482116:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619930.482201:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571687424 00000100:00100000:1.0:1788619930.482210:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571687424 00000100:00100000:1.0:1788619930.482307:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1502 00000100:00100000:1.0:1788619930.482318:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c30700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2352:x1875503571687424:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619930.482363:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619930.482391:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe53687367df lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d8bd expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.482428:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687366f1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7cf expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619930.482469:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4ba3480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589245312:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619930.482555:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590240 going to sleep for 11 seconds 00000100:00100000:1.0:1788619930.485131:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4ba3480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589245312:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619930.485156:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1039390 00010000:00010000:1.0:1788619930.485174:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe53687367df lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8bd expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.485194:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619930.485197:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619930.485202:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe53687367df lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8bd expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.485216:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683dc00, rc 0) 00000100:00100000:1.0:1788619930.485310:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c30700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2352:x1875503571687424:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2993us (3201us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619930.485319:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1502 00000100:00100000:2.0:1788619930.488320:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619930.488660:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571687552 00000100:00100000:0.0:1788619930.488669:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571687552 00000100:00100000:0.0:1788619930.488731:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1503 00000100:00100000:0.0:1788619930.488742:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c30e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2353:x1875503571687552:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619930.488760:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619930.488782:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5800/0x1cd7fe53687367e6 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d8c4 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.488829:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687366f1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7cf expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619930.488870:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c5c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589245440:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619930.488962:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5eafe40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619930.491145:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c5c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589245440:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619930.491169:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1039390 00010000:00010000:0.0:1788619930.491189:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5800/0x1cd7fe53687367e6 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8c4 expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.491211:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619930.491214:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619930.491220:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d5800/0x1cd7fe53687367e6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8c4 expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.491235:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d5800, rc 0) 00000100:00100000:0.0:1788619930.491292:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c30e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2353:x1875503571687552:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2552us (2982us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619930.491300:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1503 00000100:00100000:3.0:1788619930.493936:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619930.494024:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571687680 00000100:00100000:0.0:1788619930.494034:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571687680 00002000:00100000:0.0:1788619930.494086:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1081344->1085439) req@ffff8e78e67f1180 x1875503571687680/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/0 e 0 to 0 dl 1788619941 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619930.494097:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78e67f1180 with x1875503571687680 ext(1081344->1085439) 00002000:00010000:0.0:1788619930.494102:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78e67f1180 00000100:00100000:0.0:1788619930.494134:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 705 00000100:00100000:0.0:1788619930.494143:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571687680:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619930.494423:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571687680:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 280us (495us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619930.494431:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 705 00002000:00100000:0.0:1788619930.494438:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1081344->1085439) req@ffff8e78e67f1180 x1875503571687680/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/4536 e 0 to 0 dl 1788619941 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619930.494445:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78e67f1180 with x1875503571687680 ext(1081344->1085439) 00002000:00010000:0.0:1788619930.494448:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78e67f1180 00000100:00100000:2.0:1788619930.498233:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619930.498317:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571687808 00000100:00100000:0.0:1788619930.498351:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571687808 00002000:00100000:0.0:1788619930.498413:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1085440->1089535) req@ffff8e78c5c33100 x1875503571687808/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/0 e 0 to 0 dl 1788619941 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619930.498424:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5c33100 with x1875503571687808 ext(1085440->1089535) 00002000:00010000:0.0:1788619930.498429:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5c33100 00000100:00100000:0.0:1788619930.498463:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 706 00000100:00100000:0.0:1788619930.498473:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571687808:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619930.498729:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571687808:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 256us (505us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619930.498738:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 706 00002000:00100000:0.0:1788619930.498745:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1085440->1089535) req@ffff8e78c5c33100 x1875503571687808/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/4536 e 0 to 0 dl 1788619941 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619930.498754:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5c33100 with x1875503571687808 ext(1085440->1089535) 00002000:00010000:0.0:1788619930.498757:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5c33100 00000100:00100000:2.0:1788619930.502545:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619930.502693:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571687936 00000100:00100000:3.0:1788619930.502704:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571687936 00002000:00100000:3.0:1788619930.502759:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1089536->1093631) req@ffff8e78c5c30a80 x1875503571687936/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/0 e 0 to 0 dl 1788619941 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619930.502770:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5c30a80 with x1875503571687936 ext(1089536->1093631) 00002000:00010000:3.0:1788619930.502775:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5c30a80 00000100:00100000:3.0:1788619930.502802:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 707 00000100:00100000:3.0:1788619930.502812:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c30a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571687936:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619930.503133:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c30a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571687936:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 321us (596us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619930.503140:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 707 00002000:00100000:3.0:1788619930.503147:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1089536->1093631) req@ffff8e78c5c30a80 x1875503571687936/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/4536 e 0 to 0 dl 1788619941 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619930.503154:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5c30a80 with x1875503571687936 ext(1089536->1093631) 00002000:00010000:3.0:1788619930.503157:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5c30a80 00000100:00100000:3.0:1788619930.506093:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619930.506293:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571688064 00000100:00100000:1.0:1788619930.506303:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571688064 00002000:00100000:1.0:1788619930.506391:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1093632->1097727) req@ffff8e78e67f3b80 x1875503571688064/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/0 e 0 to 0 dl 1788619941 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619930.506403:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78e67f3b80 with x1875503571688064 ext(1093632->1097727) 00002000:00010000:1.0:1788619930.506409:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78e67f3b80 00000100:00100000:1.0:1788619930.506435:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 708 00000100:00100000:1.0:1788619930.506445:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571688064:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619930.506715:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571688064:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 269us (628us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619930.506722:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 708 00002000:00100000:1.0:1788619930.506729:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1093632->1097727) req@ffff8e78e67f3b80 x1875503571688064/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/4536 e 0 to 0 dl 1788619941 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619930.506742:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78e67f3b80 with x1875503571688064 ext(1093632->1097727) 00002000:00010000:1.0:1788619930.506744:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78e67f3b80 00000100:00100000:2.0:1788619930.511831:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619930.511977:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571688320 00000100:00100000:1.0:1788619930.511988:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571688320 00000100:00100000:1.0:1788619930.512050:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1504 00000100:00100000:1.0:1788619930.512061:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c30000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:14908:x1875503571688320:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:1.0:1788619930.512115:0:9417:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619930.512145:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d4400/0x1cd7fe5368736681 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d75f expref: 15 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.512166:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619930.512171:0:9417:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788619930.512446:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe53687367ed lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.512462:0:9417:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6c00/0x1cd7fe5368736673 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d758 expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.512471:0:9417:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe53687367ed lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.512500:0:9417:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6c00/0x1cd7fe5368736673 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d758 expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.512513:0:9417:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6c00/0x1cd7fe5368736673 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d758 expref: 12 pid: 5846 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619930.512531:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4ba3480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589245568:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619930.512615:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590780 going to sleep for 11 seconds 00000100:00100000:2.0:1788619930.515611:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619930.515827:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4ba3480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589245568:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619930.515857:0:9417:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe53687367ed lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619930.515930:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571688448 00000100:00100000:3.0:1788619930.515937:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571688448 00010000:00010000:3.0:1788619930.515973:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6c00/0x1cd7fe5368736673 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d758 expref: 13 pid: 5846 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619930.516003:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 493 00000100:00100000:3.0:1788619930.516012:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c31180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571688448:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619930.516027:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619930.516034:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6c00/0x1cd7fe5368736673 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d758 expref: 13 pid: 5846 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.516046:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6c00/0x1cd7fe5368736673 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d758 expref: 13 pid: 5846 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.516069:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6c00/0x1cd7fe5368736673 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d758 expref: 13 pid: 5846 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.516085:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619930.516091:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe53687367ed lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.516101:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe53687367ed lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.516110:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe53687367ed lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.516139:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619930.516191:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c31180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571688448:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 179us (589us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619930.516198:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 493 00010000:00010000:1.0:1788619930.516300:0:9417:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe53687367ed lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.516308:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe53687367ed lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788619930.516450:0:9417:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967819 00000004:00010000:1.0:1788619930.516500:0:9417:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1039390 -> 1039390 00000004:00010000:1.0:1788619930.516504:0:9417:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619930 -> 1788619930 00000004:00010000:1.0:1788619930.516507:0:9417:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619930 -> 1788619928 00000004:00010000:1.0:1788619930.516509:0:9417:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619930 -> 1788619930 00000004:00010000:1.0:1788619930.516511:0:9417:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1672 -> 1672 00000004:00010000:1.0:1788619930.516519:0:9417:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000000a5d1c7a reply state 0000000080e08704 transno 4294967820 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe53687367ed lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619930.516528:0:9417:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788619930.516607:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c30000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+17:14908:x1875503571688320:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 4547us (4795us total) trans 4294967820 rc 0/0 00000100:00100000:1.0:1788619930.516614:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1504 00000020:00080000:1.0:1788619930.516618:0:9417:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1788619930.522040:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619930.522116:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571688576 00000100:00100000:1.0:1788619930.522130:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571688576 00000100:00100000:1.0:1788619930.522194:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1505 00000100:00100000:1.0:1788619930.522205:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+17:14908:x1875503571688576:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619930.522231:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619930.522252:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687367f4 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d8cb expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.522278:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687366f1 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7cf expref: 11 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.522288:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687367f4 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8cb expref: 18 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.522320:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687366f1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7cf expref: 11 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.522361:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687366f1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7cf expref: 11 pid: 9417 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619930.522375:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4ba3480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589245696:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619930.522452:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590780 going to sleep for 11 seconds 00000100:00100000:1.0:1788619930.524575:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4ba3480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589245696:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619930.524611:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687367f4 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8cb expref: 18 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.524627:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619930.524631:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619930.524635:0:5844:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687367f4 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8cb expref: 18 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.524645:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d600, rc 0) 00000100:00100000:1.0:1788619930.524697:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+18:14908:x1875503571688576:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2494us (2669us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619930.524707:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1505 00010000:00010000:2.0:1788619930.524926:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe53687367ed lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.524934:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe53687367ed lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.524938:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe53687367ed lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.524942:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c683dc00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:2.0:1788619930.524945:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe53687367ed lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.524951:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe53687367ed lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.524962:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619930.524963:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788619930.524966:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe53687367ed lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.524970:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe53687367ed lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619930.526360:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619930.526476:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571688704 00000100:00100000:0.0:1788619930.526485:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571688704 00010000:00010000:0.0:1788619930.526520:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687366f1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7cf expref: 12 pid: 9417 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619930.526563:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 494 00000100:00100000:0.0:1788619930.526573:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcac050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571688704:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619930.526590:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619930.526617:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687366f1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7cf expref: 12 pid: 9417 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.526630:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687366f1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7cf expref: 12 pid: 9417 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.526648:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687366f1 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d7cf expref: 12 pid: 9417 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.526658:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619930.526662:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687367f4 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8cb expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.526669:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687367f4 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8cb expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.526675:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687367f4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8cb expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.526709:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687367f4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8cb expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619930.526723:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c4e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589245824:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619930.526802:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5eaf0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619930.529149:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c4e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589245824:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619930.529173:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619930.529232:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcac050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571688704:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 2658us (2875us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619930.529242:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 494 00000100:00100000:2.0:1788619930.531843:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619930.531970:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571688832 00000100:00100000:3.0:1788619930.531978:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571688832 00000100:00100000:3.0:1788619930.532025:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1234 00000100:00100000:3.0:1788619930.532032:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:14908:x1875503571688832:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619930.532047:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619930.532062:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000614d4133/0x1cd7fe53687367fb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d8d2 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.532086:0:6698:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000854192d0/0x1cd7fe53687367bc lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1466368->1470463) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d89a expref: 7 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.532093:0:6698:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000003566e5e5/0x1cd7fe53687367d1 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8af expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.532098:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(8 12) 00010000:00010000:3.0:1788619930.532101:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(8 12) 00010000:00010000:3.0:1788619930.532105:0:6698:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000614d4133/0x1cd7fe53687367fb lrc: 5/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8d2 expref: 7 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.532133:0:6698:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000003566e5e5/0x1cd7fe53687367d1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8af expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.532142:0:6698:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000003566e5e5/0x1cd7fe53687367d1 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8af expref: 6 pid: 7654 timeout: 523 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619930.532152:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f1c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589245952:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619930.532245:0:6698:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000854192d0/0x1cd7fe53687367bc lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1466368->1470463) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d89a expref: 7 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.532256:0:6698:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000854192d0/0x1cd7fe53687367bc lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1466368->1470463) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d89a expref: 7 pid: 6696 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619930.532266:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589246080:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619930.532291:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48ce40 going to sleep for 11 seconds 00000100:00100000:2.0:1788619930.536083:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619930.536108:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619930.536159:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571688960 00000100:00100000:0.0:1788619930.536166:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571688960 00010000:00010000:0.0:1788619930.536198:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000003566e5e5/0x1cd7fe53687367d1 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8af expref: 7 pid: 7654 timeout: 523 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619930.536237:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571689088 00000100:00100000:0.0:1788619930.536241:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571689088 00010000:00010000:0.0:1788619930.536254:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000854192d0/0x1cd7fe53687367bc lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1466368->1470463) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d89a expref: 8 pid: 6696 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619930.536275:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 495 00000100:00100000:0.0:1788619930.536285:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c31f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571688960:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619930.536301:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619930.536323:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000003566e5e5/0x1cd7fe53687367d1 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8af expref: 7 pid: 7654 timeout: 523 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.536348:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000003566e5e5/0x1cd7fe53687367d1 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8af expref: 7 pid: 7654 timeout: 523 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.536368:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619930.536425:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c31f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571688960:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 141us (349us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619930.536433:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 495 00000100:00100000:0.0:1788619930.536450:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 496 00000100:00100000:0.0:1788619930.536456:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c33b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571689088:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619930.536463:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619930.536469:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000854192d0/0x1cd7fe53687367bc lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1466368->1470463) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d89a expref: 8 pid: 6696 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.536476:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000854192d0/0x1cd7fe53687367bc lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1466368->1470463) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d89a expref: 8 pid: 6696 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.536485:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(8 12) 00010000:00010000:0.0:1788619930.536489:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(8 12) 00010000:00010000:0.0:1788619930.536495:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000614d4133/0x1cd7fe53687367fb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8d2 expref: 8 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.536502:0:13330:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000614d4133/0x1cd7fe53687367fb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8d2 expref: 8 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.536526:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000614d4133/0x1cd7fe53687367fb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8d2 expref: 8 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619930.536535:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c7480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589246208:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619930.536595:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5eaf300 going to sleep for 11 seconds 00010000:00010000:3.0:1788619930.536625:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003566e5e5/0x1cd7fe53687367d1 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8af expref: 6 pid: 7654 timeout: 523 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619930.536640:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f1c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589245952:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619930.536652:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000854192d0/0x1cd7fe53687367bc lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1466368->1470463) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d89a expref: 8 pid: 6696 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619930.536666:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589246080:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619930.536700:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000614d4133/0x1cd7fe53687367fb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8d2 expref: 7 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.536717:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d6c00, rc 0) 00000100:00100000:3.0:1788619930.536751:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:14908:x1875503571688832:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4718us (4918us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619930.536758:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1234 00000100:00100000:2.0:1788619930.540529:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619930.540577:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619930.540601:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571689344 00000100:00100000:1.0:1788619930.540610:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571689344 00002000:00010000:1.0:1788619930.540649:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:1.0:1788619930.540655:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5c32680 with x1875503571689344 ext(0->18446744073709551615) 00002000:00010000:1.0:1788619930.540668:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5c32680. 00000100:00100000:1.0:1788619930.540827:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 709 00000100:00100000:1.0:1788619930.540838:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571689344:12345-192.168.201.5@tcp:10:fsx.0 00000100:00100000:0.0:1788619930.540891:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c7480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589246208:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619930.540918:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619930.540965:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c33b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571689088:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 4508us (4853us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619930.540973:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 496 00002000:00010000:1.0:1788619930.541143:0:6702:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1998533 -> 0 00002000:00010000:1.0:1788619930.541147:0:6702:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619930 -> 1788619930 00002000:00010000:1.0:1788619930.541150:0:6702:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619930 -> 1788619929 00002000:00010000:1.0:1788619930.541153:0:6702:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619930 -> 1788619930 00002000:00010000:1.0:1788619930.541155:0:6702:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 391 -> 391 00000100:00100000:1.0:1788619930.541180:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571689344:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 343us (658us total) trans 4294967508 rc 0/0 00000100:00100000:3.0:1788619930.541187:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571689216 00000100:00100000:1.0:1788619930.541187:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 709 00002000:00010000:1.0:1788619930.541190:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00000100:00100000:3.0:1788619930.541191:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571689216 00002000:00010000:3.0:1788619930.541211:0:5857:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:3.0:1788619930.541216:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5c31500 with x1875503571689216 00000100:00100000:3.0:1788619930.541244:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 907 00000100:00100000:3.0:1788619930.541251:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c31500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2355:x1875503571689216:12345-192.168.201.5@tcp:10:fsx.0 00002000:00100000:1.0:1788619930.541309:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5c32680 with x1875503571689344 ext(0->18446744073709551615) 00002000:00010000:1.0:1788619930.541318:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5c32680. 00000004:00010000:3.0:1788619930.541500:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1039390 -> 828664 00000004:00010000:3.0:1788619930.541503:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619930 -> 1788619930 00000004:00010000:3.0:1788619930.541504:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619928 -> 1788619928 00000004:00010000:3.0:1788619930.541506:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619930 -> 1788619930 00000004:00010000:3.0:1788619930.541508:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1672 -> 1672 00000100:00100000:3.0:1788619930.541530:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c31500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2355:x1875503571689216:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 281us (955us total) trans 4294967821 rc 0/0 00000100:00100000:3.0:1788619930.541534:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 907 00002000:00010000:3.0:1788619930.541536:0:5857:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:3.0:1788619930.541538:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5c31500 with x1875503571689216 00000100:00100000:0.0:1788619930.543885:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619930.544167:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571689472 00000100:00100000:2.0:1788619930.544176:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571689472 00000100:00100000:2.0:1788619930.544230:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1506 00000100:00100000:2.0:1788619930.544240:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571689472:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619930.544263:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619930.544281:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51000/0x1cd7fe5368736802 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d8d9 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.544399:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51e00/0x1cd7fe5368736809 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.544415:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51e00/0x1cd7fe5368736809 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.544426:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51e00/0x1cd7fe5368736809 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788619930.544453:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 828664 00000004:00200000:2.0:1788619930.544457:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788619930.544471:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51e00/0x1cd7fe5368736809 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.544495:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51000/0x1cd7fe5368736802 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8d9 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.544507:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51e00/0x1cd7fe5368736809 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8d9 expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.544513:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619930.544515:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788619930.544518:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f51e00, rc 0) 00000100:00100000:2.0:1788619930.544623:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:14908:x1875503571689472:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 384us (747us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619930.544629:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1506 00000100:00100000:2.0:1788619930.546976:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619930.547085:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619930.547138:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571689728 00000100:00100000:3.0:1788619930.547145:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571689728 00000100:00100000:3.0:1788619930.547184:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1235 00000100:00100000:3.0:1788619930.547191:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fed61180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571689728:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619930.547210:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619930.547224:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003566e5e5/0x1cd7fe5368736810 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d8e7 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619930.547269:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589246336:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619930.547351:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48ce40 going to sleep for 11 seconds 00000100:00100000:1.0:1788619930.547448:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571689600 00000100:00100000:1.0:1788619930.547453:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571689600 00000100:00100000:1.0:1788619930.547490:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1507 00000100:00100000:1.0:1788619930.547499:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571689600:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619930.547510:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619930.547523:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe5368736817 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d8e0 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.547569:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687367f4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8cb expref: 17 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619930.547594:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4ba3480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589246464:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619930.547636:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590780 going to sleep for 11 seconds 00002000:00010000:3.0:1788619930.550322:0:7654:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619929 -> 1788619930 00000004:00010000:1.0:1788619930.550603:0:9417:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619928 -> 1788619930 00000100:00100000:1.0:1788619930.550628:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4ba3480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589246464:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619930.550656:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 828664 00000100:00100000:2.0:1788619930.550670:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589246336:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619930.550673:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe5368736817 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8e0 expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.550689:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003566e5e5/0x1cd7fe5368736810 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8e7 expref: 7 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.550709:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003566e5e5/0x1cd7fe5368736810 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8e7 expref: 7 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.550723:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d6600, rc 0) 00010000:00010000:1.0:1788619930.550810:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00000100:00100000:2.0:1788619930.550813:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fed61180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571689728:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3620us (3845us total) trans 0 rc 301/301 00010000:00010000:1.0:1788619930.550814:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619930.550819:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe5368736817 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8e0 expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.550832:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683da00, rc 0) 00000100:00100000:1.0:1788619930.550855:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571689600:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3358us (3770us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619930.550869:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1507 00000100:00100000:2.0:1788619930.550880:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1235 00000100:00100000:2.0:1788619930.553134:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619930.553172:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619930.553221:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571689984 00000100:00100000:0.0:1788619930.553230:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571689984 00000100:00100000:1.0:1788619930.553282:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571689856 00000100:00100000:1.0:1788619930.553287:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571689856 00000100:00100000:1.0:1788619930.553320:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1508 00000100:00100000:0.0:1788619930.553320:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1236 00000100:00100000:1.0:1788619930.553344:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df96a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2352:x1875503571689856:12345-192.168.201.5@tcp:101:fsx.0 00000100:00100000:0.0:1788619930.553344:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df96b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571689984:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619930.553360:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619930.553360:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619930.553382:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002919f738/0x1cd7fe536873681e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d8f5 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.553386:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368736825 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d8ee expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.553415:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687367f4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8cb expref: 17 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619930.553436:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4ba3480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589246720:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619930.553436:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c4000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589246592:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619930.553510:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5eaf300 going to sleep for 11 seconds 00000100:00100000:1.0:1788619930.553836:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590780 going to sleep for 11 seconds 00000100:00100000:1.0:1788619930.555855:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4ba3480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589246720:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619930.556495:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c4000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589246592:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619930.556526:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002919f738/0x1cd7fe536873681e lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8f5 expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.556562:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002919f738/0x1cd7fe536873681e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8f5 expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.556577:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d4600, rc 0) 00000100:00100000:0.0:1788619930.556673:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df96b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571689984:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3329us (3498us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619930.556682:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1236 00002000:00010000:1.0:1788619930.556815:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 828664 00010000:00010000:1.0:1788619930.556832:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368736825 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8ee expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.556849:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619930.556852:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619930.556857:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368736825 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8ee expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.556869:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683cc00, rc 0) 00000100:00100000:1.0:1788619930.556896:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df96a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2352:x1875503571689856:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3554us (3767us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619930.556903:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1508 00000100:00100000:0.0:1788619930.560568:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619930.560657:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571690240 00000100:00100000:2.0:1788619930.560667:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571690240 00000100:00100000:2.0:1788619930.560750:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1509 00000100:00100000:2.0:1788619930.560761:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+18:14908:x1875503571690240:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:2.0:1788619930.561049:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50a00/0x1cd7fe536873682c lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.561069:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51e00/0x1cd7fe5368736809 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8d9 expref: 11 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.561078:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50a00/0x1cd7fe536873682c lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.561118:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51e00/0x1cd7fe5368736809 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8d9 expref: 11 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.561133:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51e00/0x1cd7fe5368736809 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8d9 expref: 11 pid: 5845 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619930.561147:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6341880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589246848:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619930.561234:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164540 going to sleep for 11 seconds 00000100:00100000:3.0:1788619930.563354:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6341880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589246848:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619930.563467:0:5845:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50a00/0x1cd7fe536873682c lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619930.564759:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619930.564827:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571690368 00000100:00100000:3.0:1788619930.564835:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571690368 00010000:00010000:3.0:1788619930.564874:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51e00/0x1cd7fe5368736809 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8d9 expref: 12 pid: 5845 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619930.564924:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 497 00000100:00100000:3.0:1788619930.564934:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c6680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571690368:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619930.564949:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619930.564955:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51e00/0x1cd7fe5368736809 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8d9 expref: 12 pid: 5845 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.564974:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51e00/0x1cd7fe5368736809 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8d9 expref: 12 pid: 5845 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.564997:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51e00/0x1cd7fe5368736809 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8d9 expref: 12 pid: 5845 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.565014:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619930.565019:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50a00/0x1cd7fe536873682c lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.565029:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50a00/0x1cd7fe536873682c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.565037:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50a00/0x1cd7fe536873682c lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.565070:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619930.565123:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c6680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571690368:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 191us (369us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619930.565130:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 497 00010000:00010000:2.0:1788619930.565184:0:5845:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50a00/0x1cd7fe536873682c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.565192:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50a00/0x1cd7fe536873682c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788619930.565301:0:5845:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967822 00000004:00010000:2.0:1788619930.565403:0:5845:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 828664 -> 828664 00000004:00010000:2.0:1788619930.565407:0:5845:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619930 -> 1788619930 00000004:00010000:2.0:1788619930.565409:0:5845:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619930 -> 1788619928 00000004:00010000:2.0:1788619930.565411:0:5845:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619930 -> 1788619930 00000004:00010000:2.0:1788619930.565413:0:5845:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1672 -> 1672 00000004:00010000:2.0:1788619930.565421:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000010efb90e reply state 00000000d0bfe89d transno 4294967823 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50a00/0x1cd7fe536873682c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619930.565429:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788619930.565502:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+21:14908:x1875503571690240:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 4743us (4953us total) trans 4294967823 rc 0/0 00000100:00100000:2.0:1788619930.565510:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1509 00000020:00080000:2.0:1788619930.565514:0:5845:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:1.0:1788619930.570319:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50a00/0x1cd7fe536873682c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.570350:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50a00/0x1cd7fe536873682c lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.570358:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50a00/0x1cd7fe536873682c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.570363:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c5f50a00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:1.0:1788619930.570369:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50a00/0x1cd7fe536873682c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.570377:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50a00/0x1cd7fe536873682c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.570394:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619930.570396:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788619930.570401:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50a00/0x1cd7fe536873682c lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.570408:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50a00/0x1cd7fe536873682c lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619930.571299:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619930.571380:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571690496 00000100:00100000:3.0:1788619930.571393:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571690496 00002000:00010000:2.0:1788619930.571533:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1705687->18446744073709551615) 00002000:00100000:2.0:1788619930.571539:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4ba3480 with x1875503571690496 ext(1705687->18446744073709551615) 00002000:00010000:2.0:1788619930.571560:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4ba3480. 00000100:00100000:2.0:1788619930.571594:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 710 00000100:00100000:2.0:1788619930.571605:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571690496:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:2.0:1788619930.571821:0:6703:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 0 -> 1705687 00002000:00010000:2.0:1788619930.571825:0:6703:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619930 -> 1788619930 00002000:00010000:2.0:1788619930.571827:0:6703:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619930 -> 1788619929 00002000:00010000:2.0:1788619930.571830:0:6703:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619930 -> 1788619930 00002000:00010000:2.0:1788619930.571832:0:6703:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 391 -> 391 00000100:00100000:2.0:1788619930.571892:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571690496:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 288us (602us total) trans 4294967509 rc 0/0 00000100:00100000:2.0:1788619930.571900:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 710 00002000:00010000:2.0:1788619930.571915:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1705687->18446744073709551615) 00002000:00100000:2.0:1788619930.571917:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4ba3480 with x1875503571690496 ext(1705687->18446744073709551615) 00002000:00010000:2.0:1788619930.571922:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4ba3480. 00000100:00100000:0.0:1788619930.575809:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619930.576343:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571690624 00000100:00100000:1.0:1788619930.576354:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571690624 00002000:00100000:1.0:1788619930.576405:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1646592->1650687) req@ffff8e77c50c4a80 x1875503571690624/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/0 e 0 to 0 dl 1788619941 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619930.576424:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c50c4a80 with x1875503571690624 ext(1646592->1650687) 00002000:00010000:1.0:1788619930.576429:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c50c4a80 00000100:00100000:1.0:1788619930.576468:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 711 00000100:00100000:1.0:1788619930.576477:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571690624:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619930.576741:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571690624:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 264us (941us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619930.576749:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 711 00002000:00100000:1.0:1788619930.576756:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1646592->1650687) req@ffff8e77c50c4a80 x1875503571690624/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/4536 e 0 to 0 dl 1788619941 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619930.576763:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c50c4a80 with x1875503571690624 ext(1646592->1650687) 00002000:00010000:1.0:1788619930.576766:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c50c4a80 00000100:00100000:3.0:1788619930.579669:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619930.580011:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571690752 00000100:00100000:2.0:1788619930.580021:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571690752 00002000:00100000:2.0:1788619930.580065:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1650688->1654783) req@ffff8e78e67f1c00 x1875503571690752/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/0 e 0 to 0 dl 1788619941 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619930.580076:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78e67f1c00 with x1875503571690752 ext(1650688->1654783) 00002000:00010000:2.0:1788619930.580081:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78e67f1c00 00000100:00100000:2.0:1788619930.580117:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 712 00000100:00100000:2.0:1788619930.580126:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2352:x1875503571690752:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619930.580406:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2352:x1875503571690752:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 281us (747us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619930.580414:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 712 00002000:00100000:2.0:1788619930.580420:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1650688->1654783) req@ffff8e78e67f1c00 x1875503571690752/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/4536 e 0 to 0 dl 1788619941 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619930.580429:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78e67f1c00 with x1875503571690752 ext(1650688->1654783) 00002000:00010000:2.0:1788619930.580432:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78e67f1c00 00000100:00100000:3.0:1788619930.584124:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619930.584216:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571690880 00000100:00100000:1.0:1788619930.584226:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571690880 00002000:00100000:1.0:1788619930.584275:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1654784->1658879) req@ffff8e78e67f0380 x1875503571690880/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/0 e 0 to 0 dl 1788619941 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619930.584287:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78e67f0380 with x1875503571690880 ext(1654784->1658879) 00002000:00010000:1.0:1788619930.584292:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78e67f0380 00000100:00100000:1.0:1788619930.584358:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 713 00000100:00100000:1.0:1788619930.584380:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571690880:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619930.584670:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571690880:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 301us (555us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619930.584678:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 713 00002000:00100000:1.0:1788619930.584686:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1654784->1658879) req@ffff8e78e67f0380 x1875503571690880/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/4536 e 0 to 0 dl 1788619941 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619930.584695:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78e67f0380 with x1875503571690880 ext(1654784->1658879) 00002000:00010000:1.0:1788619930.584698:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78e67f0380 00000100:00100000:0.0:1788619930.587852:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619930.588183:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571691008 00000100:00100000:1.0:1788619930.588192:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571691008 00002000:00100000:1.0:1788619930.588240:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1658880->1662975) req@ffff8e77c50c4380 x1875503571691008/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/0 e 0 to 0 dl 1788619941 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619930.588253:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c50c4380 with x1875503571691008 ext(1658880->1662975) 00002000:00010000:1.0:1788619930.588258:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c50c4380 00000100:00100000:1.0:1788619930.588293:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 714 00000100:00100000:1.0:1788619930.588302:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2352:x1875503571691008:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619930.588612:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2352:x1875503571691008:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 310us (769us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619930.588619:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 714 00002000:00100000:1.0:1788619930.588626:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1658880->1662975) req@ffff8e77c50c4380 x1875503571691008/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/4536 e 0 to 0 dl 1788619941 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619930.588634:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c50c4380 with x1875503571691008 ext(1658880->1662975) 00002000:00010000:1.0:1788619930.588637:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c50c4380 00000100:00100000:3.0:1788619930.591975:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619930.592078:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571691136 00000100:00100000:2.0:1788619930.592087:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571691136 00002000:00100000:2.0:1788619930.592133:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1662976->1667071) req@ffff8e78e67f1880 x1875503571691136/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/0 e 0 to 0 dl 1788619941 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619930.592145:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78e67f1880 with x1875503571691136 ext(1662976->1667071) 00002000:00010000:2.0:1788619930.592149:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78e67f1880 00000100:00100000:2.0:1788619930.592184:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 715 00000100:00100000:2.0:1788619930.592194:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571691136:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619930.592478:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571691136:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 284us (510us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619930.592486:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 715 00002000:00100000:2.0:1788619930.592493:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1662976->1667071) req@ffff8e78e67f1880 x1875503571691136/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/4536 e 0 to 0 dl 1788619941 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619930.592502:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78e67f1880 with x1875503571691136 ext(1662976->1667071) 00002000:00010000:2.0:1788619930.592505:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78e67f1880 00000100:00100000:3.0:1788619930.595874:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619930.596495:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571691264 00000100:00100000:0.0:1788619930.596507:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571691264 00002000:00100000:0.0:1788619930.596576:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1667072->1671167) req@ffff8e78e67f3100 x1875503571691264/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/0 e 0 to 0 dl 1788619941 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619930.596587:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78e67f3100 with x1875503571691264 ext(1667072->1671167) 00002000:00010000:0.0:1788619930.596593:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78e67f3100 00000100:00100000:0.0:1788619930.596681:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 716 00000100:00100000:0.0:1788619930.596692:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571691264:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619930.597014:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571691264:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 323us (1149us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619930.597022:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 716 00002000:00100000:0.0:1788619930.597029:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1667072->1671167) req@ffff8e78e67f3100 x1875503571691264/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/4536 e 0 to 0 dl 1788619941 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619930.597037:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78e67f3100 with x1875503571691264 ext(1667072->1671167) 00002000:00010000:0.0:1788619930.597041:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78e67f3100 00000100:00100000:0.0:1788619930.600535:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619930.600632:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571691392 00000100:00100000:1.0:1788619930.600641:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571691392 00002000:00100000:1.0:1788619930.600689:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1671168->1675263) req@ffff8e77c50c6d80 x1875503571691392/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/0 e 0 to 0 dl 1788619941 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619930.600700:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c50c6d80 with x1875503571691392 ext(1671168->1675263) 00002000:00010000:1.0:1788619930.600707:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c50c6d80 00000100:00100000:1.0:1788619930.600740:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 717 00000100:00100000:1.0:1788619930.600750:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571691392:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619930.601019:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571691392:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 269us (491us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619930.601026:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 717 00002000:00100000:1.0:1788619930.601033:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1671168->1675263) req@ffff8e77c50c6d80 x1875503571691392/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/4536 e 0 to 0 dl 1788619941 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619930.601041:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c50c6d80 with x1875503571691392 ext(1671168->1675263) 00002000:00010000:1.0:1788619930.601044:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c50c6d80 00000100:00100000:3.0:1788619930.604026:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619930.604380:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571691520 00000100:00100000:0.0:1788619930.604391:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571691520 00002000:00100000:0.0:1788619930.604441:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1675264->1679359) req@ffff8e78e67f2d80 x1875503571691520/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/0 e 0 to 0 dl 1788619941 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619930.604452:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78e67f2d80 with x1875503571691520 ext(1675264->1679359) 00002000:00010000:0.0:1788619930.604458:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78e67f2d80 00000100:00100000:0.0:1788619930.604481:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 718 00000100:00100000:0.0:1788619930.604490:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571691520:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619930.604742:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571691520:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 252us (724us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619930.604749:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 718 00002000:00100000:0.0:1788619930.604756:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1675264->1679359) req@ffff8e78e67f2d80 x1875503571691520/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/4536 e 0 to 0 dl 1788619941 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619930.604764:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78e67f2d80 with x1875503571691520 ext(1675264->1679359) 00002000:00010000:0.0:1788619930.604767:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78e67f2d80 00000100:00100000:3.0:1788619930.608068:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619930.608155:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571691648 00000100:00100000:2.0:1788619930.608163:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571691648 00002000:00100000:2.0:1788619930.608207:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1679360->1683455) req@ffff8e77c6a89180 x1875503571691648/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/0 e 0 to 0 dl 1788619941 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619930.608217:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a89180 with x1875503571691648 ext(1679360->1683455) 00002000:00010000:2.0:1788619930.608222:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a89180 00000100:00100000:2.0:1788619930.608255:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 719 00000100:00100000:2.0:1788619930.608263:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571691648:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619930.608761:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a89180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571691648:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 495us (698us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619930.608769:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 719 00002000:00100000:2.0:1788619930.608777:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1679360->1683455) req@ffff8e77c6a89180 x1875503571691648/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/4536 e 0 to 0 dl 1788619941 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619930.608784:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a89180 with x1875503571691648 ext(1679360->1683455) 00002000:00010000:2.0:1788619930.608788:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a89180 00000100:00100000:2.0:1788619930.610855:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619930.611176:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571691776 00000100:00100000:1.0:1788619930.611184:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571691776 00002000:00100000:1.0:1788619930.611226:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1683456->1687551) req@ffff8e77c6342680 x1875503571691776/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/0 e 0 to 0 dl 1788619941 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619930.611237:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6342680 with x1875503571691776 ext(1683456->1687551) 00002000:00010000:1.0:1788619930.611242:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6342680 00000100:00100000:1.0:1788619930.611278:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 720 00000100:00100000:1.0:1788619930.611289:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6342680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571691776:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619930.611586:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6342680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571691776:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 297us (740us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619930.611593:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 720 00002000:00100000:1.0:1788619930.611599:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1683456->1687551) req@ffff8e77c6342680 x1875503571691776/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/4536 e 0 to 0 dl 1788619941 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619930.611607:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6342680 with x1875503571691776 ext(1683456->1687551) 00002000:00010000:1.0:1788619930.611611:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6342680 00000100:00100000:3.0:1788619930.615541:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619930.615654:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571691904 00000100:00100000:2.0:1788619930.615664:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571691904 00002000:00100000:2.0:1788619930.615714:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1687552->1691647) req@ffff8e77c6a8aa00 x1875503571691904/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/0 e 0 to 0 dl 1788619941 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619930.615726:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a8aa00 with x1875503571691904 ext(1687552->1691647) 00002000:00010000:2.0:1788619930.615732:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a8aa00 00000100:00100000:2.0:1788619930.615767:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 721 00000100:00100000:2.0:1788619930.615777:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a8aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571691904:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619930.616060:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a8aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571691904:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 284us (528us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619930.616069:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 721 00002000:00100000:2.0:1788619930.616075:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1687552->1691647) req@ffff8e77c6a8aa00 x1875503571691904/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/4536 e 0 to 0 dl 1788619941 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619930.616083:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a8aa00 with x1875503571691904 ext(1687552->1691647) 00002000:00010000:2.0:1788619930.616086:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a8aa00 00000100:00100000:3.0:1788619930.619277:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619930.619630:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571692032 00000100:00100000:0.0:1788619930.619641:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571692032 00002000:00100000:0.0:1788619930.619688:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1691648->1695743) req@ffff8e77c6a89c00 x1875503571692032/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/0 e 0 to 0 dl 1788619941 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619930.619700:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a89c00 with x1875503571692032 ext(1691648->1695743) 00002000:00010000:0.0:1788619930.619706:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a89c00 00000100:00100000:0.0:1788619930.619746:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 722 00000100:00100000:0.0:1788619930.619756:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a89c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2352:x1875503571692032:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619930.620012:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a89c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2352:x1875503571692032:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 257us (747us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619930.620020:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 722 00002000:00100000:0.0:1788619930.620027:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1691648->1695743) req@ffff8e77c6a89c00 x1875503571692032/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/4536 e 0 to 0 dl 1788619941 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619930.620034:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a89c00 with x1875503571692032 ext(1691648->1695743) 00002000:00010000:0.0:1788619930.620037:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a89c00 00000100:00100000:0.0:1788619930.623510:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619930.623617:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571692160 00000100:00100000:1.0:1788619930.623626:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571692160 00002000:00100000:1.0:1788619930.623674:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1695744->1699839) req@ffff8e77c50c7b80 x1875503571692160/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/0 e 0 to 0 dl 1788619941 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619930.623686:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c50c7b80 with x1875503571692160 ext(1695744->1699839) 00002000:00010000:1.0:1788619930.623692:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c50c7b80 00000100:00100000:1.0:1788619930.623723:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 723 00000100:00100000:1.0:1788619930.623733:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571692160:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619930.623990:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571692160:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 257us (489us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619930.623998:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 723 00002000:00100000:1.0:1788619930.624005:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1695744->1699839) req@ffff8e77c50c7b80 x1875503571692160/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/4536 e 0 to 0 dl 1788619941 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619930.624014:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c50c7b80 with x1875503571692160 ext(1695744->1699839) 00002000:00010000:1.0:1788619930.624018:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c50c7b80 00000100:00100000:2.0:1788619930.626847:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619930.626931:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571692288 00000100:00100000:0.0:1788619930.626940:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571692288 00002000:00100000:0.0:1788619930.626981:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1699840->1703935) req@ffff8e77c6343480 x1875503571692288/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/0 e 0 to 0 dl 1788619941 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619930.626991:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6343480 with x1875503571692288 ext(1699840->1703935) 00002000:00010000:0.0:1788619930.626996:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6343480 00000100:00100000:0.0:1788619930.627027:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 724 00000100:00100000:0.0:1788619930.627036:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6343480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571692288:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619930.627267:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6343480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571692288:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 231us (428us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619930.627275:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 724 00002000:00100000:0.0:1788619930.627282:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1699840->1703935) req@ffff8e77c6343480 x1875503571692288/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/4536 e 0 to 0 dl 1788619941 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619930.627289:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6343480 with x1875503571692288 ext(1699840->1703935) 00002000:00010000:0.0:1788619930.627292:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6343480 00000100:00100000:3.0:1788619930.630791:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619930.631087:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571692416 00000100:00100000:0.0:1788619930.631096:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571692416 00002000:00100000:0.0:1788619930.631144:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1703936->1708031) req@ffff8e77c6a8b100 x1875503571692416/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/0 e 0 to 0 dl 1788619941 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619930.631156:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a8b100 with x1875503571692416 ext(1703936->1708031) 00002000:00010000:0.0:1788619930.631162:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a8b100 00000100:00100000:0.0:1788619930.631200:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 725 00000100:00100000:0.0:1788619930.631210:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a8b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571692416:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619930.631925:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a8b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571692416:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 713us (1143us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619930.631934:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 725 00002000:00100000:1.0:1788619930.631942:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1703936->1708031) req@ffff8e77c6a8b100 x1875503571692416/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/4536 e 0 to 0 dl 1788619941 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619930.631952:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6a8b100 with x1875503571692416 ext(1703936->1708031) 00002000:00010000:1.0:1788619930.631956:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6a8b100 00000100:00100000:2.0:1788619930.635183:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619930.635676:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571692544 00000100:00100000:1.0:1788619930.635686:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571692544 00002000:00100000:1.0:1788619930.635736:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1646592->1705686) req@ffff8e77c6342d80 x1875503571692544/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/0 e 0 to 0 dl 1788619941 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619930.635747:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6342d80 with x1875503571692544 ext(1646592->1705686) 00002000:00010000:1.0:1788619930.635752:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6342d80 00000100:00100000:1.0:1788619930.635829:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 726 00000100:00100000:1.0:1788619930.635839:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6342d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571692544:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:1.0:1788619930.643521:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6342d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+10:2354:x1875503571692544:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 7670us (8333us total) trans 4294967510 rc 0/0 00000100:00100000:1.0:1788619930.643537:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 726 00002000:00100000:1.0:1788619930.643585:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1646592->1705686) req@ffff8e77c6342d80 x1875503571692544/t4294967510(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:26/0 lens 488/448 e 0 to 0 dl 1788619941 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619930.643597:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6342d80 with x1875503571692544 ext(1646592->1705686) 00002000:00010000:1.0:1788619930.643606:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6342d80 00000100:00100000:3.0:1788619930.647319:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619930.647627:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571692672 00000100:00100000:2.0:1788619930.647637:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571692672 00000100:00100000:2.0:1788619930.647684:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1237 00000100:00100000:2.0:1788619930.647693:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a89f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+10:2355:x1875503571692672:12345-192.168.201.5@tcp:16:fsx.0 00000001:00080000:2.0:1788619930.687153:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967507 is committed 00000001:00080000:2.0:1788619930.687185:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967510 is committed 00000100:00100000:0.0:1788619930.687531:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a89f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571692672:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 39832us (40216us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619930.687554:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1237 00000100:00100000:2.0:1788619930.691639:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619930.691749:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571692800 00000100:00100000:1.0:1788619930.691761:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571692800 00000100:00100000:1.0:1788619930.691931:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1510 00000100:00100000:1.0:1788619930.691943:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6340000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571692800:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619930.691966:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619930.691990:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368736833 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d8fc expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.692090:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe536873683a lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.692106:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe536873683a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.692120:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe536873683a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788619930.692150:0:9417:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1705687 00000004:00200000:1.0:1788619930.692154:0:9417:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788619930.692175:0:9417:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe536873683a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.692204:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368736833 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8fc expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.692226:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe536873683a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8fc expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.692235:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619930.692238:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788619930.692242:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c200, rc 0) 00000100:00100000:1.0:1788619930.692311:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6340000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+12:14908:x1875503571692800:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 371us (684us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619930.692319:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1510 00000100:00100000:2.0:1788619930.695425:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619930.695471:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619930.695604:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571692928 00000100:00100000:3.0:1788619930.695613:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571692928 00000100:00100000:3.0:1788619930.695671:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1511 00000100:00100000:3.0:1788619930.695681:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6341c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571692928:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619930.695701:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619930.695724:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7600/0x1cd7fe5368736841 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d903 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.695762:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687367f4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8cb expref: 19 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619930.695803:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a88e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589246976:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619930.695895:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c0c0 going to sleep for 11 seconds 00000004:00010000:3.0:1788619930.697946:0:5844:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619928 -> 1788619930 00000100:00100000:3.0:1788619930.697976:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a88e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589246976:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619930.697999:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 828664 00010000:00010000:3.0:1788619930.698018:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7600/0x1cd7fe5368736841 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d903 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.698037:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619930.698040:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619930.698045:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7600/0x1cd7fe5368736841 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d903 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.698064:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d7600, rc 0) 00000100:00100000:3.0:1788619930.698124:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6341c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571692928:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2443us (2707us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619930.698133:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1511 00000100:00100000:0.0:1788619930.701103:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571693056 00000100:00100000:0.0:1788619930.701121:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571693056 00000100:00100000:2.0:1788619930.701534:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1238 00000100:00100000:2.0:1788619930.701555:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6343b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571693056:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619930.701573:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619930.701593:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000828bfc6b/0x1cd7fe5368736848 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d90a expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619930.701658:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6342300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589247104:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619930.701745:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164240 going to sleep for 11 seconds 00002000:00010000:3.0:1788619930.709198:0:6698:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619929 -> 1788619930 00000100:00100000:3.0:1788619930.709229:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6342300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589247104:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619930.709265:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000828bfc6b/0x1cd7fe5368736848 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d90a expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.709292:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000828bfc6b/0x1cd7fe5368736848 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d90a expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.709304:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f50c00, rc 0) 00000100:00100000:3.0:1788619930.709409:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6343b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571693056:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7865us (13936us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619930.709418:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1238 00000100:00100000:3.0:1788619930.714897:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619930.714955:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619930.715078:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571693184 00000100:00100000:2.0:1788619930.715088:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571693184 00000100:00100000:2.0:1788619930.715138:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1512 00000100:00100000:2.0:1788619930.715147:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a88e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571693184:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619930.715167:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619930.715186:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50600/0x1cd7fe536873684f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d911 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.715220:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687367f4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8cb expref: 19 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619930.715256:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6343800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589247232:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619930.715354:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164480 going to sleep for 11 seconds 00000100:00100000:0.0:1788619930.715920:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571693312 00000100:00100000:0.0:1788619930.715927:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571693312 00000100:00100000:0.0:1788619930.716214:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1239 00000100:00100000:0.0:1788619930.716224:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a8b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571693312:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619930.716237:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619930.716250:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008b39475f/0x1cd7fe5368736856 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d918 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619930.716293:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c7100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589247360:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619930.716390:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa480 going to sleep for 11 seconds 00000100:00100000:3.0:1788619930.719405:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6343800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589247232:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619930.719431:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 828664 00010000:00010000:3.0:1788619930.719452:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50600/0x1cd7fe536873684f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d911 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.719483:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619930.719487:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619930.719493:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50600/0x1cd7fe536873684f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d911 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.719509:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f50600, rc 0) 00000100:00100000:3.0:1788619930.719578:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a88e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571693184:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4431us (4691us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619930.719586:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1512 00000100:00100000:0.0:1788619930.725031:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c7100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589247360:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619930.725074:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008b39475f/0x1cd7fe5368736856 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d918 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.725101:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008b39475f/0x1cd7fe5368736856 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d918 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.725117:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d4e00, rc 0) 00000100:00100000:0.0:1788619930.725182:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a8b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571693312:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8959us (10226us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619930.725194:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1239 00000100:00100000:2.0:1788619930.726540:0:3319:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1788619930.726572:0:3319:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4ba3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3319:1875503589247488:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1788619930.726643:0:3319:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619930.726808:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589247488 00000100:00100000:1.0:1788619930.726816:0:6699:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589247488 00000100:00100000:1.0:1788619930.726868:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 93 00000100:00100000:1.0:1788619930.726877:0:6699:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6341f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1875503589247488:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1788619930.726898:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 165216256 pending 0 free 3859808256 avail 3856662528 00000100:00100000:1.0:1788619930.726942:0:6699:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6341f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1875503589247488:12345-0@lo:13:osp-pre-0-0.0 Request processed in 67us (299us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619930.726953:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 93 00000100:00100000:2.0:1788619930.727036:0:3319:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4ba3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3319:1875503589247488:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1788619930.728208:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619930.728280:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571693440 00000100:00100000:1.0:1788619930.728285:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571693440 00000100:00100000:1.0:1788619930.728322:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1513 00000100:00100000:1.0:1788619930.728346:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6340700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+12:14908:x1875503571693440:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619930.728361:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619930.728383:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe536873685d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d91f expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.728411:0:9417:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687367f4 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8cb expref: 19 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.728423:0:9417:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe536873685d lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d91f expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.728454:0:9417:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687367f4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8cb expref: 19 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.728467:0:9417:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687367f4 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8cb expref: 19 pid: 5844 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619930.728480:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4ba2300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589247616:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619930.728773:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590840 going to sleep for 11 seconds 00000100:00100000:3.0:1788619930.741442:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619930.741785:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571693568 00000100:00100000:2.0:1788619930.741808:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571693568 00010000:00010000:2.0:1788619930.741867:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687367f4 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8cb expref: 20 pid: 5844 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619930.741911:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 498 00000100:00100000:2.0:1788619930.741922:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a8a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2354:x1875503571693568:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619930.741939:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619930.741971:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687367f4 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8cb expref: 20 pid: 5844 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.741986:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687367f4 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8cb expref: 20 pid: 5844 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.742010:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619930.742016:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe536873685d lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d91f expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.742029:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe536873685d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d91f expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.742040:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe536873685d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d91f expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.742092:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe536873685d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d91f expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619930.742106:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6340a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589247744:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619930.742202:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164d80 going to sleep for 11 seconds 00010000:00010000:1.0:1788619930.742433:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe53687367f4 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8cb expref: 20 pid: 5844 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619930.742464:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4ba2300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589247616:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619930.742491:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe536873685d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d91f expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.742503:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619930.742505:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619930.742509:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683dc00, rc 0) 00000100:00100000:1.0:1788619930.742538:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6340700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+13:14908:x1875503571693440:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 14194us (14334us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619930.742556:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1513 00000100:00100000:2.0:1788619930.748541:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619930.748622:0:13346:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571693696 00000100:00100000:1.0:1788619930.748632:0:13346:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571693696 00002000:00100000:1.0:1788619930.748681:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6340380 x1875503571693696/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:26/0 lens 488/0 e 0 to 0 dl 1788619941 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619930.748693:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6340380 with x1875503571693696 00000100:00100000:1.0:1788619930.748731:0:13346:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 908 00000100:00100000:1.0:1788619930.748741:0:13346:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6340380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571693696:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619930.749700:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6340a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589247744:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619930.749724:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619930.749767:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a8a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2354:x1875503571693568:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 7846us (8337us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619930.749778:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 498 00000100:00100000:1.0:1788619930.762743:0:13346:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6340380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571693696:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 13991us (14202us total) trans 0 rc 831488/831488 00000100:00100000:1.0:1788619930.762759:0:13346:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 908 00002000:00100000:1.0:1788619930.762773:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6340380 x1875503571693696/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:26/0 lens 488/440 e 0 to 0 dl 1788619941 ref 1 fl Complete:/600/0 rc 831488/831488 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619930.762784:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6340380 with x1875503571693696 00000100:00100000:2.0:1788619930.767111:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619930.767207:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571693824 00000100:00100000:3.0:1788619930.767217:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571693824 00000100:00100000:3.0:1788619930.767283:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1514 00000100:00100000:3.0:1788619930.767293:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6340e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+19:14908:x1875503571693824:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619930.767316:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619930.767357:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7e00/0x1cd7fe5368736864 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d926 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.767456:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7600/0x1cd7fe536873686b lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.767472:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7600/0x1cd7fe536873686b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.767488:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7600/0x1cd7fe536873686b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788619930.767514:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1705687 00000004:00200000:3.0:1788619930.767518:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788619930.767535:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7600/0x1cd7fe536873686b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.767570:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7e00/0x1cd7fe5368736864 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d926 expref: 21 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.767588:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7600/0x1cd7fe536873686b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d926 expref: 20 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.767596:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619930.767599:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788619930.767603:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d7600, rc 0) 00000100:00100000:3.0:1788619930.767664:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6340e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+20:14908:x1875503571693824:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 373us (565us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619930.767671:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1514 00000100:00100000:3.0:1788619930.772457:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619930.772601:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571693952 00000100:00100000:2.0:1788619930.772611:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571693952 00000100:00100000:2.0:1788619930.772672:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1515 00000100:00100000:2.0:1788619930.772692:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a88700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2354:x1875503571693952:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619930.772713:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619930.772733:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50c00/0x1cd7fe5368736872 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d92d expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.772767:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50c00/0x1cd7fe5368736872 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d92d expref: 21 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788619930.772779:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 828664 00010000:00010000:2.0:1788619930.772786:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50c00/0x1cd7fe5368736872 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d92d expref: 21 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.772798:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619930.772800:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619930.772804:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f50c00, rc 0) 00000100:00100000:2.0:1788619930.772866:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a88700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2354:x1875503571693952:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 185us (418us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619930.772874:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1515 00000100:00100000:0.0:1788619930.775162:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619930.775245:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571694080 00000100:00100000:2.0:1788619930.775254:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571694080 00000100:00100000:2.0:1788619930.775306:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1240 00000100:00100000:2.0:1788619930.775316:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571694080:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619930.775359:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619930.775377:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e07f4a4c/0x1cd7fe5368736879 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d934 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619930.775436:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6341500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589247872:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619930.775502:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164b40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619930.777887:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6341500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589247872:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619930.777925:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e07f4a4c/0x1cd7fe5368736879 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d934 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.777949:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e07f4a4c/0x1cd7fe5368736879 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d934 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.777962:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f50e00, rc 0) 00000100:00100000:0.0:1788619930.778019:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571694080:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2704us (2865us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619930.778027:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1240 00000100:00100000:3.0:1788619930.786252:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619930.786501:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571694208 00000100:00100000:2.0:1788619930.786509:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571694208 00000100:00100000:2.0:1788619930.786567:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1241 00000100:00100000:2.0:1788619930.786582:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a88000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571694208:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619930.786599:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619930.786618:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027fac38f/0x1cd7fe5368736880 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d93b expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619930.786678:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6343100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589248000:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619930.786950:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164f00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619930.788991:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6343100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589248000:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619930.789029:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000027fac38f/0x1cd7fe5368736880 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d93b expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.789054:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027fac38f/0x1cd7fe5368736880 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d93b expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.789073:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f50200, rc 0) 00000100:00100000:2.0:1788619930.789132:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a88000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571694208:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2551us (2886us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619930.789141:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1241 00000100:00100000:3.0:1788619930.793751:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619930.793971:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571694336 00000100:00100000:1.0:1788619930.793982:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571694336 00000100:00100000:1.0:1788619930.794042:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1516 00000100:00100000:1.0:1788619930.794052:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a88380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+13:14908:x1875503571694336:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619930.794073:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619930.794076:0:9417:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:1.0:1788619930.794120:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe536873685d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d91f expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.794139:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619930.794145:0:9417:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788619930.794159:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368736887 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d942 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.794184:0:9417:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50c00/0x1cd7fe5368736872 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d92d expref: 20 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.794194:0:9417:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368736887 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d942 expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.794228:0:9417:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50c00/0x1cd7fe5368736872 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d92d expref: 20 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.794241:0:9417:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50c00/0x1cd7fe5368736872 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d92d expref: 20 pid: 5845 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619930.794254:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4ba2300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589248128:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619930.794348:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590840 going to sleep for 11 seconds 00000100:00100000:1.0:1788619930.796603:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4ba2300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589248128:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619930.796642:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368736887 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d942 expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.796665:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619930.796668:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619930.796672:0:9417:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368736887 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d942 expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.796682:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c800, rc 0) 00000100:00100000:1.0:1788619930.796743:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a88380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+13:14908:x1875503571694336:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2692us (3004us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619930.796753:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1516 00000100:00100000:0.0:1788619930.802203:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619930.802295:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571694464 00000100:00100000:3.0:1788619930.802305:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571694464 00010000:00010000:3.0:1788619930.802389:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50c00/0x1cd7fe5368736872 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d92d expref: 21 pid: 5845 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619930.802428:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 499 00000100:00100000:3.0:1788619930.802438:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2352:x1875503571694464:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619930.802453:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619930.802480:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50c00/0x1cd7fe5368736872 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d92d expref: 21 pid: 5845 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.802491:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50c00/0x1cd7fe5368736872 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d92d expref: 21 pid: 5845 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.802514:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f50c00/0x1cd7fe5368736872 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d92d expref: 21 pid: 5845 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.802530:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619930.802537:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368736887 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d942 expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.802561:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368736887 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d942 expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.802570:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368736887 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d942 expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.802611:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368736887 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d942 expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619930.802630:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a8b800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589248256:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619930.802706:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48ccc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619930.806249:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a8b800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589248256:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619930.806278:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619930.806387:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2352:x1875503571694464:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3949us (4193us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619930.806400:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 499 00000100:00100000:1.0:1788619930.819190:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619930.819308:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571694592 00000100:00100000:3.0:1788619930.819319:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571694592 00002000:00100000:3.0:1788619930.819417:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4ba2300 x1875503571694592/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:26/0 lens 488/0 e 0 to 0 dl 1788619941 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619930.819428:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4ba2300 with x1875503571694592 00000100:00100000:3.0:1788619930.819457:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 909 00000100:00100000:3.0:1788619930.819466:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+13:2355:x1875503571694592:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619930.819734:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+13:2355:x1875503571694592:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 268us (554us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619930.819742:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 909 00002000:00100000:3.0:1788619930.819749:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c4ba2300 x1875503571694592/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:26/0 lens 488/4536 e 0 to 0 dl 1788619941 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619930.819758:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c4ba2300 with x1875503571694592 00000100:00100000:2.0:1788619930.824683:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619930.824765:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571694720 00000100:00100000:0.0:1788619930.824775:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571694720 00002000:00100000:0.0:1788619930.824838:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6343100 x1875503571694720/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:26/0 lens 4584/0 e 0 to 0 dl 1788619941 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619930.824849:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6343100 with x1875503571694720 00000100:00100000:0.0:1788619930.824889:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 910 00000100:00100000:0.0:1788619930.824899:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6343100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571694720:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:0.0:1788619930.825226:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 828664 -> 828664 00000004:00010000:0.0:1788619930.825230:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619930 -> 1788619930 00000004:00010000:0.0:1788619930.825232:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619930 -> 1788619928 00000004:00010000:0.0:1788619930.825235:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619930 -> 1788619930 00000004:00010000:0.0:1788619930.825237:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1672 -> 1672 00000100:00100000:0.0:1788619930.825303:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6343100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+15:2354:x1875503571694720:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 405us (629us total) trans 4294967824 rc 0/0 00000100:00100000:0.0:1788619930.825311:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 910 00002000:00100000:0.0:1788619930.825318:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6343100 x1875503571694720/t4294967824(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:26/0 lens 4584/448 e 0 to 0 dl 1788619941 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619930.825349:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6343100 with x1875503571694720 00000100:00100000:2.0:1788619930.830172:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619930.831390:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571694848 00000100:00100000:3.0:1788619930.831403:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571694848 00000100:00100000:3.0:1788619930.831462:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1517 00000100:00100000:3.0:1788619930.831473:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6341500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+15:2352:x1875503571694848:12345-192.168.201.5@tcp:16:fsx.0 00000001:00080000:2.0:1788619930.867870:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967824 is committed 00000001:00080000:2.0:1788619930.867911:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967816 is committed 00000001:00080000:2.0:1788619930.867918:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967820 is committed 00000001:00080000:2.0:1788619930.867930:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967823 is committed 00000100:00100000:0.0:1788619930.868165:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6341500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571694848:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 36688us (37997us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619930.868177:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1517 00000100:00100000:3.0:1788619930.870520:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619930.870626:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571694976 00000100:00100000:2.0:1788619930.870635:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571694976 00000100:00100000:2.0:1788619930.870694:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1242 00000100:00100000:2.0:1788619930.870703:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a8b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571694976:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619930.870721:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619930.870738:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c138a1e8/0x1cd7fe536873688e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d949 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619930.870796:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6340a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589248384:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619930.871428:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164f00 going to sleep for 11 seconds 00000100:00100000:3.0:1788619930.873966:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6340a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589248384:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619930.874006:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c138a1e8/0x1cd7fe536873688e lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d949 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.874037:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c138a1e8/0x1cd7fe536873688e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d949 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619930.874052:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f51600, rc 0) 00000100:00100000:3.0:1788619930.874108:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a8b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571694976:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3406us (3595us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619930.874116:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1242 00000100:00100000:3.0:1788619930.881086:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619930.881210:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571695104 00000100:00100000:0.0:1788619930.881220:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571695104 00000100:00100000:0.0:1788619930.881379:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1243 00000100:00100000:0.0:1788619930.881391:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7581c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571695104:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619930.881409:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619930.881426:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b724793c/0x1cd7fe5368736895 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d950 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619930.881484:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589248512:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619930.881583:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78e6e80c00 going to sleep for 11 seconds 00000100:00100000:0.0:1788619930.885249:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589248512:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619930.885290:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b724793c/0x1cd7fe5368736895 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d950 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.885316:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b724793c/0x1cd7fe5368736895 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d950 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.885361:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f52000, rc 0) 00000100:00100000:0.0:1788619930.885431:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7581c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571695104:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4042us (4356us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619930.885440:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1243 00000100:00100000:3.0:1788619930.888321:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619930.888492:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571695232 00000100:00100000:2.0:1788619930.888502:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571695232 00000100:00100000:2.0:1788619930.888575:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1518 00000100:00100000:2.0:1788619930.888586:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7580a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571695232:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619930.888605:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619930.888625:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51400/0x1cd7fe536873689c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d957 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.888661:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368736887 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d942 expref: 9 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619930.888700:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6343800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589248640:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619930.888785:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164300 going to sleep for 11 seconds 00000004:00010000:2.0:1788619930.891131:0:5845:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619928 -> 1788619930 00000100:00100000:2.0:1788619930.891163:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6343800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589248640:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619930.891184:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 828664 00010000:00010000:2.0:1788619930.891201:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51400/0x1cd7fe536873689c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d957 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.891221:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619930.891224:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619930.891229:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51400/0x1cd7fe536873689c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d957 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.891244:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f51400, rc 0) 00000100:00100000:2.0:1788619930.891298:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7580a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571695232:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2714us (2987us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619930.891307:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1518 00000100:00100000:3.0:1788619930.899172:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619930.899258:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571695360 00000100:00100000:1.0:1788619930.899267:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571695360 00000100:00100000:1.0:1788619930.899315:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1519 00000100:00100000:1.0:1788619930.899322:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7580000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571695360:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619930.899365:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619930.899386:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe53687368a3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d95e expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.899424:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368736887 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d942 expref: 9 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619930.899459:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4ba3100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589248768:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619930.899596:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590b40 going to sleep for 11 seconds 00000100:00100000:1.0:1788619930.901647:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4ba3100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589248768:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619930.901671:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 828664 00010000:00010000:1.0:1788619930.901782:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe53687368a3 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d95e expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.901799:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619930.901801:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619930.901805:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe53687368a3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d95e expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.901821:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c600, rc 0) 00000100:00100000:1.0:1788619930.901873:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7580000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571695360:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2550us (2707us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619930.901881:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1519 00000100:00100000:3.0:1788619930.904097:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619930.904185:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571695616 00000100:00100000:1.0:1788619930.904195:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571695616 00000100:00100000:1.0:1788619930.904253:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1520 00000100:00100000:1.0:1788619930.904264:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7583b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:14908:x1875503571695616:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:1.0:1788619930.904317:0:5844:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619930.904380:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe536873683a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8fc expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.904397:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619930.904402:0:5844:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788619930.904669:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687368aa lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.904682:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7600/0x1cd7fe536873686b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d926 expref: 12 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.904696:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687368aa lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.904730:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7600/0x1cd7fe536873686b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d926 expref: 12 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.904743:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7600/0x1cd7fe536873686b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d926 expref: 12 pid: 5844 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619930.904757:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4ba3100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589248896:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619930.904835:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590b40 going to sleep for 11 seconds 00000100:00100000:1.0:1788619930.907493:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4ba3100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589248896:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619930.907523:0:5844:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687368aa lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619930.907586:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619930.907627:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571695744 00000100:00100000:0.0:1788619930.907634:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571695744 00010000:00010000:0.0:1788619930.907669:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7600/0x1cd7fe536873686b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d926 expref: 13 pid: 5844 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619930.907835:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 500 00000100:00100000:0.0:1788619930.907845:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7583480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571695744:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619930.907862:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619930.907869:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7600/0x1cd7fe536873686b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d926 expref: 13 pid: 5844 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.907881:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7600/0x1cd7fe536873686b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d926 expref: 13 pid: 5844 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.907909:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7600/0x1cd7fe536873686b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d926 expref: 13 pid: 5844 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.907925:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619930.907929:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687368aa lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.907939:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687368aa lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.907947:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687368aa lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.907974:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788619930.907991:0:5844:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687368aa lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.907997:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687368aa lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619930.908028:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7583480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2355:x1875503571695744:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 185us (442us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619930.908035:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 500 00000001:00080000:1.0:1788619930.908102:0:5844:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967825 00000004:00010000:1.0:1788619930.908143:0:5844:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 828664 -> 828664 00000004:00010000:1.0:1788619930.908146:0:5844:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619930 -> 1788619930 00000004:00010000:1.0:1788619930.908148:0:5844:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619930 -> 1788619928 00000004:00010000:1.0:1788619930.908150:0:5844:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619930 -> 1788619930 00000004:00010000:1.0:1788619930.908152:0:5844:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1672 -> 1544 00000004:00010000:1.0:1788619930.908160:0:5844:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000000ac0c255 reply state 00000000290769e4 transno 4294967826 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687368aa lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619930.908170:0:5844:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788619930.908234:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7583b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:14908:x1875503571695616:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 3972us (4148us total) trans 4294967826 rc 0/0 00000100:00100000:1.0:1788619930.908242:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1520 00000020:00080000:1.0:1788619930.908246:0:5844:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:2.0:1788619930.910032:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687368aa lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.910043:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687368aa lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.910051:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687368aa lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.910055:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c683d200 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:2.0:1788619930.910058:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687368aa lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.910063:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687368aa lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.910074:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619930.910076:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788619930.910079:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687368aa lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.910083:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687368aa lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619930.910929:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619930.910992:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571695872 00000100:00100000:0.0:1788619930.911001:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571695872 00000100:00100000:0.0:1788619930.911071:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1244 00000100:00100000:0.0:1788619930.911078:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7582300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:14908:x1875503571695872:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619930.911090:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619930.911103:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000de8d07a/0x1cd7fe53687368b1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d965 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.911125:0:6696:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000614d4133/0x1cd7fe53687367fb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8d2 expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.911138:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(8 12) 00010000:00010000:0.0:1788619930.911143:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(9 12) 00010000:00010000:0.0:1788619930.911148:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000de8d07a/0x1cd7fe53687368b1 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d965 expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.911180:0:6696:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000614d4133/0x1cd7fe53687367fb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8d2 expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.911192:0:6696:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000614d4133/0x1cd7fe53687367fb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8d2 expref: 5 pid: 6698 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619930.911205:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589249024:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619930.911275:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78e6e80c00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619930.913100:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619930.913375:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571696000 00000100:00100000:1.0:1788619930.913382:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571696000 00010000:00010000:1.0:1788619930.913408:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000614d4133/0x1cd7fe53687367fb lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8d2 expref: 6 pid: 6698 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619930.913437:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 501 00000100:00100000:1.0:1788619930.913443:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcae850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571696000:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619930.913453:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619930.913471:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000614d4133/0x1cd7fe53687367fb lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8d2 expref: 6 pid: 6698 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.913481:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000614d4133/0x1cd7fe53687367fb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8d2 expref: 6 pid: 6698 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.913500:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(9 12) 00010000:00010000:1.0:1788619930.913503:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(9 12) 00010000:00010000:1.0:1788619930.913509:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000000de8d07a/0x1cd7fe53687368b1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d965 expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.913519:0:5829:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000de8d07a/0x1cd7fe53687368b1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d965 expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.913556:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000000de8d07a/0x1cd7fe53687368b1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d965 expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619930.913573:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4ba3100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1875503589249152:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619930.913657:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000614d4133/0x1cd7fe53687367fb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d8d2 expref: 6 pid: 6698 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619930.913674:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589249024:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619930.913771:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590b40 going to sleep for 11 seconds 00010000:00010000:0.0:1788619930.913815:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000de8d07a/0x1cd7fe53687368b1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d965 expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.913827:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f53400, rc 0) 00000100:00100000:0.0:1788619930.913855:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7582300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:14908:x1875503571695872:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2777us (3055us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619930.913866:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1244 00000100:00100000:2.0:1788619930.915858:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619930.916142:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4ba3100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1875503589249152:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619930.916165:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619930.916214:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcae850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571696000:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 2769us (3119us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619930.916225:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 501 00000100:00100000:3.0:1788619930.916957:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571696128 00000100:00100000:3.0:1788619930.916965:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571696128 00002000:00010000:0.0:1788619930.917031:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1177566->18446744073709551615) 00002000:00100000:0.0:1788619930.917036:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6343800 with x1875503571696128 ext(1177566->18446744073709551615) 00002000:00010000:0.0:1788619930.917046:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6343800. 00000100:00100000:0.0:1788619930.917080:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 727 00000100:00100000:2.0:1788619930.917086:0:6703:0:(service.c:3462:ptlrpc_start_thread()) ost_io[0] started 3 min 3 max 116 00000100:00100000:0.0:1788619930.917089:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6343800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571696128:12345-192.168.201.5@tcp:10:fsx.0 00000100:00100000:2.0:1788619930.917094:0:6703:0:(service.c:3521:ptlrpc_start_thread()) starting thread 'll_ost_io00_003' 00002000:00010000:0.0:1788619930.917487:0:6702:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1705687 -> 1177566 00002000:00010000:0.0:1788619930.917492:0:6702:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619930 -> 1788619930 00002000:00010000:0.0:1788619930.917495:0:6702:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619930 -> 1788619930 00002000:00010000:0.0:1788619930.917497:0:6702:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619930 -> 1788619930 00002000:00010000:0.0:1788619930.917499:0:6702:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 391 -> 263 00000100:00100000:0.0:1788619930.917555:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6343800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571696128:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 466us (1701us total) trans 4294967511 rc 0/0 00000100:00100000:0.0:1788619930.917562:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 727 00002000:00010000:0.0:1788619930.917566:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1177566->18446744073709551615) 00002000:00100000:0.0:1788619930.917569:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6343800 with x1875503571696128 ext(1177566->18446744073709551615) 00002000:00010000:0.0:1788619930.917574:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6343800. 00000100:00100000:3.0:1788619930.919222:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619930.919295:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571696256 00000100:00100000:0.0:1788619930.919302:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571696256 00000100:00100000:0.0:1788619930.919362:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1521 00000100:00100000:0.0:1788619930.919370:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7581500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571696256:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619930.919390:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619930.919406:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52800/0x1cd7fe53687368b8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d96c expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.919476:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f53c00/0x1cd7fe53687368bf lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.919488:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f53c00/0x1cd7fe53687368bf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.919496:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f53c00/0x1cd7fe53687368bf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619930.919515:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1177566 00000004:00200000:0.0:1788619930.919519:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619930.919531:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f53c00/0x1cd7fe53687368bf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.919563:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52800/0x1cd7fe53687368b8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d96c expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.919580:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f53c00/0x1cd7fe53687368bf lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d96c expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.919588:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619930.919591:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619930.919594:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f53c00, rc 0) 00000100:00100000:0.0:1788619930.919872:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7581500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:14908:x1875503571696256:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 502us (659us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619930.919880:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1521 00000100:00100000:1.0:1788619930.922706:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619930.922770:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571696384 00000100:00100000:0.0:1788619930.922780:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571696384 00000100:00100000:0.0:1788619930.922835:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1522 00000100:00100000:0.0:1788619930.922845:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4ba3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+12:14908:x1875503571696384:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619930.922866:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619930.922883:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f53e00/0x1cd7fe53687368c6 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d973 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.922955:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52200/0x1cd7fe53687368cd lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.922967:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52200/0x1cd7fe53687368cd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.922975:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52200/0x1cd7fe53687368cd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619930.922994:0:9417:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1177566 00000004:00200000:0.0:1788619930.922997:0:9417:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619930.923010:0:9417:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52200/0x1cd7fe53687368cd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.923029:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f53e00/0x1cd7fe53687368c6 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d973 expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.923041:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52200/0x1cd7fe53687368cd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d973 expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.923046:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619930.923049:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619930.923051:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f52200, rc 0) 00000100:00100000:0.0:1788619930.923094:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4ba3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571696384:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 251us (397us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619930.923101:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1522 00000100:00100000:0.0:1788619930.925316:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619930.925581:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571696512 00000100:00100000:1.0:1788619930.925592:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571696512 00000100:00100000:1.0:1788619930.925644:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1523 00000100:00100000:1.0:1788619930.925654:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571696512:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619930.925674:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619930.925810:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe53687368d4 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d97a expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619930.925847:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00010000:00010000:1.0:1788619930.925847:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368736887 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d942 expref: 11 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619930.925896:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c31f80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589249280:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619930.925912:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571696640 00000100:00100000:2.0:1788619930.925915:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571696640 00000100:00100000:1.0:1788619930.925973:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590180 going to sleep for 11 seconds 00000100:00100000:2.0:1788619930.925977:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1245 00000100:00100000:2.0:1788619930.925985:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7583100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571696640:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619930.925993:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619930.926004:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000091a35e8b/0x1cd7fe53687368db lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d981 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619930.926039:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6342300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589249408:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619930.926058:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164300 going to sleep for 11 seconds 00000004:00010000:1.0:1788619930.927560:0:5844:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619928 -> 1788619930 00000100:00100000:1.0:1788619930.927593:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c31f80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589249280:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619930.927615:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 828664 00000100:00100000:2.0:1788619930.927622:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6342300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589249408:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619930.927629:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe53687368d4 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d97a expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.927637:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000091a35e8b/0x1cd7fe53687368db lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d981 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.927646:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619930.927649:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619930.927651:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000091a35e8b/0x1cd7fe53687368db lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d981 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.927653:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe53687368d4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d97a expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.927666:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f51a00, rc 0) 00010000:00010000:1.0:1788619930.927666:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c600, rc 0) 00000100:00100000:2.0:1788619930.927709:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7583100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571696640:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 1725us (1862us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619930.927716:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1245 00000100:00100000:1.0:1788619930.927816:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571696512:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2164us (2506us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619930.927824:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1523 00000100:00100000:3.0:1788619930.930479:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619930.930505:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619930.930560:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571696896 00000100:00100000:2.0:1788619930.930569:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571696896 00000100:00100000:2.0:1788619930.930626:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1246 00000100:00100000:0.0:1788619930.930635:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571696768 00000100:00100000:2.0:1788619930.930636:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7583480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571696896:12345-192.168.201.5@tcp:101:fsx.0 00000100:00100000:0.0:1788619930.930640:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571696768 00010000:00010000:2.0:1788619930.930654:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619930.930672:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c1bed124/0x1cd7fe53687368e2 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d98f expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619930.930676:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1524 00000100:00100000:0.0:1788619930.930683:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7582d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571696768:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619930.930693:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619930.930706:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f53200/0x1cd7fe53687368e9 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d988 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.930741:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368736887 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d942 expref: 11 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619930.930780:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c6680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589249536:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619930.930831:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6342300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589249664:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619930.930863:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164300 going to sleep for 11 seconds 00000100:00100000:0.0:1788619930.930863:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78e6e80c00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619930.932343:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6342300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589249664:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619930.932356:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c6680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589249536:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619930.932376:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c1bed124/0x1cd7fe53687368e2 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d98f expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788619930.932376:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 828664 00010000:00010000:0.0:1788619930.932385:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f53200/0x1cd7fe53687368e9 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d988 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.932396:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c1bed124/0x1cd7fe53687368e2 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d98f expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619930.932396:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619930.932399:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619930.932403:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f53200/0x1cd7fe53687368e9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d988 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.932408:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f50a00, rc 0) 00010000:00010000:0.0:1788619930.932411:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f53200, rc 0) 00000100:00100000:2.0:1788619930.932446:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7583480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571696896:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 1811us (1938us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619930.932454:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1246 00000100:00100000:0.0:1788619930.932600:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7582d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571696768:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 1916us (2127us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619930.932609:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1524 00000100:00100000:0.0:1788619930.936593:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619930.936636:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619930.936687:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571697152 00000100:00100000:2.0:1788619930.936703:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571697152 00000100:00100000:2.0:1788619930.936753:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1247 00000100:00100000:1.0:1788619930.936753:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571697024 00000100:00100000:1.0:1788619930.936758:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571697024 00000100:00100000:2.0:1788619930.936764:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571697152:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619930.936781:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1788619930.936792:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1525 00000100:00100000:1.0:1788619930.936798:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571697024:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619930.936799:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000068c9016b/0x1cd7fe53687368f0 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d99d expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.936807:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619930.936829:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe53687368f7 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d996 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619930.936855:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6342300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589249792:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619930.936859:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368736887 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d942 expref: 11 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619930.936879:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c31f80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589249920:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619930.936929:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164300 going to sleep for 11 seconds 00000100:00100000:1.0:1788619930.938198:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590180 going to sleep for 11 seconds 00000100:00100000:2.0:1788619930.939634:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6342300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589249792:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619930.939677:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000068c9016b/0x1cd7fe53687368f0 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d99d expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.939726:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000068c9016b/0x1cd7fe53687368f0 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d99d expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619930.939740:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f51e00, rc 0) 00000100:00100000:2.0:1788619930.939825:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571697152:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3061us (3187us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619930.939834:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1247 00000100:00100000:1.0:1788619930.940891:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c31f80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589249920:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619930.940971:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 828664 00010000:00010000:1.0:1788619930.940985:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe53687368f7 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d996 expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.940998:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619930.941000:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619930.941003:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe53687368f7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d996 expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.941012:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d400, rc 0) 00000100:00100000:1.0:1788619930.941048:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571697024:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4250us (4466us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619930.941056:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1525 00000100:00100000:2.0:1788619930.942908:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619930.942964:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571697408 00000100:00100000:1.0:1788619930.942973:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571697408 00000100:00100000:2.0:1788619930.942981:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619930.943064:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571697280 00000100:00100000:0.0:1788619930.943069:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571697280 00000100:00100000:1.0:1788619930.943071:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1248 00000100:00100000:1.0:1788619930.943080:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6342300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571697408:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619930.943097:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619930.943114:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe53687368fe lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d9ab expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619930.943171:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c31f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589250048:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619930.943194:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1526 00000100:00100000:0.0:1788619930.943203:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6341880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571697280:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619930.943213:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619930.943226:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f53200/0x1cd7fe5368736905 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d9a4 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619930.943243:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8590180 going to sleep for 11 seconds 00010000:00010000:0.0:1788619930.943254:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368736887 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d942 expref: 11 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619930.943279:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c50c4e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589250176:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619930.943304:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78e6e80c00 going to sleep for 11 seconds 00000100:00100000:1.0:1788619930.944689:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c31f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589250048:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619930.944724:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe53687368fe lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9ab expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.944747:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe53687368fe lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9ab expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.944759:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c600, rc 0) 00000100:00100000:1.0:1788619930.944818:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6342300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571697408:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 1730us (1909us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619930.944826:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1248 00000100:00100000:1.0:1788619930.947448:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c50c4e00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589250176:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619930.947478:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 828664 00010000:00010000:1.0:1788619930.947499:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f53200/0x1cd7fe5368736905 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9a4 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.947519:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619930.947522:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619930.947529:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f53200/0x1cd7fe5368736905 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9a4 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619930.947553:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f53200, rc 0) 00000100:00100000:1.0:1788619930.947613:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6341880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571697280:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4410us (4630us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619930.947622:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1526 00000100:00100000:2.0:1788619930.954798:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619930.954850:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619930.954937:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571697664 00000100:00100000:1.0:1788619930.954950:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571697664 00002000:00100000:1.0:1788619930.955001:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1179647) req@ffff8e7900d5c380 x1875503571697664/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:26/0 lens 488/0 e 0 to 0 dl 1788619941 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619930.955012:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e7900d5c380 with x1875503571697664 ext(1048576->1179647) 00002000:00010000:1.0:1788619930.955018:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e7900d5c380 00000100:00100000:1.0:1788619930.955053:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 728 00000100:00100000:1.0:1788619930.955063:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2352:x1875503571697664:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619930.956045:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571697536 00000100:00100000:3.0:1788619930.956054:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571697536 00002000:00100000:3.0:1788619930.956087:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6340a80 x1875503571697536/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:26/0 lens 504/0 e 0 to 0 dl 1788619941 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619930.956098:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6340a80 with x1875503571697536 00000100:00100000:3.0:1788619930.956137:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 911 00000100:00100000:3.0:1788619930.956146:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6340a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2353:x1875503571697536:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619930.977781:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6340a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2353:x1875503571697536:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 21624us (22981us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619930.977796:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 911 00002000:00100000:0.0:1788619930.977817:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6340a80 x1875503571697536/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:26/0 lens 504/440 e 0 to 0 dl 1788619941 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619930.977828:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6340a80 with x1875503571697536 00000100:00100000:1.0:1788619930.980489:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2352:x1875503571697664:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 25415us (25626us total) trans 0 rc 131072/131072 00000100:00100000:1.0:1788619930.980503:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 728 00002000:00100000:1.0:1788619930.980514:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1179647) req@ffff8e7900d5c380 x1875503571697664/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:26/0 lens 488/440 e 0 to 0 dl 1788619941 ref 1 fl Complete:/600/0 rc 131072/131072 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619930.980525:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e7900d5c380 with x1875503571697664 ext(1048576->1179647) 00002000:00010000:1.0:1788619930.980532:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e7900d5c380 00000100:00100000:2.0:1788619930.980719:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619930.981583:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571697792 00000100:00100000:1.0:1788619930.981593:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571697792 00002000:00100000:1.0:1788619930.981637:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900d5ed80 x1875503571697792/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:26/0 lens 488/0 e 0 to 0 dl 1788619941 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619930.981648:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900d5ed80 with x1875503571697792 00000100:00100000:1.0:1788619930.981678:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 912 00000100:00100000:1.0:1788619930.981689:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571697792:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619930.981932:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571697792:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 244us (1212us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619930.981940:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 912 00002000:00100000:1.0:1788619930.981947:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900d5ed80 x1875503571697792/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:26/0 lens 488/4536 e 0 to 0 dl 1788619941 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619930.981956:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900d5ed80 with x1875503571697792 00000100:00100000:2.0:1788619930.984456:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619930.984626:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571697920 00000100:00100000:1.0:1788619930.984636:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571697920 00002000:00100000:1.0:1788619930.984689:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900d5f100 x1875503571697920/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:26/0 lens 488/0 e 0 to 0 dl 1788619941 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619930.984700:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900d5f100 with x1875503571697920 00000100:00100000:1.0:1788619930.984773:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 913 00000100:00100000:1.0:1788619930.984783:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2352:x1875503571697920:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619930.985108:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2352:x1875503571697920:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 324us (658us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619930.985117:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 913 00002000:00100000:1.0:1788619930.985124:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900d5f100 x1875503571697920/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:26/0 lens 488/4536 e 0 to 0 dl 1788619941 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619930.985132:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900d5f100 with x1875503571697920 00000100:00100000:2.0:1788619930.988675:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619930.988772:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571698048 00000100:00100000:1.0:1788619930.988781:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571698048 00002000:00100000:1.0:1788619930.988883:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900d5dc00 x1875503571698048/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:26/0 lens 488/0 e 0 to 0 dl 1788619941 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619930.988895:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900d5dc00 with x1875503571698048 00000100:00100000:1.0:1788619930.988965:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 914 00000100:00100000:1.0:1788619930.988975:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2353:x1875503571698048:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619930.989238:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2353:x1875503571698048:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 263us (572us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619930.989246:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 914 00002000:00100000:1.0:1788619930.989253:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900d5dc00 x1875503571698048/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:26/0 lens 488/4536 e 0 to 0 dl 1788619941 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619930.989260:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900d5dc00 with x1875503571698048 00000100:00100000:0.0:1788619930.991806:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619930.991934:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571698176 00000100:00100000:3.0:1788619930.991943:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571698176 00002000:00100000:3.0:1788619930.991990:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c50c4e00 x1875503571698176/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:26/0 lens 488/0 e 0 to 0 dl 1788619941 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619930.992001:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c50c4e00 with x1875503571698176 00000100:00100000:3.0:1788619930.992037:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 915 00000100:00100000:3.0:1788619930.992047:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c50c4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2352:x1875503571698176:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619930.992298:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c50c4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2352:x1875503571698176:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 251us (500us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619930.992306:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 915 00002000:00100000:3.0:1788619930.992313:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c50c4e00 x1875503571698176/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:26/0 lens 488/4536 e 0 to 0 dl 1788619941 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619930.992320:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c50c4e00 with x1875503571698176 00000100:00100000:3.0:1788619930.995015:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619930.995195:0:13346:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571698304 00000100:00100000:2.0:1788619930.995206:0:13346:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571698304 00002000:00100000:2.0:1788619930.995251:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c4a4ca80 x1875503571698304/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:26/0 lens 488/0 e 0 to 0 dl 1788619941 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619930.995260:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c4a4ca80 with x1875503571698304 00000100:00100000:2.0:1788619930.995289:0:13346:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 916 00000100:00100000:2.0:1788619930.995299:0:13346:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+12:2353:x1875503571698304:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619930.995572:0:13346:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+12:2353:x1875503571698304:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 273us (566us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619930.995579:0:13346:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 916 00002000:00100000:2.0:1788619930.995587:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c4a4ca80 x1875503571698304/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:26/0 lens 488/4536 e 0 to 0 dl 1788619941 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619930.995595:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c4a4ca80 with x1875503571698304 00000100:00100000:2.0:1788619931.000553:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619931.000640:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571698432 00000100:00100000:0.0:1788619931.000650:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571698432 00002000:00100000:0.0:1788619931.000775:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900d5ea00 x1875503571698432/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:27/0 lens 488/0 e 0 to 0 dl 1788619942 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619931.000786:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900d5ea00 with x1875503571698432 00000100:00100000:0.0:1788619931.000831:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 917 00000100:00100000:0.0:1788619931.000841:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2353:x1875503571698432:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619931.001300:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2353:x1875503571698432:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 459us (767us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619931.001309:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 917 00002000:00100000:0.0:1788619931.001317:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900d5ea00 x1875503571698432/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:27/0 lens 488/4536 e 0 to 0 dl 1788619942 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619931.001347:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900d5ea00 with x1875503571698432 00000100:00100000:2.0:1788619931.005015:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619931.005081:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571698560 00000100:00100000:3.0:1788619931.005091:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571698560 00002000:00100000:3.0:1788619931.005140:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900d5c700 x1875503571698560/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:27/0 lens 488/0 e 0 to 0 dl 1788619942 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619931.005149:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900d5c700 with x1875503571698560 00000100:00100000:3.0:1788619931.005189:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 918 00000100:00100000:3.0:1788619931.005199:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2352:x1875503571698560:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619931.005729:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2352:x1875503571698560:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 530us (722us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619931.005742:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 918 00002000:00100000:3.0:1788619931.005749:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900d5c700 x1875503571698560/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:27/0 lens 488/4536 e 0 to 0 dl 1788619942 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619931.005757:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900d5c700 with x1875503571698560 00000100:00100000:2.0:1788619931.009009:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619931.009105:0:13346:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571698688 00000100:00100000:3.0:1788619931.009114:0:13346:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571698688 00002000:00100000:3.0:1788619931.009160:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900d5ca80 x1875503571698688/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:27/0 lens 488/0 e 0 to 0 dl 1788619942 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619931.009172:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900d5ca80 with x1875503571698688 00000100:00100000:3.0:1788619931.009243:0:13346:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 919 00000100:00100000:3.0:1788619931.009254:0:13346:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+12:2352:x1875503571698688:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619931.009479:0:13346:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+12:2352:x1875503571698688:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 227us (478us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619931.009487:0:13346:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 919 00002000:00100000:3.0:1788619931.009494:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900d5ca80 x1875503571698688/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:27/0 lens 488/4536 e 0 to 0 dl 1788619942 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619931.009503:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900d5ca80 with x1875503571698688 00000100:00100000:0.0:1788619931.013959:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619931.014063:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571698816 00000100:00100000:1.0:1788619931.014075:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571698816 00002000:00100000:1.0:1788619931.014126:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b36d80 x1875503571698816/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:27/0 lens 488/0 e 0 to 0 dl 1788619942 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619931.014138:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b36d80 with x1875503571698816 00000100:00100000:1.0:1788619931.014209:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 920 00000100:00100000:1.0:1788619931.014219:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b36d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2352:x1875503571698816:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619931.014609:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b36d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2352:x1875503571698816:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 390us (659us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619931.014618:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 920 00002000:00100000:1.0:1788619931.014625:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b36d80 x1875503571698816/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:27/0 lens 488/4536 e 0 to 0 dl 1788619942 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619931.014633:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b36d80 with x1875503571698816 00000100:00100000:1.0:1788619931.018018:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619931.018124:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571698944 00000100:00100000:0.0:1788619931.018134:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571698944 00002000:00100000:0.0:1788619931.018184:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5c31f80 x1875503571698944/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:27/0 lens 488/0 e 0 to 0 dl 1788619942 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619931.018197:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5c31f80 with x1875503571698944 00000100:00100000:0.0:1788619931.018263:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 921 00000100:00100000:0.0:1788619931.018273:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c31f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2352:x1875503571698944:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619931.018583:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c31f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2352:x1875503571698944:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 310us (577us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619931.018591:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 921 00002000:00100000:0.0:1788619931.018599:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5c31f80 x1875503571698944/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:27/0 lens 488/4536 e 0 to 0 dl 1788619942 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619931.018613:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5c31f80 with x1875503571698944 00000100:00100000:2.0:1788619931.022760:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619931.022950:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571699072 00000100:00100000:3.0:1788619931.022960:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571699072 00002000:00100000:3.0:1788619931.023010:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900d5e680 x1875503571699072/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:27/0 lens 488/0 e 0 to 0 dl 1788619942 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619931.023020:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900d5e680 with x1875503571699072 00000100:00100000:3.0:1788619931.023059:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 922 00000100:00100000:3.0:1788619931.023068:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2353:x1875503571699072:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619931.023278:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2353:x1875503571699072:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 211us (528us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619931.023286:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 922 00002000:00100000:3.0:1788619931.023293:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900d5e680 x1875503571699072/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:27/0 lens 488/4536 e 0 to 0 dl 1788619942 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619931.023301:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900d5e680 with x1875503571699072 00000100:00100000:2.0:1788619931.027932:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619931.028322:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571699200 00000100:00100000:0.0:1788619931.028353:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571699200 00000100:00100000:0.0:1788619931.028428:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1527 00000100:00100000:0.0:1788619931.028438:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571699200:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619931.028457:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619931.028480:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f53600/0x1cd7fe536873690c lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d9b2 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.028514:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368736887 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d942 expref: 11 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.028524:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f53600/0x1cd7fe536873690c lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9b2 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.028576:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368736887 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d942 expref: 11 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.028589:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368736887 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d942 expref: 11 pid: 9417 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619931.028602:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b35c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589250304:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619931.028728:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78e6e803c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619931.031485:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b35c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589250304:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619931.031516:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f53600/0x1cd7fe536873690c lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9b2 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.031530:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619931.031532:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619931.031536:0:5845:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f53600/0x1cd7fe536873690c lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9b2 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.031555:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f53600, rc 0) 00000100:00100000:0.0:1788619931.031617:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571699200:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3179us (3693us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619931.031625:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1527 00000100:00100000:1.0:1788619931.032972:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619931.033595:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571699328 00000100:00100000:3.0:1788619931.033605:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571699328 00010000:00010000:2.0:1788619931.033792:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368736887 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d942 expref: 12 pid: 9417 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619931.033927:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 502 00000100:00100000:2.0:1788619931.033938:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c33b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571699328:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619931.033953:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619931.033979:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368736887 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d942 expref: 12 pid: 9417 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619931.033993:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368736887 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d942 expref: 12 pid: 9417 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619931.034014:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368736887 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d942 expref: 12 pid: 9417 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619931.034029:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619931.034034:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f53600/0x1cd7fe536873690c lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9b2 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619931.034050:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f53600/0x1cd7fe536873690c lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9b2 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619931.034059:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f53600/0x1cd7fe536873690c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9b2 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619931.034094:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f53600/0x1cd7fe536873690c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9b2 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619931.034106:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900d5d500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589250432:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619931.034187:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81649c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619931.036295:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900d5d500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589250432:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619931.036322:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619931.036409:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c33b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571699328:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 2472us (3439us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619931.036422:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 502 00000100:00100000:3.0:1788619931.039130:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619931.039215:0:13346:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571699456 00000100:00100000:2.0:1788619931.039224:0:13346:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571699456 00000100:00100000:2.0:1788619931.039297:0:13346:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 923 00000100:00100000:2.0:1788619931.039308:0:13346:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571699456:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:2.0:1788619931.039643:0:13346:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 828664 -> 828664 00000004:00010000:2.0:1788619931.039647:0:13346:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619930 -> 1788619930 00000004:00010000:2.0:1788619931.039649:0:13346:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619930 -> 1788619930 00000004:00010000:2.0:1788619931.039651:0:13346:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619930 -> 1788619930 00000004:00010000:2.0:1788619931.039653:0:13346:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1544 -> 1544 00000100:00100000:2.0:1788619931.039728:0:13346:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571699456:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 422us (609us total) trans 4294967827 rc 0/0 00000100:00100000:2.0:1788619931.039736:0:13346:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 923 00000100:00100000:2.0:1788619931.045727:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619931.046023:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571699584 00000100:00100000:3.0:1788619931.046033:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571699584 00000100:00100000:3.0:1788619931.046108:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1249 00000100:00100000:3.0:1788619931.046116:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571699584:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619931.046131:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619931.046148:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e6a43d0/0x1cd7fe5368736913 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d9b9 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619931.046207:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c4a4d180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589250560:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619931.046287:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c780 going to sleep for 11 seconds 00000100:00100000:3.0:1788619931.048103:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c4a4d180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589250560:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619931.048139:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005e6a43d0/0x1cd7fe5368736913 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9b9 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619931.048164:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e6a43d0/0x1cd7fe5368736913 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9b9 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619931.048183:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d6a00, rc 0) 00000100:00100000:3.0:1788619931.048234:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571699584:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2117us (2517us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619931.048242:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1249 00000100:00100000:0.0:1788619931.052120:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619931.052627:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571699712 00000100:00100000:3.0:1788619931.052637:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571699712 00000100:00100000:2.0:1788619931.052916:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1250 00000100:00100000:2.0:1788619931.052929:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b35c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571699712:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619931.052947:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619931.052964:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000de364e00/0x1cd7fe536873691a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d9c0 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619931.053025:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900d5ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589250688:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619931.053110:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164000 going to sleep for 11 seconds 00000100:00100000:0.0:1788619931.055045:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900d5ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589250688:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619931.055078:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000de364e00/0x1cd7fe536873691a lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9c0 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.055102:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000de364e00/0x1cd7fe536873691a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9c0 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.055115:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f51800, rc 0) 00000100:00100000:0.0:1788619931.055282:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b35c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571699712:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2355us (3172us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619931.055292:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1250 00000100:00100000:3.0:1788619931.058595:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619931.058718:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571699840 00000100:00100000:2.0:1788619931.058728:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571699840 00000100:00100000:2.0:1788619931.058787:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1251 00000100:00100000:2.0:1788619931.058798:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571699840:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619931.058816:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619931.058834:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000056957b7a/0x1cd7fe5368736921 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d9c7 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619931.058891:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900d5c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589250816:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619931.058973:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164840 going to sleep for 11 seconds 00000100:00100000:2.0:1788619931.060955:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900d5c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589250816:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619931.060989:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000056957b7a/0x1cd7fe5368736921 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9c7 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619931.061011:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000056957b7a/0x1cd7fe5368736921 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9c7 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619931.061029:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f51400, rc 0) 00000100:00100000:2.0:1788619931.061110:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571699840:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2314us (2526us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619931.061119:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1251 00000100:00100000:0.0:1788619931.067810:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619931.067978:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571699968 00000100:00100000:1.0:1788619931.067990:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571699968 00002000:00100000:1.0:1788619931.068042:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b37100 x1875503571699968/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:27/0 lens 488/0 e 0 to 0 dl 1788619942 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619931.068053:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b37100 with x1875503571699968 00000100:00100000:1.0:1788619931.068090:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 924 00000100:00100000:1.0:1788619931.068100:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b37100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2352:x1875503571699968:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619931.068321:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b37100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2352:x1875503571699968:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 221us (520us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619931.068360:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 924 00002000:00100000:1.0:1788619931.068369:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b37100 x1875503571699968/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:27/0 lens 488/4536 e 0 to 0 dl 1788619942 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619931.068376:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b37100 with x1875503571699968 00000100:00100000:3.0:1788619931.072424:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619931.072597:0:13346:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571700096 00000100:00100000:2.0:1788619931.072607:0:13346:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571700096 00002000:00100000:2.0:1788619931.072668:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c4a4e680 x1875503571700096/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:27/0 lens 488/0 e 0 to 0 dl 1788619942 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619931.072788:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c4a4e680 with x1875503571700096 00000100:00100000:2.0:1788619931.072825:0:13346:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 925 00000100:00100000:2.0:1788619931.072836:0:13346:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2353:x1875503571700096:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619931.073037:0:13346:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2353:x1875503571700096:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 202us (623us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619931.073044:0:13346:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 925 00002000:00100000:2.0:1788619931.073050:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c4a4e680 x1875503571700096/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:27/0 lens 488/4536 e 0 to 0 dl 1788619942 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619931.073057:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c4a4e680 with x1875503571700096 00000100:00100000:0.0:1788619931.076601:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619931.076800:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571700224 00000100:00100000:1.0:1788619931.076867:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571700224 00002000:00100000:1.0:1788619931.076919:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b37800 x1875503571700224/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:27/0 lens 488/0 e 0 to 0 dl 1788619942 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619931.076931:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b37800 with x1875503571700224 00000100:00100000:1.0:1788619931.076965:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 926 00000100:00100000:1.0:1788619931.076974:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b37800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2352:x1875503571700224:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619931.077235:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b37800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2352:x1875503571700224:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 260us (644us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619931.077245:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 926 00002000:00100000:1.0:1788619931.077252:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b37800 x1875503571700224/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:27/0 lens 488/4536 e 0 to 0 dl 1788619942 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619931.077260:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b37800 with x1875503571700224 00000100:00100000:0.0:1788619931.087686:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619931.087917:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571700352 00000100:00100000:3.0:1788619931.087928:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571700352 00002000:00100000:3.0:1788619931.087982:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b35180 x1875503571700352/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:27/0 lens 488/0 e 0 to 0 dl 1788619942 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619931.087994:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b35180 with x1875503571700352 00000100:00100000:3.0:1788619931.088034:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 927 00000100:00100000:3.0:1788619931.088046:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b35180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2352:x1875503571700352:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619931.088518:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b35180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2352:x1875503571700352:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 473us (848us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619931.088527:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 927 00002000:00100000:3.0:1788619931.088534:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b35180 x1875503571700352/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:27/0 lens 488/4536 e 0 to 0 dl 1788619942 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619931.088557:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b35180 with x1875503571700352 00000100:00100000:2.0:1788619931.097627:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619931.098033:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571700480 00000100:00100000:0.0:1788619931.098043:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571700480 00002000:00100000:0.0:1788619931.098093:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900d5c000 x1875503571700480/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:27/0 lens 488/0 e 0 to 0 dl 1788619942 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619931.098106:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900d5c000 with x1875503571700480 00000100:00100000:0.0:1788619931.098146:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 928 00000100:00100000:0.0:1788619931.098155:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2352:x1875503571700480:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619931.098423:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2352:x1875503571700480:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 268us (813us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619931.098432:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 928 00002000:00100000:0.0:1788619931.098438:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900d5c000 x1875503571700480/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:27/0 lens 488/4536 e 0 to 0 dl 1788619942 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619931.098447:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900d5c000 with x1875503571700480 00000100:00100000:2.0:1788619931.106689:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619931.106773:0:13346:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571700608 00000100:00100000:3.0:1788619931.106784:0:13346:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571700608 00002000:00100000:3.0:1788619931.106848:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900d5f480 x1875503571700608/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:27/0 lens 488/0 e 0 to 0 dl 1788619942 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619931.106860:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900d5f480 with x1875503571700608 00000100:00100000:3.0:1788619931.106895:0:13346:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 929 00000100:00100000:3.0:1788619931.106905:0:13346:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571700608:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619931.107211:0:13346:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571700608:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 306us (543us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619931.107221:0:13346:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 929 00002000:00100000:3.0:1788619931.107228:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900d5f480 x1875503571700608/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:27/0 lens 488/4536 e 0 to 0 dl 1788619942 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619931.107237:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900d5f480 with x1875503571700608 00000100:00100000:2.0:1788619931.111582:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619931.111796:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571700736 00000100:00100000:0.0:1788619931.111815:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571700736 00002000:00100000:0.0:1788619931.111866:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900d5ce00 x1875503571700736/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:27/0 lens 488/0 e 0 to 0 dl 1788619942 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619931.111879:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900d5ce00 with x1875503571700736 00000100:00100000:0.0:1788619931.111917:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 930 00000100:00100000:0.0:1788619931.111927:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2353:x1875503571700736:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619931.112134:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2353:x1875503571700736:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 207us (561us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619931.112140:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 930 00002000:00100000:0.0:1788619931.112147:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900d5ce00 x1875503571700736/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:27/0 lens 488/4536 e 0 to 0 dl 1788619942 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619931.112160:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900d5ce00 with x1875503571700736 00000100:00100000:1.0:1788619931.115608:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619931.115709:0:13346:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571700864 00000100:00100000:3.0:1788619931.115719:0:13346:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571700864 00002000:00100000:3.0:1788619931.115769:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6a8a300 x1875503571700864/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:27/0 lens 488/0 e 0 to 0 dl 1788619942 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619931.115780:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6a8a300 with x1875503571700864 00000100:00100000:3.0:1788619931.115821:0:13346:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 931 00000100:00100000:3.0:1788619931.115832:0:13346:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a8a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2352:x1875503571700864:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619931.116223:0:13346:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a8a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2352:x1875503571700864:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 392us (627us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619931.116231:0:13346:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 931 00002000:00100000:3.0:1788619931.116239:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6a8a300 x1875503571700864/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:27/0 lens 488/4536 e 0 to 0 dl 1788619942 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619931.116246:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6a8a300 with x1875503571700864 00000100:00100000:2.0:1788619931.120932:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619931.121079:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571700992 00000100:00100000:0.0:1788619931.121090:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571700992 00002000:00100000:0.0:1788619931.121139:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900d5d500 x1875503571700992/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:27/0 lens 488/0 e 0 to 0 dl 1788619942 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619931.121151:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900d5d500 with x1875503571700992 00000100:00100000:0.0:1788619931.121182:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 932 00000100:00100000:0.0:1788619931.121191:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900d5d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2353:x1875503571700992:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:1.0:1788619931.125588:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 828664 -> 828664 00000004:00010000:1.0:1788619931.125598:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619930 -> 1788619931 00000004:00010000:1.0:1788619931.125601:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619930 -> 1788619930 00000004:00010000:1.0:1788619931.125604:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619930 -> 1788619931 00000004:00010000:1.0:1788619931.125606:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1544 -> 1544 00000100:00100000:1.0:1788619931.127214:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900d5d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2353:x1875503571700992:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 6014us (6282us total) trans 4294967828 rc 0/0 00000100:00100000:1.0:1788619931.127229:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 932 00002000:00100000:1.0:1788619931.127242:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e7900d5d500 x1875503571700992/t4294967828(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:27/0 lens 488/448 e 0 to 0 dl 1788619942 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619931.127253:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e7900d5d500 with x1875503571700992 00000100:00100000:0.0:1788619931.129790:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619931.129935:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571701120 00000100:00100000:1.0:1788619931.129945:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571701120 00000100:00100000:1.0:1788619931.130044:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1528 00000100:00100000:1.0:1788619931.130056:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b34700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2355:x1875503571701120:12345-192.168.201.5@tcp:16:fsx.0 00000001:00080000:0.0:1788619931.185986:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967828 is committed 00000001:00080000:0.0:1788619931.186007:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967826 is committed 00000100:00100000:1.0:1788619931.186210:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b34700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571701120:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 56152us (56423us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619931.186224:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1528 00000100:00100000:0.0:1788619931.190570:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619931.190654:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571701248 00000100:00100000:1.0:1788619931.190664:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571701248 00000100:00100000:1.0:1788619931.190735:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1529 00000100:00100000:1.0:1788619931.190746:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b37480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571701248:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619931.190768:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619931.190791:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368736928 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d9ce expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.190848:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f53600/0x1cd7fe536873690c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9b2 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619931.190895:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c71f1c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589250944:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619931.191082:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccc00 going to sleep for 11 seconds 00000100:00100000:1.0:1788619931.193617:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c71f1c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589250944:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619931.193646:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 828664 00010000:00010000:1.0:1788619931.193664:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368736928 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9ce expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.193757:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619931.193762:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619931.193768:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368736928 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9ce expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.193784:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683de00, rc 0) 00000100:00100000:1.0:1788619931.193998:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b37480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571701248:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3253us (3438us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619931.194007:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1529 00000100:00100000:2.0:1788619931.197291:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619931.197467:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571701376 00000100:00100000:0.0:1788619931.197476:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571701376 00000100:00100000:0.0:1788619931.197562:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1530 00000100:00100000:0.0:1788619931.197572:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571701376:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619931.197592:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619931.197610:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f53a00/0x1cd7fe536873692f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d9d5 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.197643:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f53600/0x1cd7fe536873690c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9b2 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619931.197742:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b36300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589251072:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619931.197830:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78e6e803c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619931.200642:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b36300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589251072:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619931.200671:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 828664 00010000:00010000:0.0:1788619931.200700:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f53a00/0x1cd7fe536873692f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9d5 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.200721:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619931.200724:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619931.200730:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f53a00/0x1cd7fe536873692f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9d5 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.200745:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f53a00, rc 0) 00000100:00100000:0.0:1788619931.201007:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571701376:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3435us (3723us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619931.201017:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1530 00000100:00100000:2.0:1788619931.204479:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619931.204560:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571701504 00000100:00100000:1.0:1788619931.204570:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571701504 00000100:00100000:1.0:1788619931.204644:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1252 00000100:00100000:1.0:1788619931.204653:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571701504:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619931.204670:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619931.204695:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003b84bbc9/0x1cd7fe5368736936 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d9dc expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619931.204751:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c71f1c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589251200:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619931.204827:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccc00 going to sleep for 11 seconds 00002000:00010000:1.0:1788619931.207796:0:6696:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime: 1788619930 -> 1788619931 00002000:00010000:1.0:1788619931.207817:0:6696:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime: 1788619930 -> 1788619931 00000100:00100000:1.0:1788619931.207843:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c71f1c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589251200:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619931.207881:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003b84bbc9/0x1cd7fe5368736936 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9dc expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.207916:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b84bbc9/0x1cd7fe5368736936 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9dc expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.207931:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683de00, rc 0) 00000100:00100000:1.0:1788619931.208126:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571701504:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3472us (3653us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619931.208135:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1252 00000100:00100000:2.0:1788619931.212297:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619931.212506:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571701632 00000100:00100000:3.0:1788619931.212516:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571701632 00000100:00100000:3.0:1788619931.212581:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1253 00000100:00100000:3.0:1788619931.212593:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571701632:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619931.212610:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619931.212627:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e6a43d0/0x1cd7fe536873693d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d9e3 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619931.212691:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c4a4d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589251328:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619931.212844:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c780 going to sleep for 11 seconds 00000100:00100000:3.0:1788619931.216206:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c4a4d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589251328:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619931.216243:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005e6a43d0/0x1cd7fe536873693d lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9e3 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619931.216267:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e6a43d0/0x1cd7fe536873693d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9e3 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619931.216279:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d6a00, rc 0) 00000100:00100000:3.0:1788619931.216465:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571701632:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3874us (4177us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619931.216475:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1253 00000100:00100000:1.0:1788619931.220179:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619931.220281:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571701888 00000100:00100000:0.0:1788619931.220292:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571701888 00000100:00100000:1.0:1788619931.220618:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1531 00000100:00100000:1.0:1788619931.220630:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571701888:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:1.0:1788619931.220758:0:11397:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619931.220783:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52200/0x1cd7fe53687368cd lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d973 expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.221037:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619931.221043:0:11397:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788619931.221300:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368736944 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.221315:0:11397:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f53c00/0x1cd7fe53687368bf lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d96c expref: 8 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.223486:0:11397:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368736944 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.223561:0:11397:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f53c00/0x1cd7fe53687368bf lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d96c expref: 8 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.223583:0:11397:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f53c00/0x1cd7fe53687368bf lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d96c expref: 8 pid: 5845 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619931.223600:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c71f1f80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589251456:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619931.225616:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccc00 going to sleep for 11 seconds 00000100:00100000:1.0:1788619931.226441:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c71f1f80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589251456:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619931.226470:0:11397:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368736944 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619931.229624:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619931.229707:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571702016 00000100:00100000:0.0:1788619931.229716:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571702016 00010000:00010000:0.0:1788619931.229762:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f53c00/0x1cd7fe53687368bf lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d96c expref: 9 pid: 5845 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619931.229894:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 503 00000100:00100000:1.0:1788619931.229918:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571702016:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619931.229933:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619931.229941:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f53c00/0x1cd7fe53687368bf lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d96c expref: 9 pid: 5845 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.229954:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f53c00/0x1cd7fe53687368bf lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d96c expref: 9 pid: 5845 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.229978:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f53c00/0x1cd7fe53687368bf lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d96c expref: 9 pid: 5845 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.229994:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619931.230000:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368736944 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.230009:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368736944 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.230016:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368736944 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.230049:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619931.230098:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+8:2355:x1875503571702016:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 195us (483us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619931.230106:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 503 00010000:00010000:2.0:1788619931.230171:0:11397:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368736944 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619931.230178:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368736944 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788619931.230292:0:11397:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967829 00000004:00010000:2.0:1788619931.230375:0:11397:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 828664 -> 828664 00000004:00010000:2.0:1788619931.230379:0:11397:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619931 -> 1788619931 00000004:00010000:2.0:1788619931.230382:0:11397:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619930 -> 1788619930 00000004:00010000:2.0:1788619931.230384:0:11397:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619931 -> 1788619931 00000004:00010000:2.0:1788619931.230387:0:11397:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1544 -> 1544 00000004:00010000:2.0:1788619931.230393:0:11397:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000ff0ad605 reply state 00000000a6009482 transno 4294967830 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368736944 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619931.230401:0:11397:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788619931.230467:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+16:14908:x1875503571701888:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 9839us (10297us total) trans 4294967830 rc 0/0 00000100:00100000:2.0:1788619931.230474:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1531 00000020:00080000:2.0:1788619931.230478:0:11397:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788619931.236128:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368736944 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.236148:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368736944 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.236157:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368736944 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.236168:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c683de00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788619931.236174:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368736944 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.236183:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368736944 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.236200:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619931.236204:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788619931.236209:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368736944 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.236216:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368736944 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619931.238166:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619931.238307:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571702144 00000100:00100000:0.0:1788619931.238318:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571702144 00000100:00100000:0.0:1788619931.238407:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1254 00000100:00100000:0.0:1788619931.238417:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:14908:x1875503571702144:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619931.238435:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619931.238453:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001cfa5e89/0x1cd7fe536873694b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d9ea expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.238486:0:6698:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000000de8d07a/0x1cd7fe53687368b1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d965 expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.238497:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(9 12) 00010000:00010000:0.0:1788619931.238502:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(10 12) 00010000:00010000:0.0:1788619931.238509:0:6698:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001cfa5e89/0x1cd7fe536873694b lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9ea expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.238554:0:6698:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000000de8d07a/0x1cd7fe53687368b1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d965 expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.238570:0:6698:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000000de8d07a/0x1cd7fe53687368b1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d965 expref: 6 pid: 6696 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619931.238583:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b36300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589251584:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619931.238652:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78e6e803c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619931.244643:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619931.244770:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571702272 00000100:00100000:1.0:1788619931.244780:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571702272 00010000:00010000:1.0:1788619931.244833:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000000de8d07a/0x1cd7fe53687368b1 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d965 expref: 7 pid: 6696 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619931.244876:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 504 00000100:00100000:1.0:1788619931.244886:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571702272:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619931.245104:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619931.245133:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000000de8d07a/0x1cd7fe53687368b1 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d965 expref: 7 pid: 6696 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.245149:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000000de8d07a/0x1cd7fe53687368b1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d965 expref: 7 pid: 6696 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.245176:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(10 12) 00010000:00010000:1.0:1788619931.245181:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(10 12) 00010000:00010000:1.0:1788619931.245189:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000001cfa5e89/0x1cd7fe536873694b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9ea expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.245199:0:13330:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001cfa5e89/0x1cd7fe536873694b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9ea expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.245237:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000001cfa5e89/0x1cd7fe536873694b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9ea expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619931.245250:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c71f0a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589251712:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619931.245352:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccc00 going to sleep for 11 seconds 00000100:00100000:1.0:1788619931.247705:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c71f0a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589251712:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619931.247733:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619931.247789:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571702272:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 2903us (3157us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619931.247800:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 504 00010000:00010000:0.0:1788619931.249445:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000de8d07a/0x1cd7fe53687368b1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d965 expref: 6 pid: 6696 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619931.249470:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b36300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589251584:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619931.249503:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001cfa5e89/0x1cd7fe536873694b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9ea expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.249520:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f53000, rc 0) 00000100:00100000:1.0:1788619931.249718:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:14908:x1875503571702144:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 11300us (11557us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619931.249728:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1254 00000100:00100000:2.0:1788619931.253038:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619931.253117:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619931.253195:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571702528 00000100:00100000:1.0:1788619931.253204:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571702528 00002000:00010000:1.0:1788619931.253250:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:1.0:1788619931.253255:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c501d500 with x1875503571702528 ext(0->18446744073709551615) 00002000:00010000:1.0:1788619931.253267:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c501d500. 00000100:00100000:1.0:1788619931.253304:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 729 00000100:00100000:1.0:1788619931.253316:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571702528:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:1.0:1788619931.253642:0:6702:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1177566 -> 0 00002000:00010000:1.0:1788619931.253646:0:6702:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619931 -> 1788619931 00002000:00010000:1.0:1788619931.253649:0:6702:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619930 -> 1788619930 00002000:00010000:1.0:1788619931.253651:0:6702:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619931 -> 1788619931 00002000:00010000:1.0:1788619931.253654:0:6702:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 263 -> 263 00000100:00100000:1.0:1788619931.253768:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571702528:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 453us (650us total) trans 4294967512 rc 0/0 00000100:00100000:1.0:1788619931.253776:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 729 00002000:00010000:1.0:1788619931.253780:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:1.0:1788619931.253783:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c501d500 with x1875503571702528 ext(0->18446744073709551615) 00002000:00010000:1.0:1788619931.253788:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c501d500. 00000100:00100000:0.0:1788619931.254121:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571702400 00000100:00100000:0.0:1788619931.254127:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571702400 00002000:00010000:0.0:1788619931.254150:0:5857:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:0.0:1788619931.254155:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c501ce00 with x1875503571702400 00000100:00100000:0.0:1788619931.254218:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 933 00000100:00100000:0.0:1788619931.254227:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571702400:12345-192.168.201.5@tcp:10:fsx.0 00000004:00010000:0.0:1788619931.254386:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 828664 -> 999842 00000004:00010000:0.0:1788619931.254390:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619931 -> 1788619931 00000004:00010000:0.0:1788619931.254394:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619930 -> 1788619930 00000004:00010000:0.0:1788619931.254397:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619931 -> 1788619931 00000004:00010000:0.0:1788619931.254400:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1544 -> 1544 00000100:00100000:0.0:1788619931.254431:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571702400:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 205us (1412us total) trans 4294967831 rc 0/0 00000100:00100000:0.0:1788619931.254439:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 933 00002000:00010000:0.0:1788619931.254443:0:5857:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:0.0:1788619931.254445:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c501ce00 with x1875503571702400 00000100:00100000:3.0:1788619931.257172:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619931.257499:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571702656 00000100:00100000:1.0:1788619931.257509:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571702656 00000100:00100000:1.0:1788619931.257576:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1532 00000100:00100000:1.0:1788619931.257586:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:14908:x1875503571702656:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619931.257609:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619931.257630:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368736952 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d9f1 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.257946:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe5368736959 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.257963:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe5368736959 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.257975:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe5368736959 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788619931.258002:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 999842 00000004:00200000:1.0:1788619931.258006:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788619931.258023:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe5368736959 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.258050:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368736952 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9f1 expref: 18 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.258069:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe5368736959 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9f1 expref: 17 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.258078:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619931.258081:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788619931.258086:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d800, rc 0) 00000100:00100000:1.0:1788619931.258186:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+17:14908:x1875503571702656:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 601us (1023us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619931.258194:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1532 00000100:00100000:3.0:1788619931.263941:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619931.264096:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571702784 00000100:00100000:2.0:1788619931.264105:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571702784 00000100:00100000:2.0:1788619931.264157:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1255 00000100:00100000:2.0:1788619931.264167:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:14908:x1875503571702784:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619931.264184:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619931.264201:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c138a1e8/0x1cd7fe5368736960 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d9f8 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619931.264232:0:7654:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000001cfa5e89/0x1cd7fe536873694b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9ea expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619931.264242:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(10 12) 00010000:00010000:2.0:1788619931.264247:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(11 12) 00010000:00010000:2.0:1788619931.264252:0:7654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c138a1e8/0x1cd7fe5368736960 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1953792->1990655] (req 1953792->1990655) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9f8 expref: 7 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619931.264283:0:7654:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000001cfa5e89/0x1cd7fe536873694b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9ea expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619931.264296:0:7654:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000001cfa5e89/0x1cd7fe536873694b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9ea expref: 6 pid: 6698 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619931.264310:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589251840:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619931.264415:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164a80 going to sleep for 11 seconds 00000100:00100000:2.0:1788619931.267162:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619931.267248:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589251840:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619931.267282:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c138a1e8/0x1cd7fe5368736960 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1953792->1990655] (req 1953792->1990655) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9f8 expref: 7 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.267303:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f51600, rc 0) 00000100:00100000:0.0:1788619931.267381:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+7:14908:x1875503571702784:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3215us (3450us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619931.267390:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1255 00000100:00100000:3.0:1788619931.267452:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571702912 00000100:00100000:3.0:1788619931.267458:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571702912 00010000:00010000:3.0:1788619931.267487:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000001cfa5e89/0x1cd7fe536873694b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9ea expref: 7 pid: 6698 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619931.267523:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 505 00000100:00100000:3.0:1788619931.267531:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571702912:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619931.267553:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619931.267575:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000001cfa5e89/0x1cd7fe536873694b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9ea expref: 7 pid: 6698 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619931.267586:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000001cfa5e89/0x1cd7fe536873694b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9ea expref: 7 pid: 6698 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619931.267609:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001cfa5e89/0x1cd7fe536873694b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9ea expref: 7 pid: 6698 timeout: 523 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619931.267623:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(11 12) 00010000:00010000:3.0:1788619931.267627:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(11 12) 00010000:00010000:3.0:1788619931.267635:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000c138a1e8/0x1cd7fe5368736960 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1953792->1990655) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9f8 expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619931.267644:0:5829:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c138a1e8/0x1cd7fe5368736960 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1953792->1990655) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9f8 expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619931.267686:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000c138a1e8/0x1cd7fe5368736960 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1953792->1990655) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9f8 expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619931.267699:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c4a4f800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1875503589251968:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619931.267750:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c780 going to sleep for 11 seconds 00000100:00100000:3.0:1788619931.271875:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c4a4f800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1875503589251968:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619931.271900:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619931.271948:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571702912:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 4416us (4797us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619931.271959:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 505 00000100:00100000:2.0:1788619931.276504:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619931.276629:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571703040 00000100:00100000:1.0:1788619931.276638:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571703040 00000100:00100000:1.0:1788619931.276732:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1256 00000100:00100000:1.0:1788619931.276743:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571703040:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619931.277150:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619931.277170:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe5368736967 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1d9ff expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619931.277232:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c71f0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589252096:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619931.277401:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccc00 going to sleep for 11 seconds 00002000:00010000:1.0:1788619931.280632:0:6696:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size: 0 -> 1987168 00000100:00100000:1.0:1788619931.280661:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c71f0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589252096:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619931.280763:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe5368736967 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9ff expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.280789:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe5368736967 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9ff expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.280918:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683ca00, rc 0) 00000100:00100000:1.0:1788619931.281015:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571703040:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4273us (4521us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619931.281025:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1256 00000100:00100000:0.0:1788619931.286576:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619931.286668:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571703168 00000100:00100000:3.0:1788619931.286741:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571703168 00000100:00100000:3.0:1788619931.286949:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1257 00000100:00100000:3.0:1788619931.286961:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b36300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571703168:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619931.286980:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619931.286998:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002a635b6d/0x1cd7fe536873696e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1da06 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619931.287060:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c4a4f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589252224:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619931.287144:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c780 going to sleep for 11 seconds 00000100:00100000:3.0:1788619931.289587:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c4a4f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589252224:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619931.289624:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002a635b6d/0x1cd7fe536873696e lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da06 expref: 7 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619931.289648:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002a635b6d/0x1cd7fe536873696e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da06 expref: 7 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619931.289662:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d7e00, rc 0) 00000100:00100000:3.0:1788619931.289819:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b36300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571703168:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2859us (3252us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619931.289829:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1257 00000100:00100000:0.0:1788619931.296323:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619931.296428:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571703296 00000100:00100000:3.0:1788619931.296439:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571703296 00002000:00100000:3.0:1788619931.296483:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1249280->1253375) req@ffff8e77c6b36680 x1875503571703296/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:27/0 lens 488/0 e 0 to 0 dl 1788619942 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619931.296494:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6b36680 with x1875503571703296 ext(1249280->1253375) 00002000:00010000:3.0:1788619931.296500:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6b36680 00000100:00100000:3.0:1788619931.296525:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 730 00000100:00100000:3.0:1788619931.296543:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b36680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571703296:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619931.296835:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b36680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571703296:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 301us (521us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619931.296843:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 730 00002000:00100000:3.0:1788619931.296850:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1249280->1253375) req@ffff8e77c6b36680 x1875503571703296/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:27/0 lens 488/440 e 0 to 0 dl 1788619942 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619931.296859:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6b36680 with x1875503571703296 ext(1249280->1253375) 00002000:00010000:3.0:1788619931.296862:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6b36680 00000100:00100000:1.0:1788619931.299777:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619931.300148:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571703424 00000100:00100000:0.0:1788619931.300159:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571703424 00002000:00100000:0.0:1788619931.300229:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1253376->1257471) req@ffff8e77c71f0a80 x1875503571703424/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:27/0 lens 488/0 e 0 to 0 dl 1788619942 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619931.300240:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c71f0a80 with x1875503571703424 ext(1253376->1257471) 00002000:00010000:0.0:1788619931.300247:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c71f0a80 00000100:00100000:0.0:1788619931.300320:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 731 00000100:00100000:0.0:1788619931.300368:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571703424:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619931.300650:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571703424:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 293us (886us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619931.300657:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 731 00002000:00100000:0.0:1788619931.300664:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1253376->1257471) req@ffff8e77c71f0a80 x1875503571703424/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:27/0 lens 488/440 e 0 to 0 dl 1788619942 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619931.300790:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c71f0a80 with x1875503571703424 ext(1253376->1257471) 00002000:00010000:0.0:1788619931.300795:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c71f0a80 00000100:00100000:3.0:1788619931.303757:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619931.304295:0:13347:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571703552 00000100:00100000:2.0:1788619931.304305:0:13347:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571703552 00002000:00100000:2.0:1788619931.304383:0:13347:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x4:0x0] (1257472->1261567) req@ffff8e78c4a4f800 x1875503571703552/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:27/0 lens 488/0 e 0 to 0 dl 1788619942 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619931.304395:0:13347:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c4a4f800 with x1875503571703552 ext(1257472->1261567) 00002000:00010000:2.0:1788619931.304401:0:13347:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c4a4f800 00000100:00100000:2.0:1788619931.304427:0:13347:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 732 00000100:00100000:2.0:1788619931.304437:0:13347:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571703552:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619931.305082:0:13347:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571703552:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 643us (1332us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619931.305091:0:13347:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 732 00002000:00100000:2.0:1788619931.305098:0:13347:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x4:0x0] (1257472->1261567) req@ffff8e78c4a4f800 x1875503571703552/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:27/0 lens 488/440 e 0 to 0 dl 1788619942 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619931.305107:0:13347:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c4a4f800 with x1875503571703552 ext(1257472->1261567) 00002000:00010000:2.0:1788619931.305111:0:13347:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c4a4f800 00000100:00100000:1.0:1788619931.308448:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619931.308544:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571703680 00000100:00100000:3.0:1788619931.308554:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571703680 00002000:00100000:3.0:1788619931.308600:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1261568->1265663) req@ffff8e78df25d450 x1875503571703680/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:27/0 lens 488/0 e 0 to 0 dl 1788619942 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619931.308611:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df25d450 with x1875503571703680 ext(1261568->1265663) 00002000:00010000:3.0:1788619931.308617:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df25d450 00000100:00100000:3.0:1788619931.308688:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 733 00000100:00100000:3.0:1788619931.308699:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df25d450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571703680:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619931.309133:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df25d450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571703680:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 433us (692us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619931.309142:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 733 00002000:00100000:3.0:1788619931.309149:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1261568->1265663) req@ffff8e78df25d450 x1875503571703680/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:27/0 lens 488/440 e 0 to 0 dl 1788619942 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619931.309159:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df25d450 with x1875503571703680 ext(1261568->1265663) 00002000:00010000:3.0:1788619931.309163:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df25d450 00000100:00100000:0.0:1788619931.314318:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619931.314500:0:13347:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571703808 00000100:00100000:2.0:1788619931.314511:0:13347:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571703808 00002000:00100000:2.0:1788619931.314573:0:13347:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x4:0x0] (1265664->1269759) req@ffff8e77c6b34a80 x1875503571703808/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:27/0 lens 488/0 e 0 to 0 dl 1788619942 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619931.314584:0:13347:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6b34a80 with x1875503571703808 ext(1265664->1269759) 00002000:00010000:2.0:1788619931.314589:0:13347:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6b34a80 00000100:00100000:2.0:1788619931.314622:0:13347:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 734 00000100:00100000:2.0:1788619931.314632:0:13347:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b34a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571703808:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619931.315052:0:13347:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b34a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571703808:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 420us (742us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619931.315060:0:13347:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 734 00002000:00100000:2.0:1788619931.315067:0:13347:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x4:0x0] (1265664->1269759) req@ffff8e77c6b34a80 x1875503571703808/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:27/0 lens 488/440 e 0 to 0 dl 1788619942 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619931.315076:0:13347:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6b34a80 with x1875503571703808 ext(1265664->1269759) 00002000:00010000:2.0:1788619931.315080:0:13347:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6b34a80 00000100:00100000:2.0:1788619931.318131:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619931.318229:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571703936 00000100:00100000:0.0:1788619931.318241:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571703936 00002000:00100000:0.0:1788619931.318293:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1269760->1273855) req@ffff8e77c501d180 x1875503571703936/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:27/0 lens 488/0 e 0 to 0 dl 1788619942 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619931.318306:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c501d180 with x1875503571703936 ext(1269760->1273855) 00002000:00010000:0.0:1788619931.318311:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c501d180 00000100:00100000:0.0:1788619931.318375:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 735 00000100:00100000:0.0:1788619931.318385:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2352:x1875503571703936:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619931.318631:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2352:x1875503571703936:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 247us (510us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619931.318640:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 735 00002000:00100000:0.0:1788619931.318647:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1269760->1273855) req@ffff8e77c501d180 x1875503571703936/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:27/0 lens 488/440 e 0 to 0 dl 1788619942 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619931.318663:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c501d180 with x1875503571703936 ext(1269760->1273855) 00002000:00010000:0.0:1788619931.318666:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c501d180 00000100:00100000:1.0:1788619931.322109:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619931.322206:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571704064 00000100:00100000:2.0:1788619931.322217:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571704064 00000100:00100000:2.0:1788619931.322272:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1258 00000100:00100000:2.0:1788619931.322282:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571704064:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619931.322301:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619931.322317:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000075c8d00b/0x1cd7fe5368736975 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1da0d expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619931.322420:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589252352:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619931.322500:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164600 going to sleep for 11 seconds 00002000:00010000:2.0:1788619931.325113:0:6698:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619930 -> 1788619931 00000100:00100000:2.0:1788619931.325149:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589252352:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619931.325192:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000075c8d00b/0x1cd7fe5368736975 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da0d expref: 7 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619931.325220:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000075c8d00b/0x1cd7fe5368736975 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da0d expref: 7 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619931.325234:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f50800, rc 0) 00000100:00100000:2.0:1788619931.325292:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571704064:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3011us (3193us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619931.325302:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1258 00000100:00100000:3.0:1788619931.328984:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619931.329188:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571704192 00000100:00100000:0.0:1788619931.329199:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571704192 00000100:00100000:0.0:1788619931.329265:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1259 00000100:00100000:0.0:1788619931.329276:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571704192:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619931.329294:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619931.329315:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d87d0241/0x1cd7fe536873697c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1da14 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619931.329420:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b36a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589252480:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619931.329496:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78e6e803c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619931.333416:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b36a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589252480:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619931.333465:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d87d0241/0x1cd7fe536873697c lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da14 expref: 7 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.333492:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d87d0241/0x1cd7fe536873697c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da14 expref: 7 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.333506:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f52e00, rc 0) 00000100:00100000:0.0:1788619931.333573:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571704192:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4298us (4599us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619931.333582:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1259 00000100:00100000:1.0:1788619931.336848:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619931.336929:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571704320 00000100:00100000:0.0:1788619931.336939:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571704320 00000100:00100000:0.0:1788619931.336997:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1260 00000100:00100000:0.0:1788619931.337007:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571704320:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619931.337025:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619931.337041:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001cfa5e89/0x1cd7fe5368736983 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1da1b expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619931.337099:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b36a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589252608:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619931.337180:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78e6e803c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619931.340632:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b36a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589252608:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619931.340683:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001cfa5e89/0x1cd7fe5368736983 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da1b expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.340960:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001cfa5e89/0x1cd7fe5368736983 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da1b expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.340976:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f53000, rc 0) 00000100:00100000:0.0:1788619931.341032:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571704320:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4026us (4196us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619931.341041:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1260 00000100:00100000:1.0:1788619931.347893:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619931.347995:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571704448 00000100:00100000:3.0:1788619931.348004:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571704448 00000100:00100000:3.0:1788619931.348078:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1261 00000100:00100000:3.0:1788619931.348086:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571704448:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619931.348103:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619931.348120:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e6a43d0/0x1cd7fe536873698a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1da22 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619931.348167:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c4a4dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589252736:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619931.348236:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c780 going to sleep for 11 seconds 00000100:00100000:3.0:1788619931.350975:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c4a4dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589252736:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619931.351017:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005e6a43d0/0x1cd7fe536873698a lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da22 expref: 7 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619931.351042:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e6a43d0/0x1cd7fe536873698a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da22 expref: 7 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619931.351056:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d6a00, rc 0) 00000100:00100000:3.0:1788619931.351114:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571704448:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3027us (3310us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619931.351122:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1261 00000100:00100000:2.0:1788619931.361001:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619931.361270:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571704576 00000100:00100000:3.0:1788619931.361281:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571704576 00000100:00100000:3.0:1788619931.361363:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1262 00000100:00100000:3.0:1788619931.361375:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571704576:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619931.361393:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619931.361411:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002a635b6d/0x1cd7fe5368736991 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1da29 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619931.361469:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c4a4dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589252864:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619931.361558:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c780 going to sleep for 11 seconds 00000100:00100000:3.0:1788619931.365119:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c4a4dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589252864:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619931.365161:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002a635b6d/0x1cd7fe5368736991 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da29 expref: 7 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619931.365186:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002a635b6d/0x1cd7fe5368736991 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da29 expref: 7 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619931.365199:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d7e00, rc 0) 00000100:00100000:3.0:1788619931.365266:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571704576:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3893us (4277us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619931.365276:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1262 00000100:00100000:2.0:1788619931.369226:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619931.369791:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571704704 00000100:00100000:0.0:1788619931.369802:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571704704 00000100:00100000:0.0:1788619931.369865:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1263 00000100:00100000:0.0:1788619931.369875:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900987850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571704704:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619931.369894:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619931.369913:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001cfa5e89/0x1cd7fe5368736998 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1da30 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619931.369980:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b36a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589252992:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619931.370114:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78e6e803c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619931.377754:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b36a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589252992:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619931.377796:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001cfa5e89/0x1cd7fe5368736998 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da30 expref: 7 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.377821:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001cfa5e89/0x1cd7fe5368736998 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da30 expref: 7 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.377835:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f53000, rc 0) 00000100:00100000:0.0:1788619931.377901:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900987850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571704704:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8026us (8686us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619931.377910:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1263 00000100:00100000:1.0:1788619931.383661:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619931.383759:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571704832 00000100:00100000:0.0:1788619931.383769:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571704832 00000100:00100000:0.0:1788619931.383841:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1533 00000100:00100000:0.0:1788619931.383851:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+8:2355:x1875503571704832:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619931.383873:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619931.383894:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52e00/0x1cd7fe536873699f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1da37 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.383932:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f53600/0x1cd7fe536873690c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9b2 expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619931.383979:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b35c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589253120:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619931.384068:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78e6e803c0 going to sleep for 11 seconds 00000004:00010000:0.0:1788619931.392179:0:5845:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619930 -> 1788619931 00000100:00100000:0.0:1788619931.392213:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b35c00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589253120:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619931.392238:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 999842 00010000:00010000:0.0:1788619931.392257:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52e00/0x1cd7fe536873699f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da37 expref: 9 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.392277:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619931.392281:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619931.392287:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52e00/0x1cd7fe536873699f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da37 expref: 9 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.392304:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f52e00, rc 0) 00000100:00100000:0.0:1788619931.392404:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+8:2355:x1875503571704832:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8553us (8753us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619931.392413:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1533 00000100:00100000:0.0:1788619931.395040:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619931.395139:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571704960 00000100:00100000:2.0:1788619931.395153:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571704960 00000100:00100000:2.0:1788619931.395228:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1534 00000100:00100000:2.0:1788619931.395240:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b35c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+8:14908:x1875503571704960:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619931.395261:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619931.395283:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51000/0x1cd7fe53687369a6 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1da3e expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619931.395314:0:11397:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f53600/0x1cd7fe536873690c lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9b2 expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619931.395362:0:11397:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51000/0x1cd7fe53687369a6 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da3e expref: 9 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619931.395399:0:11397:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f53600/0x1cd7fe536873690c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9b2 expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619931.395412:0:11397:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f53600/0x1cd7fe536873690c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9b2 expref: 16 pid: 5845 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619931.395426:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589253248:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619931.395504:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164600 going to sleep for 11 seconds 00000100:00100000:1.0:1788619931.403156:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619931.403232:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589253248:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619931.403268:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51000/0x1cd7fe53687369a6 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da3e expref: 9 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619931.403288:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619931.403292:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619931.403296:0:11397:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51000/0x1cd7fe53687369a6 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da3e expref: 9 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619931.403307:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f51000, rc 0) 00000100:00100000:2.0:1788619931.403385:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b35c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571704960:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8147us (8356us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619931.403394:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1534 00000100:00100000:3.0:1788619931.408453:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571705088 00000100:00100000:3.0:1788619931.408469:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571705088 00010000:00010000:2.0:1788619931.409942:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f53600/0x1cd7fe536873690c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9b2 expref: 17 pid: 5845 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619931.410000:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 506 00000100:00100000:2.0:1788619931.410012:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2354:x1875503571705088:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619931.410030:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619931.410055:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f53600/0x1cd7fe536873690c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9b2 expref: 17 pid: 5845 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619931.410069:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f53600/0x1cd7fe536873690c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9b2 expref: 17 pid: 5845 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619931.410107:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f53600/0x1cd7fe536873690c lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9b2 expref: 17 pid: 5845 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619931.410126:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619931.410133:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51000/0x1cd7fe53687369a6 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da3e expref: 8 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619931.410146:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51000/0x1cd7fe53687369a6 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da3e expref: 8 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619931.410158:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51000/0x1cd7fe53687369a6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da3e expref: 8 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619931.410194:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51000/0x1cd7fe53687369a6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da3e expref: 8 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619931.410208:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501ea00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589253376:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619931.410304:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164600 going to sleep for 11 seconds 00000100:00100000:2.0:1788619931.418945:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501ea00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589253376:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619931.418973:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619931.419036:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571705088:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 9026us (15888us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619931.419048:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 506 00000100:00100000:3.0:1788619931.424978:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619931.425153:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571705216 00000100:00100000:1.0:1788619931.425164:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571705216 00002000:00100000:1.0:1788619931.425216:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1953791) req@ffff8e78c4a4dc00 x1875503571705216/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:27/0 lens 504/0 e 0 to 0 dl 1788619942 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619931.425227:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c4a4dc00 with x1875503571705216 ext(1048576->1953791) 00002000:00010000:1.0:1788619931.425232:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c4a4dc00 00000100:00100000:1.0:1788619931.425266:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 736 00000100:00100000:1.0:1788619931.425275:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571705216:12345-192.168.201.5@tcp:3:fsx.0 00010000:00010000:3.0:1788619931.426995:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788619931.434634:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619931.438174:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571705344 00000100:00100000:1.0:1788619931.438192:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571705344 00002000:00100000:1.0:1788619931.438254:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b36300 x1875503571705344/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:27/0 lens 488/0 e 0 to 0 dl 1788619942 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619931.438265:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b36300 with x1875503571705344 00000100:00100000:1.0:1788619931.438312:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 934 00000100:00100000:1.0:1788619931.438322:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b36300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571705344:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619931.440155:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571705216:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 14868us (15175us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619931.440170:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 736 00002000:00100000:3.0:1788619931.440182:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1953791) req@ffff8e78c4a4dc00 x1875503571705216/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:27/0 lens 504/440 e 0 to 0 dl 1788619942 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619931.440193:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c4a4dc00 with x1875503571705216 ext(1048576->1953791) 00002000:00010000:3.0:1788619931.440199:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c4a4dc00 00000100:00100000:0.0:1788619931.473649:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b36300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571705344:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 35316us (39015us total) trans 0 rc 1003520/1003520 00000100:00100000:0.0:1788619931.473663:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 934 00002000:00100000:0.0:1788619931.473677:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6b36300 x1875503571705344/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:27/0 lens 488/440 e 0 to 0 dl 1788619942 ref 1 fl Complete:/600/0 rc 1003520/1003520 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619931.473689:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6b36300 with x1875503571705344 00000100:00100000:3.0:1788619931.478136:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619931.478233:0:13346:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571705472 00000100:00100000:2.0:1788619931.478244:0:13346:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571705472 00002000:00100000:2.0:1788619931.478295:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c4a4c380 x1875503571705472/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:27/0 lens 8680/0 e 0 to 0 dl 1788619942 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619931.478306:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c4a4c380 with x1875503571705472 00000100:00100000:0.0:1788619931.478502:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 935 00000100:00100000:0.0:1788619931.478515:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571705472:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:0.0:1788619931.479072:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 999842 -> 999842 00000004:00010000:0.0:1788619931.479077:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619931 -> 1788619931 00000004:00010000:0.0:1788619931.479079:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619931 -> 1788619930 00000004:00010000:0.0:1788619931.479082:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619931 -> 1788619931 00000004:00010000:0.0:1788619931.479084:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1544 -> 1544 00000100:00100000:0.0:1788619931.479244:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571705472:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 731us (1117us total) trans 4294967832 rc 0/0 00000100:00100000:0.0:1788619931.479253:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 935 00002000:00100000:0.0:1788619931.479261:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c4a4c380 x1875503571705472/t4294967832(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:27/0 lens 8680/448 e 0 to 0 dl 1788619942 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619931.479270:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c4a4c380 with x1875503571705472 00000100:00100000:2.0:1788619931.488499:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619931.488591:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571705600 00000100:00100000:1.0:1788619931.488604:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571705600 00000100:00100000:1.0:1788619931.488671:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1535 00000100:00100000:1.0:1788619931.488682:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+11:2353:x1875503571705600:12345-192.168.201.5@tcp:16:fsx.0 00000001:00080000:2.0:1788619931.564769:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967832 is committed 00000001:00080000:2.0:1788619931.564787:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967831 is committed 00000100:00100000:0.0:1788619931.565240:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571705600:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 76554us (76746us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619931.565253:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1535 00000100:00100000:3.0:1788619931.568386:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619931.573076:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571705728 00000100:00100000:0.0:1788619931.573094:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571705728 00000100:00100000:0.0:1788619931.581188:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1536 00000100:00100000:0.0:1788619931.581203:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571705728:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619931.581224:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619931.581247:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f53c00/0x1cd7fe53687369ad lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1da45 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.581360:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52200/0x1cd7fe53687369b4 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.581378:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52200/0x1cd7fe53687369b4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.581389:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52200/0x1cd7fe53687369b4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619931.581417:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 999842 00000004:00200000:0.0:1788619931.581423:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619931.581441:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52200/0x1cd7fe53687369b4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.581466:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f53c00/0x1cd7fe53687369ad lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da45 expref: 11 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.581484:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52200/0x1cd7fe53687369b4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da45 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.581493:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619931.581495:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619931.581499:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f52200, rc 0) 00000100:00100000:0.0:1788619931.581569:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:14908:x1875503571705728:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 368us (13192us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619931.581576:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1536 00000100:00100000:3.0:1788619931.596616:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619931.596653:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619931.596760:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571705856 00000100:00100000:1.0:1788619931.596774:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571705856 00000100:00100000:1.0:1788619931.596858:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1537 00000100:00100000:1.0:1788619931.596868:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571705856:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619931.596890:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619931.596912:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe53687369bb lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1da4c expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.596949:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51000/0x1cd7fe53687369a6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da3e expref: 9 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619931.596991:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c71f0000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589253504:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619931.597069:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54ccc00 going to sleep for 11 seconds 00000100:00100000:0.0:1788619931.597863:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571705984 00000100:00100000:0.0:1788619931.597869:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571705984 00000100:00100000:0.0:1788619931.597902:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1264 00000100:00100000:0.0:1788619931.597910:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571705984:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619931.597921:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619931.597933:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f3df901/0x1cd7fe53687369c2 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1da53 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619931.597973:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b36a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589253632:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619931.598018:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5d2de40 going to sleep for 11 seconds 00000004:00010000:1.0:1788619931.601178:0:11397:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619930 -> 1788619931 00000100:00100000:1.0:1788619931.601211:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c71f0000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589253504:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619931.601236:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 999842 00010000:00010000:1.0:1788619931.601257:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe53687369bb lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da4c expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.601278:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619931.601281:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619931.601286:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe53687369bb lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da4c expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.601301:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683cc00, rc 0) 00000100:00100000:1.0:1788619931.601398:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571705856:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4530us (4742us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619931.601407:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1537 00000100:00100000:1.0:1788619931.603060:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b36a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589253632:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619931.603093:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002f3df901/0x1cd7fe53687369c2 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da53 expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.603114:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f3df901/0x1cd7fe53687369c2 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da53 expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.603125:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f53a00, rc 0) 00000100:00100000:1.0:1788619931.603176:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571705984:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5266us (6570us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619931.603183:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1264 00000100:00100000:3.0:1788619931.608779:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619931.609027:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571706112 00000100:00100000:2.0:1788619931.609037:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571706112 00000100:00100000:2.0:1788619931.609100:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1265 00000100:00100000:2.0:1788619931.609110:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:14908:x1875503571706112:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619931.609128:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619931.609146:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001dc9de0/0x1cd7fe53687369c9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1da5a expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619931.609181:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000c138a1e8/0x1cd7fe5368736960 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1953792->1990655) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9f8 expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619931.609191:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=1s v=(11 12) 00010000:00010000:2.0:1788619931.609195:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(1 11) 00010000:00010000:2.0:1788619931.609202:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001dc9de0/0x1cd7fe53687369c9 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1429504->1433599] (req 1429504->1433599) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da5a expref: 7 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619931.609234:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000c138a1e8/0x1cd7fe5368736960 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1953792->1990655) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9f8 expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619931.609249:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000c138a1e8/0x1cd7fe5368736960 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1953792->1990655) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9f8 expref: 6 pid: 7654 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619931.609264:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589253760:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619931.609376:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164000 going to sleep for 11 seconds 00000100:00100000:2.0:1788619931.615095:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619931.615178:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589253760:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619931.615211:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000001dc9de0/0x1cd7fe53687369c9 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1429504->1433599] (req 1429504->1433599) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da5a expref: 7 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619931.615230:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6a00, rc 0) 00000100:00100000:3.0:1788619931.615288:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:14908:x1875503571706112:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6179us (6520us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619931.615296:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1265 00000100:00100000:0.0:1788619931.616826:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571706240 00000100:00100000:0.0:1788619931.616838:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571706240 00002000:00100000:0.0:1788619931.616887:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1953792->1987167) req@ffff8e77c501c700 x1875503571706240/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:27/0 lens 488/0 e 0 to 0 dl 1788619942 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619931.616897:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c501c700 with x1875503571706240 ext(1953792->1987167) 00010000:00010000:0.0:1788619931.616905:0:6702:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000c138a1e8/0x1cd7fe5368736960 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1953792->1990655) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9f8 expref: 7 pid: 7654 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.616921:0:6702:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000c138a1e8/0x1cd7fe5368736960 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1953792->1990655) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9f8 expref: 7 pid: 7654 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788619931.616929:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff8e77c501c700 00000100:00100000:0.0:1788619931.616965:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 25 00000100:00100000:0.0:1788619931.616975:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571706240:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:0.0:1788619931.620545:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571706240:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 3550us (5437us total) trans 4294967513 rc 0/0 00000100:00100000:0.0:1788619931.620560:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 25 00002000:00100000:0.0:1788619931.620573:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1953792->1987167) req@ffff8e77c501c700 x1875503571706240/t4294967513(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:27/0 lens 488/448 e 0 to 0 dl 1788619942 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619931.620585:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c501c700 with x1875503571706240 ext(1953792->1987167) 00010000:00010000:0.0:1788619931.620597:0:6702:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000c138a1e8/0x1cd7fe5368736960 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1953792->1990655) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9f8 expref: 9 pid: 7654 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.620617:0:6702:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000c138a1e8/0x1cd7fe5368736960 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1953792->1990655) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9f8 expref: 9 pid: 7654 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788619931.620626:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff8e77c501c700 00000100:00100000:2.0:1788619931.629478:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619931.629684:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571706368 00000100:00100000:0.0:1788619931.629695:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571706368 00010000:00010000:0.0:1788619931.629747:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000c138a1e8/0x1cd7fe5368736960 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1953792->1990655) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9f8 expref: 9 pid: 7654 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619931.629791:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 507 00000100:00100000:0.0:1788619931.629813:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571706368:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619931.629831:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619931.629860:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000c138a1e8/0x1cd7fe5368736960 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1953792->1990655) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9f8 expref: 9 pid: 7654 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.629874:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000c138a1e8/0x1cd7fe5368736960 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1953792->1990655) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9f8 expref: 9 pid: 7654 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.629915:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c138a1e8/0x1cd7fe5368736960 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1953792->1990655) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9f8 expref: 9 pid: 7654 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.629935:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(1 11) 00010000:00010000:0.0:1788619931.629939:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(1 11) 00010000:00010000:0.0:1788619931.629950:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000001dc9de0/0x1cd7fe53687369c9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1429504->1433599) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da5a expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.629960:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001dc9de0/0x1cd7fe53687369c9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1429504->1433599) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da5a expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.629998:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000001dc9de0/0x1cd7fe53687369c9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1429504->1433599) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da5a expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619931.630013:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b34380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589253888:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619931.630109:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5d2d3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619931.634694:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b34380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589253888:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619931.634733:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619931.634805:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+8:2352:x1875503571706368:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 4984us (5326us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619931.634816:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 507 00000100:00100000:1.0:1788619931.638138:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619931.638233:0:13347:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571706496 00000100:00100000:2.0:1788619931.638244:0:13347:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571706496 00002000:00100000:2.0:1788619931.638290:0:13347:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x4:0x0] (1429504->1433599) req@ffff8e77c71f0000 x1875503571706496/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:27/0 lens 488/0 e 0 to 0 dl 1788619942 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619931.638309:0:13347:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c71f0000 with x1875503571706496 ext(1429504->1433599) 00002000:00010000:2.0:1788619931.638315:0:13347:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c71f0000 00000100:00100000:2.0:1788619931.638383:0:13347:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 737 00000100:00100000:2.0:1788619931.638393:0:13347:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571706496:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619931.638645:0:13347:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571706496:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 252us (519us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619931.638653:0:13347:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 737 00002000:00100000:2.0:1788619931.638660:0:13347:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x4:0x0] (1429504->1433599) req@ffff8e77c71f0000 x1875503571706496/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:27/0 lens 488/4536 e 0 to 0 dl 1788619942 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619931.638670:0:13347:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c71f0000 with x1875503571706496 ext(1429504->1433599) 00002000:00010000:2.0:1788619931.638674:0:13347:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c71f0000 00000100:00100000:2.0:1788619931.642317:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619931.642700:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571706624 00000100:00100000:3.0:1788619931.642718:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571706624 00002000:00100000:3.0:1788619931.642767:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1433600->1437695) req@ffff8e77c501f100 x1875503571706624/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:27/0 lens 488/0 e 0 to 0 dl 1788619942 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619931.642779:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c501f100 with x1875503571706624 ext(1433600->1437695) 00002000:00010000:3.0:1788619931.642785:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c501f100 00000100:00100000:3.0:1788619931.642926:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 738 00000100:00100000:3.0:1788619931.642936:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571706624:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619931.643185:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571706624:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 250us (878us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619931.643193:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 738 00002000:00100000:3.0:1788619931.643199:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1433600->1437695) req@ffff8e77c501f100 x1875503571706624/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:27/0 lens 488/4536 e 0 to 0 dl 1788619942 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619931.643207:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c501f100 with x1875503571706624 ext(1433600->1437695) 00002000:00010000:3.0:1788619931.643211:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c501f100 00000100:00100000:0.0:1788619931.662178:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619931.662283:0:13347:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571706752 00000100:00100000:2.0:1788619931.662294:0:13347:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571706752 00002000:00100000:2.0:1788619931.662385:0:13347:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x4:0x0] (1437696->1441791) req@ffff8e77c6b34380 x1875503571706752/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:27/0 lens 488/0 e 0 to 0 dl 1788619942 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619931.662397:0:13347:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6b34380 with x1875503571706752 ext(1437696->1441791) 00002000:00010000:2.0:1788619931.662403:0:13347:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6b34380 00000100:00100000:2.0:1788619931.662475:0:13347:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 739 00000100:00100000:2.0:1788619931.662486:0:13347:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b34380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571706752:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619931.663767:0:13347:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b34380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571706752:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 1214us (1529us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619931.663780:0:13347:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 739 00002000:00100000:2.0:1788619931.663791:0:13347:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x4:0x0] (1437696->1441791) req@ffff8e77c6b34380 x1875503571706752/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:27/0 lens 488/4536 e 0 to 0 dl 1788619942 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619931.663926:0:13347:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6b34380 with x1875503571706752 ext(1437696->1441791) 00002000:00010000:2.0:1788619931.663931:0:13347:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6b34380 00000100:00100000:0.0:1788619931.684974:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619931.692893:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571706880 00000100:00100000:3.0:1788619931.692910:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571706880 00002000:00100000:3.0:1788619931.692974:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1441792->1445887) req@ffff8e77c6b35880 x1875503571706880/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:27/0 lens 488/0 e 0 to 0 dl 1788619942 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619931.692986:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6b35880 with x1875503571706880 ext(1441792->1445887) 00002000:00010000:3.0:1788619931.692993:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6b35880 00000100:00100000:3.0:1788619931.693022:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 740 00000100:00100000:3.0:1788619931.693031:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b35880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2353:x1875503571706880:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619931.693300:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b35880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2353:x1875503571706880:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 269us (8334us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619931.693308:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 740 00002000:00100000:3.0:1788619931.693315:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1441792->1445887) req@ffff8e77c6b35880 x1875503571706880/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:27/0 lens 488/4536 e 0 to 0 dl 1788619942 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619931.693341:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6b35880 with x1875503571706880 ext(1441792->1445887) 00002000:00010000:3.0:1788619931.693346:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6b35880 00000100:00100000:1.0:1788619931.702990:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619931.703069:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571707008 00000100:00100000:3.0:1788619931.703081:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571707008 00002000:00100000:3.0:1788619931.703135:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1445888->1449983) req@ffff8e77c71f0380 x1875503571707008/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:27/0 lens 488/0 e 0 to 0 dl 1788619942 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619931.703148:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c71f0380 with x1875503571707008 ext(1445888->1449983) 00002000:00010000:3.0:1788619931.703155:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c71f0380 00000100:00100000:3.0:1788619931.703191:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 741 00000100:00100000:3.0:1788619931.703200:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2352:x1875503571707008:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619931.703471:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2352:x1875503571707008:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 271us (490us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619931.703480:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 741 00002000:00100000:3.0:1788619931.703487:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1445888->1449983) req@ffff8e77c71f0380 x1875503571707008/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:27/0 lens 488/4536 e 0 to 0 dl 1788619942 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619931.703496:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c71f0380 with x1875503571707008 ext(1445888->1449983) 00002000:00010000:3.0:1788619931.703500:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c71f0380 00000100:00100000:2.0:1788619931.709669:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619931.709908:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571707136 00000100:00100000:0.0:1788619931.709920:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571707136 00002000:00100000:0.0:1788619931.709971:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1449984->1454079) req@ffff8e77c501ca80 x1875503571707136/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:27/0 lens 488/0 e 0 to 0 dl 1788619942 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619931.709984:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c501ca80 with x1875503571707136 ext(1449984->1454079) 00002000:00010000:0.0:1788619931.709990:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c501ca80 00000100:00100000:0.0:1788619931.710027:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 742 00000100:00100000:0.0:1788619931.710037:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2353:x1875503571707136:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619931.710402:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2353:x1875503571707136:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 363us (742us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619931.710410:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 742 00002000:00100000:0.0:1788619931.710417:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1449984->1454079) req@ffff8e77c501ca80 x1875503571707136/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:27/0 lens 488/4536 e 0 to 0 dl 1788619942 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619931.710427:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c501ca80 with x1875503571707136 ext(1449984->1454079) 00002000:00010000:0.0:1788619931.710430:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c501ca80 00000100:00100000:1.0:1788619931.723110:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619931.723204:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571707264 00000100:00100000:3.0:1788619931.723214:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571707264 00002000:00100000:3.0:1788619931.723264:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1454080->1458175) req@ffff8e77c71f0e00 x1875503571707264/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:27/0 lens 488/0 e 0 to 0 dl 1788619942 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619931.723283:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c71f0e00 with x1875503571707264 ext(1454080->1458175) 00002000:00010000:3.0:1788619931.723289:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c71f0e00 00000100:00100000:3.0:1788619931.723322:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 743 00000100:00100000:3.0:1788619931.723427:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571707264:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619931.723872:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571707264:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 447us (773us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619931.723881:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 743 00002000:00100000:3.0:1788619931.723888:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1454080->1458175) req@ffff8e77c71f0e00 x1875503571707264/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:27/0 lens 488/4536 e 0 to 0 dl 1788619942 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619931.723897:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c71f0e00 with x1875503571707264 ext(1454080->1458175) 00002000:00010000:3.0:1788619931.723901:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c71f0e00 00000100:00100000:0.0:1788619931.730382:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619931.730758:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571707392 00000100:00100000:3.0:1788619931.730768:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571707392 00002000:00100000:3.0:1788619931.730833:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1458176->1462271) req@ffff8e77c6b35f80 x1875503571707392/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:27/0 lens 488/0 e 0 to 0 dl 1788619942 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619931.730844:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6b35f80 with x1875503571707392 ext(1458176->1462271) 00002000:00010000:3.0:1788619931.730849:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6b35f80 00000100:00100000:3.0:1788619931.730882:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 744 00000100:00100000:3.0:1788619931.730891:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b35f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571707392:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619931.731137:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b35f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571707392:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 247us (821us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619931.731145:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 744 00002000:00100000:3.0:1788619931.731152:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1458176->1462271) req@ffff8e77c6b35f80 x1875503571707392/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:27/0 lens 488/4536 e 0 to 0 dl 1788619942 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619931.731161:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6b35f80 with x1875503571707392 ext(1458176->1462271) 00002000:00010000:3.0:1788619931.731165:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6b35f80 00000100:00100000:2.0:1788619931.735857:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619931.736090:0:13347:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571707520 00000100:00100000:3.0:1788619931.736100:0:13347:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571707520 00002000:00100000:3.0:1788619931.736150:0:13347:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x4:0x0] (1462272->1466367) req@ffff8e77c501d880 x1875503571707520/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:27/0 lens 488/0 e 0 to 0 dl 1788619942 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619931.736160:0:13347:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c501d880 with x1875503571707520 ext(1462272->1466367) 00002000:00010000:3.0:1788619931.736165:0:13347:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c501d880 00000100:00100000:3.0:1788619931.736199:0:13347:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 745 00000100:00100000:3.0:1788619931.736209:0:13347:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2353:x1875503571707520:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619931.736495:0:13347:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2353:x1875503571707520:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 286us (645us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619931.736503:0:13347:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 745 00002000:00100000:3.0:1788619931.736509:0:13347:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x4:0x0] (1462272->1466367) req@ffff8e77c501d880 x1875503571707520/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:27/0 lens 488/4536 e 0 to 0 dl 1788619942 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619931.736517:0:13347:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c501d880 with x1875503571707520 ext(1462272->1466367) 00002000:00010000:3.0:1788619931.736519:0:13347:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c501d880 00000100:00100000:1.0:1788619931.744675:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619931.744774:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571707648 00000100:00100000:3.0:1788619931.744785:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571707648 00002000:00100000:3.0:1788619931.744850:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1466368->1470463) req@ffff8e77c71f1880 x1875503571707648/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:27/0 lens 488/0 e 0 to 0 dl 1788619942 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619931.744862:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c71f1880 with x1875503571707648 ext(1466368->1470463) 00002000:00010000:3.0:1788619931.744868:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c71f1880 00000100:00100000:3.0:1788619931.744904:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 746 00000100:00100000:3.0:1788619931.744915:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2352:x1875503571707648:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619931.745170:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2352:x1875503571707648:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 256us (509us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619931.745177:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 746 00002000:00100000:3.0:1788619931.745184:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1466368->1470463) req@ffff8e77c71f1880 x1875503571707648/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:27/0 lens 488/4536 e 0 to 0 dl 1788619942 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619931.745192:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c71f1880 with x1875503571707648 ext(1466368->1470463) 00002000:00010000:3.0:1788619931.745195:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c71f1880 00000100:00100000:0.0:1788619931.756153:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619931.756409:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571707776 00000100:00100000:1.0:1788619931.756419:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571707776 00002000:00100000:1.0:1788619931.756470:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1470464->1474559) req@ffff8e77c6b37b80 x1875503571707776/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:27/0 lens 488/0 e 0 to 0 dl 1788619942 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619931.756482:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6b37b80 with x1875503571707776 ext(1470464->1474559) 00002000:00010000:1.0:1788619931.756488:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6b37b80 00000100:00100000:1.0:1788619931.756523:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 747 00000100:00100000:1.0:1788619931.756542:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b37b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2353:x1875503571707776:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619931.757082:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b37b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2353:x1875503571707776:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 539us (939us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619931.757090:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 747 00002000:00100000:1.0:1788619931.757097:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1470464->1474559) req@ffff8e77c6b37b80 x1875503571707776/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:27/0 lens 488/4536 e 0 to 0 dl 1788619942 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619931.757106:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6b37b80 with x1875503571707776 ext(1470464->1474559) 00002000:00010000:1.0:1788619931.757110:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6b37b80 00000100:00100000:2.0:1788619931.767298:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619931.767426:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571707904 00000100:00100000:3.0:1788619931.767438:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571707904 00002000:00100000:3.0:1788619931.767490:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1474560->1478655) req@ffff8e77c501e680 x1875503571707904/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:27/0 lens 488/0 e 0 to 0 dl 1788619942 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619931.767501:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c501e680 with x1875503571707904 ext(1474560->1478655) 00002000:00010000:3.0:1788619931.767507:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c501e680 00000100:00100000:3.0:1788619931.767542:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 748 00000100:00100000:3.0:1788619931.767552:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2352:x1875503571707904:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619931.768123:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2352:x1875503571707904:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 570us (835us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619931.768133:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 748 00002000:00100000:3.0:1788619931.768140:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1474560->1478655) req@ffff8e77c501e680 x1875503571707904/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:27/0 lens 488/4536 e 0 to 0 dl 1788619942 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619931.768150:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c501e680 with x1875503571707904 ext(1474560->1478655) 00002000:00010000:3.0:1788619931.768155:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c501e680 00000100:00100000:0.0:1788619931.777781:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619931.777948:0:13347:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571708032 00000100:00100000:3.0:1788619931.777959:0:13347:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571708032 00002000:00100000:3.0:1788619931.778011:0:13347:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x4:0x0] (1478656->1482751) req@ffff8e77c6b36a00 x1875503571708032/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:27/0 lens 488/0 e 0 to 0 dl 1788619942 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619931.778022:0:13347:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6b36a00 with x1875503571708032 ext(1478656->1482751) 00002000:00010000:3.0:1788619931.778028:0:13347:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6b36a00 00000100:00100000:3.0:1788619931.778112:0:13347:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 749 00000100:00100000:3.0:1788619931.778124:0:13347:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b36a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571708032:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619931.778414:0:13347:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b36a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571708032:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 291us (643us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619931.778421:0:13347:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 749 00002000:00100000:3.0:1788619931.778428:0:13347:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x4:0x0] (1478656->1482751) req@ffff8e77c6b36a00 x1875503571708032/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:27/0 lens 488/4536 e 0 to 0 dl 1788619942 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619931.778436:0:13347:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6b36a00 with x1875503571708032 ext(1478656->1482751) 00002000:00010000:3.0:1788619931.778439:0:13347:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6b36a00 00000100:00100000:2.0:1788619931.788691:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619931.788786:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571708160 00000100:00100000:3.0:1788619931.788808:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571708160 00002000:00100000:3.0:1788619931.788862:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1482752->1486847) req@ffff8e77c501c000 x1875503571708160/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:27/0 lens 488/0 e 0 to 0 dl 1788619942 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619931.788874:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c501c000 with x1875503571708160 ext(1482752->1486847) 00002000:00010000:3.0:1788619931.788880:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c501c000 00000100:00100000:3.0:1788619931.788914:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 750 00000100:00100000:3.0:1788619931.788924:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2353:x1875503571708160:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619931.789188:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2353:x1875503571708160:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 263us (505us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619931.789196:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 750 00002000:00100000:3.0:1788619931.789203:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1482752->1486847) req@ffff8e77c501c000 x1875503571708160/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:27/0 lens 488/4536 e 0 to 0 dl 1788619942 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619931.789212:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c501c000 with x1875503571708160 ext(1482752->1486847) 00002000:00010000:3.0:1788619931.789216:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c501c000 00000100:00100000:2.0:1788619931.796578:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619931.796690:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571708288 00000100:00100000:1.0:1788619931.796708:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571708288 00002000:00100000:1.0:1788619931.796758:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1429504->1486847) req@ffff8e77c501dc00 x1875503571708288/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:27/0 lens 488/0 e 0 to 0 dl 1788619942 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619931.796770:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c501dc00 with x1875503571708288 ext(1429504->1486847) 00002000:00010000:1.0:1788619931.796776:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c501dc00 00000100:00100000:1.0:1788619931.796905:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 751 00000100:00100000:1.0:1788619931.796917:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571708288:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:1.0:1788619931.804347:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571708288:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 7405us (7752us total) trans 4294967514 rc 0/0 00000100:00100000:1.0:1788619931.804373:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 751 00002000:00100000:1.0:1788619931.804386:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1429504->1486847) req@ffff8e77c501dc00 x1875503571708288/t4294967514(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:27/0 lens 488/448 e 0 to 0 dl 1788619942 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619931.804396:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c501dc00 with x1875503571708288 ext(1429504->1486847) 00002000:00010000:1.0:1788619931.804402:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c501dc00 00000100:00100000:2.0:1788619931.806590:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619931.806681:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571708416 00000100:00100000:3.0:1788619931.806689:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571708416 00000100:00100000:3.0:1788619931.806789:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1266 00000100:00100000:3.0:1788619931.806811:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2353:x1875503571708416:12345-192.168.201.5@tcp:16:fsx.0 00000001:00080000:2.0:1788619931.850447:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967513 is committed 00000001:00080000:2.0:1788619931.850482:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967514 is committed 00000100:00100000:0.0:1788619931.850657:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571708416:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 43843us (44071us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619931.850669:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1266 00000100:00100000:0.0:1788619931.858103:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619931.859283:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571708544 00000100:00100000:1.0:1788619931.859296:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571708544 00000100:00100000:1.0:1788619931.859394:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1538 00000100:00100000:1.0:1788619931.859408:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b35500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571708544:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619931.859432:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619931.859457:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687369d0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1da61 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.859495:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51000/0x1cd7fe53687369a6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da3e expref: 9 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619931.859551:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c71f0700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589254016:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619931.859777:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619931.874550:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c71f0700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589254016:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619931.874580:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 999842 00010000:00010000:1.0:1788619931.874600:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687369d0 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da61 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.874622:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619931.874625:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619931.874631:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687369d0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da61 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.874646:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d200, rc 0) 00000100:00100000:1.0:1788619931.874711:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b35500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571708544:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 15306us (16615us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619931.874720:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1538 00000100:00100000:2.0:1788619931.882131:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619931.882411:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571708672 00000100:00100000:3.0:1788619931.882421:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571708672 00000100:00100000:3.0:1788619931.882484:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1539 00000100:00100000:3.0:1788619931.882495:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571708672:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619931.882534:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619931.882557:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7200/0x1cd7fe53687369d7 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1da68 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619931.882595:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51000/0x1cd7fe53687369a6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da3e expref: 9 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619931.882637:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c4a4e300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589254144:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619931.882731:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c000 going to sleep for 11 seconds 00000100:00100000:2.0:1788619931.889836:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c4a4e300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589254144:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619931.889871:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 999842 00010000:00010000:2.0:1788619931.889894:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7200/0x1cd7fe53687369d7 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da68 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619931.889917:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619931.889920:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619931.889926:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7200/0x1cd7fe53687369d7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da68 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619931.889941:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d7200, rc 0) 00000100:00100000:2.0:1788619931.890017:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571708672:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7522us (7894us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619931.890025:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1539 00000100:00100000:0.0:1788619931.897894:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619931.900060:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571708928 00000100:00100000:1.0:1788619931.900080:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571708928 00000100:00100000:0.0:1788619931.904935:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1540 00000100:00100000:0.0:1788619931.904953:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b34e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571708928:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:0.0:1788619931.905013:0:11397:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619931.905042:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe5368736959 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1d9f1 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.905065:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619931.905071:0:11397:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788619931.905374:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52600/0x1cd7fe53687369de lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.905393:0:11397:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52200/0x1cd7fe53687369b4 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da45 expref: 9 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.905403:0:11397:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52600/0x1cd7fe53687369de lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.905437:0:11397:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52200/0x1cd7fe53687369b4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da45 expref: 9 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.905450:0:11397:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52200/0x1cd7fe53687369b4 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da45 expref: 9 pid: 5845 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619931.905465:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b34000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589254272:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619931.905592:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5d2d0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619931.910633:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619931.910749:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b34000 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589254272:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619931.910780:0:11397:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52600/0x1cd7fe53687369de lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619931.910808:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571709056 00000100:00100000:3.0:1788619931.910816:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571709056 00010000:00010000:3.0:1788619931.910857:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52200/0x1cd7fe53687369b4 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da45 expref: 10 pid: 5845 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619931.910902:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 508 00000100:00100000:3.0:1788619931.910913:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcad450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571709056:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619931.911069:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619931.911078:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52200/0x1cd7fe53687369b4 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da45 expref: 10 pid: 5845 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619931.911091:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52200/0x1cd7fe53687369b4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da45 expref: 10 pid: 5845 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619931.911112:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52200/0x1cd7fe53687369b4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da45 expref: 10 pid: 5845 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619931.911127:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619931.911131:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52600/0x1cd7fe53687369de lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619931.911141:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52600/0x1cd7fe53687369de lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619931.911149:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52600/0x1cd7fe53687369de lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619931.911177:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788619931.911203:0:11397:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52600/0x1cd7fe53687369de lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.911210:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52600/0x1cd7fe53687369de lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619931.911261:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcad450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571709056:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 350us (641us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619931.911271:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 508 00000001:00080000:1.0:1788619931.911340:0:11397:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967833 00000004:00010000:1.0:1788619931.911394:0:11397:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 999842 -> 999842 00000004:00010000:1.0:1788619931.911398:0:11397:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619931 -> 1788619931 00000004:00010000:1.0:1788619931.911401:0:11397:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619931 -> 1788619930 00000004:00010000:1.0:1788619931.911403:0:11397:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619931 -> 1788619931 00000004:00010000:1.0:1788619931.911405:0:11397:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1544 -> 1544 00000004:00010000:1.0:1788619931.911413:0:11397:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000078578428 reply state 00000000548a80e7 transno 4294967834 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52600/0x1cd7fe53687369de lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619931.911422:0:11397:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788619931.911493:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b34e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+15:14908:x1875503571708928:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 6545us (13610us total) trans 4294967834 rc 0/0 00000100:00100000:1.0:1788619931.911500:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1540 00000020:00080000:1.0:1788619931.911504:0:11397:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:2.0:1788619931.920181:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52600/0x1cd7fe53687369de lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619931.920199:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52600/0x1cd7fe53687369de lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619931.920208:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52600/0x1cd7fe53687369de lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619931.920214:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c5f52600 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:2.0:1788619931.920220:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52600/0x1cd7fe53687369de lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619931.920230:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52600/0x1cd7fe53687369de lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619931.920248:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619931.920252:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788619931.920257:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52600/0x1cd7fe53687369de lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619931.920263:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52600/0x1cd7fe53687369de lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619931.920436:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619931.920620:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571709184 00000100:00100000:1.0:1788619931.920630:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571709184 00000100:00100000:1.0:1788619931.920686:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1541 00000100:00100000:1.0:1788619931.920696:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b36680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571709184:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619931.920722:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619931.920740:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687369e5 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1da6f expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.920768:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51000/0x1cd7fe53687369a6 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da3e expref: 8 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.920778:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687369e5 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da6f expref: 15 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.920812:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51000/0x1cd7fe53687369a6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da3e expref: 8 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.920825:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51000/0x1cd7fe53687369a6 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da3e expref: 8 pid: 11397 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619931.920839:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c71f0700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589254400:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619931.921988:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619931.924273:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619931.924422:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571709312 00000100:00100000:0.0:1788619931.924430:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571709312 00010000:00010000:0.0:1788619931.924466:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51000/0x1cd7fe53687369a6 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da3e expref: 9 pid: 11397 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619931.924501:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 509 00000100:00100000:0.0:1788619931.924508:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571709312:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619931.924520:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619931.924558:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51000/0x1cd7fe53687369a6 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da3e expref: 9 pid: 11397 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.924578:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51000/0x1cd7fe53687369a6 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da3e expref: 9 pid: 11397 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.924599:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619931.924606:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687369e5 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da6f expref: 15 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.924623:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687369e5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da6f expref: 15 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.924633:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687369e5 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da6f expref: 15 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.924672:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687369e5 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da6f expref: 15 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.924674:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f51000/0x1cd7fe53687369a6 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da3e expref: 9 pid: 11397 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619931.924685:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b34000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589254528:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619931.924690:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c71f0700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589254400:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619931.924713:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687369e5 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da6f expref: 15 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.924722:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619931.924726:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619931.924728:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d200, rc 0) 00000100:00100000:1.0:1788619931.924754:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b36680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:14908:x1875503571709184:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4060us (4323us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619931.924759:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5d2d240 going to sleep for 11 seconds 00000100:00100000:1.0:1788619931.924761:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1541 00000100:00100000:0.0:1788619931.927967:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619931.928159:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571709440 00000100:00100000:1.0:1788619931.928169:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571709440 00002000:00010000:1.0:1788619931.928210:0:6701:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1962250->18446744073709551615) 00002000:00100000:1.0:1788619931.928215:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6d02a00 with x1875503571709440 ext(1962250->18446744073709551615) 00002000:00010000:1.0:1788619931.928226:0:6701:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6d02a00. 00000100:00100000:1.0:1788619931.928261:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 752 00000100:00100000:1.0:1788619931.928271:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d02a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571709440:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:1.0:1788619931.928669:0:6701:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1987168 -> 1962250 00002000:00010000:1.0:1788619931.928673:0:6701:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619931 -> 1788619931 00002000:00010000:1.0:1788619931.928675:0:6701:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619931 -> 1788619931 00002000:00010000:1.0:1788619931.928678:0:6701:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619931 -> 1788619931 00002000:00010000:1.0:1788619931.928680:0:6701:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 263 -> 519 00000100:00100000:1.0:1788619931.928744:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d02a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571709440:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 473us (787us total) trans 4294967515 rc 0/0 00000100:00100000:1.0:1788619931.928752:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 752 00002000:00010000:1.0:1788619931.928756:0:6701:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1962250->18446744073709551615) 00002000:00100000:1.0:1788619931.928759:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6d02a00 with x1875503571709440 ext(1962250->18446744073709551615) 00002000:00010000:1.0:1788619931.928782:0:6701:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6d02a00. 00000100:00100000:3.0:1788619931.929583:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b34000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589254528:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619931.929608:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619931.929851:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+8:2354:x1875503571709312:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 5340us (5583us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619931.929860:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 509 00000100:00100000:2.0:1788619931.931313:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619931.931519:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571709568 00000100:00100000:0.0:1788619931.931539:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571709568 00000100:00100000:0.0:1788619931.931584:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1542 00000100:00100000:0.0:1788619931.931594:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:14908:x1875503571709568:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619931.931615:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619931.931636:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52400/0x1cd7fe53687369ec lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1da76 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.931722:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f53e00/0x1cd7fe53687369f3 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.931739:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f53e00/0x1cd7fe53687369f3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.931752:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f53e00/0x1cd7fe53687369f3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619931.931780:0:5846:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1962250 00000004:00200000:0.0:1788619931.931785:0:5846:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619931.931815:0:5846:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f53e00/0x1cd7fe53687369f3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.931843:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52400/0x1cd7fe53687369ec lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da76 expref: 17 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.931860:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f53e00/0x1cd7fe53687369f3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da76 expref: 16 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.931871:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619931.931874:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619931.931878:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f53e00, rc 0) 00000100:00100000:0.0:1788619931.931931:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:14908:x1875503571709568:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 339us (622us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619931.931939:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1542 00000100:00100000:0.0:1788619931.939515:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619931.939663:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571709696 00000100:00100000:1.0:1788619931.939673:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571709696 00000100:00100000:1.0:1788619931.939734:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1543 00000100:00100000:1.0:1788619931.939745:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d03b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+8:14908:x1875503571709696:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619931.939767:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619931.939800:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe53687369fa lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1da7d expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.939892:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368736a01 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.939909:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368736a01 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.939921:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368736a01 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788619931.939950:0:11397:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1962250 00000004:00200000:1.0:1788619931.939954:0:11397:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788619931.939975:0:11397:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368736a01 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.940004:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe53687369fa lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da7d expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.940022:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368736a01 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da7d expref: 9 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.940031:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619931.940034:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788619931.940037:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c600, rc 0) 00000100:00100000:1.0:1788619931.940112:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d03b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571709696:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 367us (607us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619931.940123:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1543 00000100:00100000:3.0:1788619931.955999:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619931.956038:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619931.956161:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571709952 00000100:00100000:0.0:1788619931.956171:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571709952 00000100:00100000:0.0:1788619931.956224:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1267 00000100:00100000:0.0:1788619931.956234:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571709952:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619931.956251:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619931.956268:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b7d61c1/0x1cd7fe5368736a08 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1da8b expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619931.956363:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6d01500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589254656:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619931.956434:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5d2d240 going to sleep for 11 seconds 00000100:00100000:1.0:1788619931.956503:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571709824 00000100:00100000:1.0:1788619931.956507:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571709824 00000100:00100000:1.0:1788619931.956551:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1544 00000100:00100000:1.0:1788619931.956558:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571709824:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619931.956568:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619931.956580:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368736a0f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1da84 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.956608:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687369e5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da6f expref: 15 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619931.956630:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c71f0700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589254784:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619931.956652:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc0c0 going to sleep for 11 seconds 00000004:00010000:1.0:1788619931.960241:0:5844:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619930 -> 1788619931 00000100:00100000:1.0:1788619931.960273:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c71f0700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589254784:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619931.960298:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 999842 00010000:00010000:1.0:1788619931.960318:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368736a0f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da84 expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.960375:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619931.960378:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619931.960384:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368736a0f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da84 expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.960399:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c200, rc 0) 00000100:00100000:1.0:1788619931.960462:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571709824:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3903us (4471us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619931.960471:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1544 00000100:00100000:2.0:1788619931.960618:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6d01500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589254656:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619931.960641:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002b7d61c1/0x1cd7fe5368736a08 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da8b expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619931.960657:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b7d61c1/0x1cd7fe5368736a08 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da8b expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619931.960672:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f53200, rc 0) 00000100:00100000:2.0:1788619931.960695:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571709952:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4463us (4656us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619931.960703:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1267 00000100:00100000:2.0:1788619931.968426:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619931.968556:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571710080 00000100:00100000:0.0:1788619931.968566:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571710080 00000100:00100000:0.0:1788619931.968627:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1545 00000100:00100000:0.0:1788619931.968638:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571710080:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619931.968658:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619931.968678:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52800/0x1cd7fe5368736a16 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1da92 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.968713:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687369e5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da6f expref: 15 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619931.968751:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6d03100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589254912:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619931.968841:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5d2d780 going to sleep for 11 seconds 00000100:00100000:0.0:1788619931.970771:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6d03100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589254912:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619931.970811:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 999842 00010000:00010000:0.0:1788619931.970831:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52800/0x1cd7fe5368736a16 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da92 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.970859:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619931.970863:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619931.970869:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52800/0x1cd7fe5368736a16 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da92 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.970884:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f52800, rc 0) 00000100:00100000:0.0:1788619931.970941:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571710080:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2304us (2523us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619931.970950:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1545 00000100:00100000:2.0:1788619931.972036:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619931.972118:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571710208 00000100:00100000:1.0:1788619931.972128:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571710208 00000100:00100000:1.0:1788619931.972182:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1268 00000100:00100000:1.0:1788619931.972192:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571710208:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619931.972210:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619931.972226:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe5368736a1d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1da99 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619931.972285:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c71f0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589255040:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619931.972389:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619931.974692:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c71f0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589255040:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619931.974739:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe5368736a1d lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da99 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.974763:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002ab297cb/0x1cd7fe5368736a1d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da99 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.974777:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c200, rc 0) 00000100:00100000:1.0:1788619931.974834:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571710208:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2643us (2804us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619931.974842:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1268 00000100:00100000:3.0:1788619931.978897:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619931.979016:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571710336 00000100:00100000:0.0:1788619931.979027:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571710336 00000100:00100000:0.0:1788619931.979103:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 936 00000100:00100000:0.0:1788619931.979114:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2353:x1875503571710336:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:0.0:1788619931.979391:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 999842 -> 999842 00000004:00010000:0.0:1788619931.979397:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619931 -> 1788619931 00000004:00010000:0.0:1788619931.979400:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619931 -> 1788619930 00000004:00010000:0.0:1788619931.979402:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619931 -> 1788619931 00000004:00010000:0.0:1788619931.979405:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1544 -> 1544 00000100:00100000:0.0:1788619931.979466:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2353:x1875503571710336:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 354us (579us total) trans 4294967835 rc 0/0 00000100:00100000:0.0:1788619931.979475:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 936 00000100:00100000:2.0:1788619931.984069:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619931.984129:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619931.984212:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571710464 00000100:00100000:3.0:1788619931.984222:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571710464 00000100:00100000:3.0:1788619931.984282:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1546 00000100:00100000:3.0:1788619931.984293:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571710464:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619931.984313:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619931.984370:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6e00/0x1cd7fe5368736a24 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1daa0 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619931.984408:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687369e5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da6f expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619931.984448:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c4a4d180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589255168:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619931.984533:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48ce40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619931.985105:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571710592 00000100:00100000:0.0:1788619931.985112:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571710592 00000100:00100000:0.0:1788619931.985154:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1269 00000100:00100000:0.0:1788619931.985162:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c0b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571710592:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619931.985172:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619931.985185:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000632bef10/0x1cd7fe5368736a2b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1daa7 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619931.985227:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6d03100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589255296:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619931.985278:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5d2d780 going to sleep for 11 seconds 00000004:00010000:3.0:1788619931.988921:0:9417:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619930 -> 1788619931 00000100:00100000:3.0:1788619931.988958:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c4a4d180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589255168:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619931.988987:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 999842 00010000:00010000:3.0:1788619931.989010:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6e00/0x1cd7fe5368736a24 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1daa0 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619931.989031:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619931.989035:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619931.989042:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6e00/0x1cd7fe5368736a24 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1daa0 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619931.989060:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d6e00, rc 0) 00000100:00100000:3.0:1788619931.989128:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571710464:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4836us (5068us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619931.989138:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1546 00000100:00100000:0.0:1788619931.989735:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6d03100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589255296:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619931.989760:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000632bef10/0x1cd7fe5368736a2b lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1daa7 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.989779:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000632bef10/0x1cd7fe5368736a2b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1daa7 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619931.989802:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f52800, rc 0) 00000100:00100000:0.0:1788619931.989947:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c0b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571710592:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4785us (5817us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619931.989956:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1269 00000100:00100000:2.0:1788619931.994223:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619931.994275:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619931.994387:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571710848 00000100:00100000:0.0:1788619931.994397:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571710848 00000100:00100000:0.0:1788619931.994450:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1270 00000100:00100000:0.0:1788619931.994460:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c09500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571710848:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619931.994478:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619931.994496:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c27e6ea/0x1cd7fe5368736a32 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dab5 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619931.994580:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6d03100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589255424:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619931.994667:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5d2d780 going to sleep for 11 seconds 00000100:00100000:1.0:1788619931.995192:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571710720 00000100:00100000:1.0:1788619931.995199:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571710720 00000100:00100000:1.0:1788619931.995244:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1547 00000100:00100000:1.0:1788619931.995253:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c0b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571710720:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619931.995267:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619931.995283:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368736a39 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1daae expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.995315:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687369e5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da6f expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619931.995386:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c71f0700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589255552:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619931.995425:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619931.998623:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c71f0700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589255552:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619931.998665:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 999842 00010000:00010000:1.0:1788619931.998686:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368736a39 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1daae expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.998707:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619931.998710:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619931.998716:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368736a39 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1daae expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619931.998730:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c200, rc 0) 00000100:00100000:1.0:1788619931.998799:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c0b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571710720:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3547us (4585us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619931.998808:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1547 00000100:00100000:0.0:1788619932.002139:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6d03100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589255424:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619932.002183:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004c27e6ea/0x1cd7fe5368736a32 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dab5 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619932.002208:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c27e6ea/0x1cd7fe5368736a32 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dab5 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619932.002223:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f53c00, rc 0) 00000100:00100000:0.0:1788619932.002286:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c09500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571710848:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7826us (8009us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619932.002294:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1270 00000100:00100000:3.0:1788619932.005259:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619932.005377:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571711104 00000100:00100000:0.0:1788619932.005387:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571711104 00000100:00100000:0.0:1788619932.005459:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1548 00000100:00100000:0.0:1788619932.005471:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+17:14908:x1875503571711104:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:0.0:1788619932.005576:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619932.005608:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f53e00/0x1cd7fe53687369f3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da76 expref: 17 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619932.005629:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619932.005635:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788619932.005870:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52800/0x1cd7fe5368736a40 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619932.005887:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368736a01 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da7d expref: 8 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619932.005898:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52800/0x1cd7fe5368736a40 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619932.005934:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368736a01 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da7d expref: 8 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619932.005950:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368736a01 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da7d expref: 8 pid: 11397 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619932.005966:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6d03100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589255680:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619932.006044:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5d2d780 going to sleep for 11 seconds 00000100:00080000:3.0:1788619932.006994:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1788619932.009621:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619932.009694:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571711232 00000100:00100000:1.0:1788619932.009704:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571711232 00010000:00010000:1.0:1788619932.009755:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368736a01 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da7d expref: 9 pid: 11397 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619932.009804:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 510 00000100:00100000:1.0:1788619932.009815:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c0bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571711232:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619932.009831:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619932.009838:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368736a01 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da7d expref: 9 pid: 11397 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.009852:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368736a01 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da7d expref: 9 pid: 11397 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.009876:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619932.009882:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52800/0x1cd7fe5368736a40 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.009893:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52800/0x1cd7fe5368736a40 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1788619932.010134:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:1.0:1788619932.010260:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52800/0x1cd7fe5368736a40 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.010283:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619932.010368:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c0bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571711232:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 554us (756us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619932.010376:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 510 00010000:00010000:0.0:1788619932.011616:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368736a01 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da7d expref: 8 pid: 11397 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619932.011640:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6d03100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589255680:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619932.011662:0:5845:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52800/0x1cd7fe5368736a40 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619932.011674:0:5845:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52800/0x1cd7fe5368736a40 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619932.011681:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52800/0x1cd7fe5368736a40 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788619932.011901:0:5845:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967836 00000004:00010000:0.0:1788619932.012107:0:5845:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 999842 -> 999842 00000004:00010000:0.0:1788619932.012112:0:5845:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619931 -> 1788619931 00000004:00010000:0.0:1788619932.012114:0:5845:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619931 -> 1788619930 00000004:00010000:0.0:1788619932.012117:0:5845:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619931 -> 1788619931 00000004:00010000:0.0:1788619932.012120:0:5845:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1544 -> 1544 00000004:00010000:0.0:1788619932.012127:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000002aa43d4d reply state 000000001f69fa13 transno 4294967837 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52800/0x1cd7fe5368736a40 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619932.012136:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788619932.012279:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+19:14908:x1875503571711104:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 6809us (7030us total) trans 4294967837 rc 0/0 00000100:00100000:0.0:1788619932.012288:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1548 00000020:00080000:0.0:1788619932.012293:0:5845:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1788619932.015367:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619932.015444:0:13347:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571711360 00000100:00100000:3.0:1788619932.015454:0:13347:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571711360 00002000:00010000:3.0:1788619932.015512:0:13347:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1410173->18446744073709551615) 00002000:00100000:3.0:1788619932.015517:0:13347:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5c08700 with x1875503571711360 ext(1410173->18446744073709551615) 00002000:00010000:3.0:1788619932.015541:0:13347:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5c08700. 00000100:00100000:3.0:1788619932.015576:0:13347:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 753 00000100:00100000:3.0:1788619932.015586:0:13347:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c08700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571711360:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:3.0:1788619932.015966:0:13347:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1962250 -> 1410173 00002000:00010000:3.0:1788619932.015971:0:13347:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619931 -> 1788619931 00002000:00010000:3.0:1788619932.015974:0:13347:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619931 -> 1788619931 00002000:00010000:3.0:1788619932.015976:0:13347:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619931 -> 1788619931 00002000:00010000:3.0:1788619932.015979:0:13347:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 519 -> 519 00000100:00100000:3.0:1788619932.016041:0:13347:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c08700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571711360:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 456us (692us total) trans 4294967516 rc 0/0 00000100:00100000:3.0:1788619932.016049:0:13347:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 753 00002000:00010000:3.0:1788619932.016053:0:13347:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1410173->18446744073709551615) 00002000:00100000:3.0:1788619932.016056:0:13347:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5c08700 with x1875503571711360 ext(1410173->18446744073709551615) 00002000:00010000:3.0:1788619932.016060:0:13347:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5c08700. 00000100:00080000:2.0:1788619932.016705:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1788619932.018573:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52800/0x1cd7fe5368736a40 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619932.018591:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52800/0x1cd7fe5368736a40 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619932.018599:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52800/0x1cd7fe5368736a40 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619932.018605:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c5f52800 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788619932.018610:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52800/0x1cd7fe5368736a40 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619932.018619:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52800/0x1cd7fe5368736a40 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619932.018641:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619932.018644:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788619932.018649:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52800/0x1cd7fe5368736a40 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619932.018655:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52800/0x1cd7fe5368736a40 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619932.019618:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619932.019691:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571711488 00000100:00100000:3.0:1788619932.019699:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571711488 00000100:00100000:3.0:1788619932.019747:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1549 00000100:00100000:3.0:1788619932.019757:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+8:14908:x1875503571711488:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619932.019776:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619932.019803:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7800/0x1cd7fe5368736a47 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dabc expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.019884:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6e00/0x1cd7fe5368736a4e lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.019898:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6e00/0x1cd7fe5368736a4e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.019920:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6e00/0x1cd7fe5368736a4e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788619932.019948:0:9417:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1410173 00000004:00200000:3.0:1788619932.019953:0:9417:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788619932.019971:0:9417:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6e00/0x1cd7fe5368736a4e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.019996:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7800/0x1cd7fe5368736a47 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dabc expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.020012:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6e00/0x1cd7fe5368736a4e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dabc expref: 9 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.020021:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619932.020024:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788619932.020028:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d6e00, rc 0) 00000100:00100000:3.0:1788619932.020086:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571711488:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 330us (474us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619932.020094:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1549 00000100:00100000:3.0:1788619932.027488:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619932.027543:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619932.027635:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571711616 00000100:00100000:1.0:1788619932.027646:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571711616 00000100:00100000:1.0:1788619932.027703:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1550 00000100:00100000:1.0:1788619932.027713:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571711616:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619932.027733:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619932.027756:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368736a55 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dac3 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.027807:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687369e5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da6f expref: 17 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619932.027849:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c71f3480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589255808:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619932.027933:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619932.028783:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571711744 00000100:00100000:2.0:1788619932.028802:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571711744 00000100:00100000:2.0:1788619932.028860:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1271 00000100:00100000:2.0:1788619932.028867:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571711744:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619932.028879:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619932.028891:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041aa2bf0/0x1cd7fe5368736a5c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1daca expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619932.028933:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c0b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589255936:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619932.028992:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81640c0 going to sleep for 11 seconds 00000100:00080000:3.0:1788619932.029373:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000004:00010000:1.0:1788619932.032159:0:5844:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619930 -> 1788619931 00000100:00100000:1.0:1788619932.032195:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c71f3480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589255808:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619932.032221:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 999842 00010000:00010000:1.0:1788619932.032242:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368736a55 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dac3 expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.032275:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619932.032278:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619932.032284:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c600/0x1cd7fe5368736a55 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dac3 expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.032299:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c600, rc 0) 00000100:00100000:1.0:1788619932.032421:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571711616:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4708us (4876us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619932.032429:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1550 00000100:00100000:2.0:1788619932.033996:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c0b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589255936:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619932.034030:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000041aa2bf0/0x1cd7fe5368736a5c lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1daca expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.034052:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041aa2bf0/0x1cd7fe5368736a5c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1daca expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.034063:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7a00, rc 0) 00000100:00100000:2.0:1788619932.034114:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571711744:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5247us (6638us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619932.034123:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1271 00000100:00100000:3.0:1788619932.038828:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619932.038862:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619932.038955:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571711872 00000100:00100000:0.0:1788619932.038967:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571711872 00000100:00100000:0.0:1788619932.039023:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1551 00000100:00100000:0.0:1788619932.039033:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7581c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571711872:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619932.039052:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619932.039071:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52800/0x1cd7fe5368736a63 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dad1 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619932.039105:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687369e5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da6f expref: 17 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619932.039141:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6d03100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589256064:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619932.039217:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5d2d780 going to sleep for 11 seconds 00000100:00100000:2.0:1788619932.041260:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571712000 00000100:00100000:2.0:1788619932.041271:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571712000 00000100:00100000:2.0:1788619932.041488:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1272 00000100:00100000:2.0:1788619932.041501:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b34000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571712000:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619932.041530:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619932.041551:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003a2d9d44/0x1cd7fe5368736a6a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dad8 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619932.041614:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c0b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589256192:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619932.041708:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81640c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619932.043187:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6d03100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589256064:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619932.043215:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 999842 00010000:00010000:2.0:1788619932.043234:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52800/0x1cd7fe5368736a63 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dad1 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.043253:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619932.043257:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619932.043263:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f52800/0x1cd7fe5368736a63 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dad1 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.043277:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5f52800, rc 0) 00000100:00100000:2.0:1788619932.045770:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7581c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571711872:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4311us (4479us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619932.046021:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1551 00000100:00100000:3.0:1788619932.046108:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c0b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589256192:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619932.046147:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003a2d9d44/0x1cd7fe5368736a6a lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dad8 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.046174:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a2d9d44/0x1cd7fe5368736a6a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dad8 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.046186:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7600, rc 0) 00000100:00100000:3.0:1788619932.046248:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b34000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571712000:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4745us (7423us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619932.049630:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619932.050560:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571712128 00000100:00100000:1.0:1788619932.050571:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571712128 00000100:00100000:1.0:1788619932.050638:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1552 00000100:00100000:1.0:1788619932.050649:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+18:14908:x1875503571712128:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619932.050674:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619932.050693:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368736a71 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dadf expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.050785:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368736a78 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.050886:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368736a78 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.050899:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368736a78 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788619932.050939:0:11397:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1410173 00000004:00200000:1.0:1788619932.050945:0:11397:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788619932.050966:0:11397:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368736a78 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.050998:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368736a71 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dadf expref: 20 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.051019:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368736a78 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dadf expref: 19 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.051029:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619932.051033:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788619932.051037:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683ce00, rc 0) 00000100:00100000:1.0:1788619932.051102:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+19:14908:x1875503571712128:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 455us (1482us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619932.051108:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1552 00000100:00100000:1.0:1788619932.055067:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619932.055132:0:13346:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571712256 00000100:00100000:2.0:1788619932.055141:0:13346:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571712256 00000100:00100000:2.0:1788619932.055209:0:13346:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 937 00000100:00100000:2.0:1788619932.055219:0:13346:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2352:x1875503571712256:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:2.0:1788619932.055514:0:13346:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 999842 -> 999842 00000004:00010000:2.0:1788619932.055519:0:13346:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619931 -> 1788619931 00000004:00010000:2.0:1788619932.055533:0:13346:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619931 -> 1788619930 00000004:00010000:2.0:1788619932.055535:0:13346:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619931 -> 1788619931 00000004:00010000:2.0:1788619932.055537:0:13346:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1544 -> 1544 00000100:00100000:2.0:1788619932.055595:0:13346:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2352:x1875503571712256:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 378us (537us total) trans 4294967838 rc 0/0 00000100:00100000:2.0:1788619932.055603:0:13346:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 937 00000100:00100000:3.0:1788619932.058434:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1272 00000100:00100000:1.0:1788619932.058594:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619932.058659:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619932.058708:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571712384 00000100:00100000:2.0:1788619932.058716:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571712384 00000100:00100000:2.0:1788619932.058764:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1553 00000100:00100000:2.0:1788619932.058773:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571712384:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619932.058800:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619932.058819:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7a00/0x1cd7fe5368736a7f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dae6 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.058858:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687369e5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da6f expref: 19 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619932.058894:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c0aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589256320:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619932.058970:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164480 going to sleep for 11 seconds 00000004:00010000:2.0:1788619932.061153:0:5844:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619930 -> 1788619931 00000100:00100000:2.0:1788619932.061187:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c0aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589256320:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619932.061210:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 999842 00010000:00010000:2.0:1788619932.061229:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7a00/0x1cd7fe5368736a7f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dae6 expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.061249:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619932.061252:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619932.061258:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc7a00/0x1cd7fe5368736a7f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dae6 expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.061273:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7a00, rc 0) 00000100:00100000:2.0:1788619932.061367:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571712384:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2593us (2705us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619932.061375:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1553 00000100:00100000:0.0:1788619932.062677:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571712512 00000100:00100000:0.0:1788619932.062688:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571712512 00000100:00100000:1.0:1788619932.062925:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1273 00000100:00100000:1.0:1788619932.062937:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571712512:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619932.062954:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619932.062972:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000095a50c3e/0x1cd7fe5368736a86 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1daed expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619932.063028:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c71f2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589256448:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619932.063115:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc0c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619932.065585:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c71f2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589256448:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619932.065628:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000095a50c3e/0x1cd7fe5368736a86 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1daed expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.065652:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000095a50c3e/0x1cd7fe5368736a86 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1daed expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.065665:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d600, rc 0) 00000100:00100000:3.0:1788619932.065727:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571712512:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2792us (7133us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619932.065735:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1273 00000100:00100000:0.0:1788619932.072558:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619932.072593:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619932.072720:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571712640 00000100:00100000:2.0:1788619932.072731:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571712640 00000100:00100000:2.0:1788619932.072900:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1554 00000100:00100000:2.0:1788619932.072922:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d00380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571712640:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619932.072944:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619932.072964:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6800/0x1cd7fe5368736a8d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1daf4 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.073002:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687369e5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da6f expref: 19 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619932.073041:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c0aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589256576:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619932.073120:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164480 going to sleep for 11 seconds 00000100:00100000:3.0:1788619932.073712:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571712768 00000100:00100000:3.0:1788619932.073719:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571712768 00000100:00100000:3.0:1788619932.073765:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1274 00000100:00100000:3.0:1788619932.073774:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d02680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571712768:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619932.073783:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619932.073806:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e4805681/0x1cd7fe5368736a94 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dafb expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619932.073847:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c7582300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589256704:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619932.073892:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48ce40 going to sleep for 11 seconds 00000100:00100000:2.0:1788619932.076772:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c0aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589256576:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619932.076814:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 999842 00010000:00010000:2.0:1788619932.076835:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6800/0x1cd7fe5368736a8d lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1daf4 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.076859:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619932.076863:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619932.076869:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6800/0x1cd7fe5368736a8d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1daf4 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.076887:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6800, rc 0) 00000100:00100000:2.0:1788619932.076962:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d00380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571712640:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4043us (4414us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619932.076972:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1554 00000100:00100000:3.0:1788619932.077641:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c7582300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589256704:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619932.077666:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e4805681/0x1cd7fe5368736a94 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dafb expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.077684:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e4805681/0x1cd7fe5368736a94 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dafb expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.077694:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d6400, rc 0) 00000100:00100000:3.0:1788619932.077722:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d02680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571712768:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3951us (5128us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619932.077730:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1274 00000100:00100000:0.0:1788619932.084778:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619932.084822:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619932.084868:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571713024 00000100:00100000:2.0:1788619932.084879:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571713024 00000100:00100000:1.0:1788619932.084899:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571712896 00002000:00100000:2.0:1788619932.084929:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6d03800 x1875503571713024/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619932.084940:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6d03800 with x1875503571713024 00000100:00100000:2.0:1788619932.084980:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 938 00000100:00100000:2.0:1788619932.084991:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d03800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2353:x1875503571713024:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619932.085063:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571712896 00002000:00100000:1.0:1788619932.085089:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1413119) req@ffff8e77c6d00a80 x1875503571712896/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619932.085098:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6d00a80 with x1875503571712896 ext(1048576->1413119) 00002000:00010000:1.0:1788619932.085102:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6d00a80 00000100:00100000:1.0:1788619932.085135:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 754 00000100:00100000:1.0:1788619932.085144:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d00a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2352:x1875503571712896:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619932.101140:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d03800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2353:x1875503571713024:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 16137us (16302us total) trans 0 rc 1003520/1003520 00000100:00100000:3.0:1788619932.101157:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 938 00002000:00100000:3.0:1788619932.101172:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6d03800 x1875503571713024/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:28/0 lens 488/440 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 1003520/1003520 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619932.101184:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6d03800 with x1875503571713024 00000100:00100000:1.0:1788619932.103571:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d00a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2352:x1875503571712896:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 18416us (18790us total) trans 0 rc 364544/364544 00000100:00100000:1.0:1788619932.103589:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 754 00002000:00100000:1.0:1788619932.103602:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1413119) req@ffff8e77c6d00a80 x1875503571712896/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:28/0 lens 488/440 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 364544/364544 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619932.103612:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6d00a80 with x1875503571712896 ext(1048576->1413119) 00002000:00010000:1.0:1788619932.103619:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6d00a80 00000100:00100000:2.0:1788619932.107723:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619932.107766:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619932.107892:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571713280 00000100:00100000:3.0:1788619932.108020:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571713152 00000100:00100000:3.0:1788619932.108026:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571713152 00000100:00100000:0.0:1788619932.108097:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571713280 00000100:00100000:3.0:1788619932.108189:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1555 00000100:00100000:3.0:1788619932.108201:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c09c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571713152:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619932.108225:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619932.108246:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7800/0x1cd7fe5368736a9b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1db02 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.108285:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687369e5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da6f expref: 19 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619932.108359:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c7582300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589256832:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619932.108435:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48ce40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619932.109203:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1275 00000100:00100000:0.0:1788619932.109217:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c0aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571713280:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619932.109230:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619932.109245:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007da7fefa/0x1cd7fe5368736aa2 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1db09 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619932.109289:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6d01c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589256960:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619932.109369:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5d2de40 going to sleep for 11 seconds 00000100:00100000:3.0:1788619932.111960:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c7582300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589256832:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619932.111987:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 999842 00010000:00010000:3.0:1788619932.112006:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7800/0x1cd7fe5368736a9b lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db02 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.112027:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619932.112030:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619932.112036:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7800/0x1cd7fe5368736a9b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db02 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.112051:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d7800, rc 0) 00000100:00100000:3.0:1788619932.112118:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c09c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571713152:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3917us (4349us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619932.112126:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1555 00000100:00100000:0.0:1788619932.112571:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6d01c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589256960:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619932.112596:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007da7fefa/0x1cd7fe5368736aa2 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db09 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619932.112613:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007da7fefa/0x1cd7fe5368736aa2 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db09 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619932.112624:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c1495000, rc 0) 00000100:00100000:0.0:1788619932.112663:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c0aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571713280:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3451us (4950us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619932.112670:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1275 00000100:00100000:2.0:1788619932.116175:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619932.116255:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571713408 00000100:00100000:3.0:1788619932.116265:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571713408 00000100:00100000:3.0:1788619932.116359:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1276 00000100:00100000:3.0:1788619932.116370:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c0a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:14908:x1875503571713408:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619932.116387:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619932.116405:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e4805681/0x1cd7fe5368736aa9 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1db10 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.116438:0:6696:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000001dc9de0/0x1cd7fe53687369c9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1429504->1433599) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da5a expref: 7 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.116449:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(1 11) 00010000:00010000:3.0:1788619932.116453:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 11) 00010000:00010000:3.0:1788619932.116460:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e4805681/0x1cd7fe5368736aa9 lrc: 4/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1376256->1380351] (req 1376256->1380351) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db10 expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.116494:0:6696:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000001dc9de0/0x1cd7fe53687369c9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1429504->1433599) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da5a expref: 7 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.116509:0:6696:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000001dc9de0/0x1cd7fe53687369c9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1429504->1433599) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da5a expref: 7 pid: 6697 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619932.116535:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c7582300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589257088:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619932.116612:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48ce40 going to sleep for 11 seconds 00000100:00100000:3.0:1788619932.120193:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619932.120266:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571713536 00000100:00100000:0.0:1788619932.120276:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571713536 00010000:00010000:0.0:1788619932.120313:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000001dc9de0/0x1cd7fe53687369c9 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1429504->1433599) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da5a expref: 8 pid: 6697 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619932.120392:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 511 00000100:00100000:0.0:1788619932.120403:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7583100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571713536:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619932.120419:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619932.120441:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000001dc9de0/0x1cd7fe53687369c9 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1429504->1433599) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da5a expref: 8 pid: 6697 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619932.120456:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000001dc9de0/0x1cd7fe53687369c9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1429504->1433599) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da5a expref: 8 pid: 6697 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619932.120482:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 11) 00010000:00010000:0.0:1788619932.120487:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 11) 00010000:00010000:0.0:1788619932.120496:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000e4805681/0x1cd7fe5368736aa9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1376256->1380351) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db10 expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619932.120508:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e4805681/0x1cd7fe5368736aa9 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1376256->1380351) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db10 expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619932.120557:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000e4805681/0x1cd7fe5368736aa9 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1376256->1380351) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db10 expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619932.120572:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6d01c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589257216:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619932.120646:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5d2de40 going to sleep for 11 seconds 00010000:00010000:2.0:1788619932.121653:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001dc9de0/0x1cd7fe53687369c9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1429504->1433599) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da5a expref: 8 pid: 6697 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619932.121673:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c7582300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589257088:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619932.121698:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e4805681/0x1cd7fe5368736aa9 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1376256->1380351) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db10 expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.121712:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d6400, rc 0) 00000100:00100000:2.0:1788619932.121764:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c0a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:14908:x1875503571713408:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5396us (5598us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619932.121773:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1276 00000100:00100000:0.0:1788619932.123728:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6d01c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589257216:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619932.123753:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619932.123805:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7583100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571713536:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3402us (3620us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619932.123821:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 511 00000100:00100000:2.0:1788619932.126758:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619932.126876:0:13347:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571713664 00000100:00100000:3.0:1788619932.126885:0:13347:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571713664 00002000:00100000:3.0:1788619932.126944:0:13347:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x4:0x0] (1376256->1380351) req@ffff8e78c5c0a300 x1875503571713664/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619932.126957:0:13347:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5c0a300 with x1875503571713664 ext(1376256->1380351) 00002000:00010000:3.0:1788619932.126963:0:13347:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5c0a300 00000100:00100000:3.0:1788619932.126996:0:13347:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 755 00000100:00100000:3.0:1788619932.127006:0:13347:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c0a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571713664:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619932.127195:0:13347:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c0a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571713664:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 190us (447us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619932.127204:0:13347:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 755 00002000:00100000:3.0:1788619932.127211:0:13347:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x4:0x0] (1376256->1380351) req@ffff8e78c5c0a300 x1875503571713664/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619932.127221:0:13347:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5c0a300 with x1875503571713664 ext(1376256->1380351) 00002000:00010000:3.0:1788619932.127224:0:13347:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5c0a300 00000100:00100000:1.0:1788619932.130585:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619932.130683:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571713792 00000100:00100000:3.0:1788619932.130693:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571713792 00002000:00100000:3.0:1788619932.130744:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1380352->1384447) req@ffff8e77c71f2a00 x1875503571713792/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619932.130756:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c71f2a00 with x1875503571713792 ext(1380352->1384447) 00002000:00010000:3.0:1788619932.130762:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c71f2a00 00000100:00100000:3.0:1788619932.130883:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 756 00000100:00100000:3.0:1788619932.130894:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571713792:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619932.131283:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571713792:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 390us (707us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619932.131292:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 756 00002000:00100000:3.0:1788619932.131300:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1380352->1384447) req@ffff8e77c71f2a00 x1875503571713792/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619932.131310:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c71f2a00 with x1875503571713792 ext(1380352->1384447) 00002000:00010000:3.0:1788619932.131314:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c71f2a00 00000100:00100000:2.0:1788619932.135491:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619932.135574:0:13347:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571713920 00000100:00100000:3.0:1788619932.135584:0:13347:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571713920 00002000:00100000:3.0:1788619932.135633:0:13347:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x4:0x0] (1384448->1388543) req@ffff8e78c5c09f80 x1875503571713920/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619932.135650:0:13347:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5c09f80 with x1875503571713920 ext(1384448->1388543) 00002000:00010000:3.0:1788619932.135656:0:13347:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5c09f80 00000100:00100000:3.0:1788619932.135735:0:13347:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 757 00000100:00100000:3.0:1788619932.135745:0:13347:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c09f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571713920:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619932.136113:0:13347:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c09f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571713920:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 368us (632us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619932.136122:0:13347:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 757 00002000:00100000:3.0:1788619932.136129:0:13347:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x4:0x0] (1384448->1388543) req@ffff8e78c5c09f80 x1875503571713920/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619932.136139:0:13347:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5c09f80 with x1875503571713920 ext(1384448->1388543) 00002000:00010000:3.0:1788619932.136143:0:13347:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5c09f80 00000100:00100000:2.0:1788619932.139517:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619932.139636:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571714048 00000100:00100000:1.0:1788619932.139646:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571714048 00002000:00100000:1.0:1788619932.139692:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1388544->1392639) req@ffff8e78c5c0ad80 x1875503571714048/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619932.139703:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5c0ad80 with x1875503571714048 ext(1388544->1392639) 00002000:00010000:1.0:1788619932.139708:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5c0ad80 00000100:00100000:1.0:1788619932.139780:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 758 00000100:00100000:1.0:1788619932.139855:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c0ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571714048:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619932.140080:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c0ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571714048:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 290us (572us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619932.140088:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 758 00002000:00100000:1.0:1788619932.140095:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1388544->1392639) req@ffff8e78c5c0ad80 x1875503571714048/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619932.140104:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5c0ad80 with x1875503571714048 ext(1388544->1392639) 00002000:00010000:1.0:1788619932.140108:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5c0ad80 00000100:00100000:2.0:1788619932.143214:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619932.143781:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571714176 00000100:00100000:3.0:1788619932.144030:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571714176 00000100:00100000:2.0:1788619932.144225:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1277 00000100:00100000:2.0:1788619932.144236:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c08e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2353:x1875503571714176:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619932.144255:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619932.144276:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d2409dc/0x1cd7fe5368736ab0 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1db17 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.144307:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 11) 00010000:00010000:2.0:1788619932.144312:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 11) 00010000:00010000:2.0:1788619932.144340:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d2409dc/0x1cd7fe5368736ab0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db17 expref: 8 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.144371:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000d2409dc/0x1cd7fe5368736ab0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db17 expref: 8 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.144385:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6000, rc 0) 00000100:00100000:2.0:1788619932.144448:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c08e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571714176:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 213us (1242us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619932.144454:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1277 00000100:00100000:3.0:1788619932.147581:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619932.147730:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571714304 00000100:00100000:1.0:1788619932.147740:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571714304 00000100:00100000:1.0:1788619932.147808:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1556 00000100:00100000:1.0:1788619932.147818:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7583480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571714304:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619932.147838:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619932.147858:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe5368736ab7 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1db1e expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.147897:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687369e5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da6f expref: 19 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619932.147938:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c71f1500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589257344:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619932.148022:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc600 going to sleep for 11 seconds 00000100:00100000:1.0:1788619932.150662:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c71f1500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589257344:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619932.150689:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 999842 00010000:00010000:1.0:1788619932.150709:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe5368736ab7 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db1e expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.150733:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619932.150736:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619932.150742:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe5368736ab7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db1e expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.150758:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683da00, rc 0) 00000100:00100000:1.0:1788619932.150833:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7583480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571714304:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3016us (3262us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619932.150841:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1556 00000100:00100000:0.0:1788619932.153714:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619932.153788:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571714432 00000100:00100000:2.0:1788619932.153806:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571714432 00000100:00100000:2.0:1788619932.153864:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1557 00000100:00100000:2.0:1788619932.153875:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d01c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571714432:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619932.153895:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619932.153925:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6600/0x1cd7fe5368736abe lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1db25 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.153962:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687369e5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da6f expref: 19 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619932.154004:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c09880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589257472:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619932.154092:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164480 going to sleep for 11 seconds 00000100:00100000:2.0:1788619932.156614:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c09880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589257472:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619932.156642:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 999842 00010000:00010000:2.0:1788619932.156661:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6600/0x1cd7fe5368736abe lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db25 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.156682:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619932.156685:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619932.156692:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6600/0x1cd7fe5368736abe lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db25 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.156708:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6600, rc 0) 00000100:00100000:2.0:1788619932.156771:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d01c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571714432:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2897us (3064us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619932.156780:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1557 00000100:00100000:0.0:1788619932.160451:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619932.160630:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571714560 00000100:00100000:2.0:1788619932.160640:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571714560 00000100:00100000:2.0:1788619932.160779:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1278 00000100:00100000:2.0:1788619932.160799:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d00000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+8:14908:x1875503571714560:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619932.160819:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619932.160841:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067546236/0x1cd7fe5368736ac5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1db2c expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.160874:0:7654:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000e4805681/0x1cd7fe5368736aa9 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1376256->1380351) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db10 expref: 5 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.160884:0:7654:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000000d2409dc/0x1cd7fe5368736ab0 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db17 expref: 9 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.160897:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 11) 00010000:00010000:2.0:1788619932.160900:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 11) 00010000:00010000:2.0:1788619932.160921:0:7654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067546236/0x1cd7fe5368736ac5 lrc: 5/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [1048576->1060863] (req 1048576->1060863) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db2c expref: 9 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.160956:0:7654:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000000d2409dc/0x1cd7fe5368736ab0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db17 expref: 9 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.160970:0:7654:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000000d2409dc/0x1cd7fe5368736ab0 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db17 expref: 9 pid: 6697 timeout: 524 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619932.160983:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c09880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589257600:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619932.161079:0:7654:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000e4805681/0x1cd7fe5368736aa9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1376256->1380351) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db10 expref: 5 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.161089:0:7654:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000e4805681/0x1cd7fe5368736aa9 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1376256->1380351) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db10 expref: 5 pid: 6696 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619932.161098:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c08a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589257728:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619932.161148:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164480 going to sleep for 11 seconds 00000100:00100000:2.0:1788619932.163962:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c09880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589257600:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619932.163979:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c08a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589257728:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619932.164015:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000067546236/0x1cd7fe5368736ac5 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [1048576->1060863] (req 1048576->1060863) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db2c expref: 9 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.164038:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6400, rc 0) 00000100:00100000:2.0:1788619932.164094:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d00000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+9:14908:x1875503571714560:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3297us (3655us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619932.164102:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1278 00000100:00100000:0.0:1788619932.166893:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619932.166930:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619932.167055:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571714688 00000100:00100000:2.0:1788619932.167066:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571714688 00010000:00010000:2.0:1788619932.167105:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000000d2409dc/0x1cd7fe5368736ab0 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db17 expref: 9 pid: 6697 timeout: 524 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619932.167142:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571714816 00000100:00100000:2.0:1788619932.167145:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571714816 00010000:00010000:2.0:1788619932.167159:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e4805681/0x1cd7fe5368736aa9 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1376256->1380351) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db10 expref: 6 pid: 6696 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619932.167183:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 512 00000100:00100000:2.0:1788619932.167194:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d02300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571714688:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619932.167208:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619932.167231:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000000d2409dc/0x1cd7fe5368736ab0 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db17 expref: 9 pid: 6697 timeout: 524 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.167246:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000000d2409dc/0x1cd7fe5368736ab0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db17 expref: 9 pid: 6697 timeout: 524 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.167275:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d2409dc/0x1cd7fe5368736ab0 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db17 expref: 9 pid: 6697 timeout: 524 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.167293:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619932.167379:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d02300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571714688:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 187us (495us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619932.167387:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 512 00000100:00100000:2.0:1788619932.167404:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 513 00000100:00100000:2.0:1788619932.167410:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d01180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571714816:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619932.167419:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619932.167428:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000e4805681/0x1cd7fe5368736aa9 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1376256->1380351) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db10 expref: 6 pid: 6696 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.167439:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e4805681/0x1cd7fe5368736aa9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1376256->1380351) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db10 expref: 6 pid: 6696 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.167452:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e4805681/0x1cd7fe5368736aa9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1376256->1380351) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db10 expref: 6 pid: 6696 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.167463:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 11) 00010000:00010000:2.0:1788619932.167468:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 11) 00010000:00010000:2.0:1788619932.167476:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000067546236/0x1cd7fe5368736ac5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1060863) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db2c expref: 7 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.167493:0:13330:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067546236/0x1cd7fe5368736ac5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1060863) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db2c expref: 7 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.167543:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000067546236/0x1cd7fe5368736ac5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1060863) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db2c expref: 7 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619932.167557:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c08a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589257856:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619932.167607:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164480 going to sleep for 11 seconds 00000100:00100000:2.0:1788619932.170850:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c08a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589257856:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619932.170878:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619932.170935:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d01180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571714816:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3523us (4003us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619932.170946:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 513 00000100:00100000:0.0:1788619932.172191:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619932.172274:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571714944 00000100:00100000:1.0:1788619932.172283:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571714944 00002000:00100000:1.0:1788619932.172378:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1056768->1060863) req@ffff8e77c6d03480 x1875503571714944/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619932.172390:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6d03480 with x1875503571714944 ext(1056768->1060863) 00002000:00010000:1.0:1788619932.172396:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6d03480 00000100:00100000:1.0:1788619932.172424:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 759 00000100:00100000:1.0:1788619932.172435:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d03480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571714944:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619932.172694:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d03480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571714944:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 260us (508us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619932.172702:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 759 00002000:00100000:1.0:1788619932.172708:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1056768->1060863) req@ffff8e77c6d03480 x1875503571714944/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619932.172718:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c6d03480 with x1875503571714944 ext(1056768->1060863) 00002000:00010000:1.0:1788619932.172721:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c6d03480 00000100:00100000:0.0:1788619932.177429:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619932.177472:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619932.177604:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571715072 00000100:00100000:3.0:1788619932.177614:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571715072 00000100:00100000:3.0:1788619932.177673:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1558 00000100:00100000:3.0:1788619932.177683:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d02d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571715072:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619932.177705:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619932.177725:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7800/0x1cd7fe5368736acc lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1db33 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.177762:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687369e5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da6f expref: 19 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619932.177816:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c7582680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589257984:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619932.177894:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c000 going to sleep for 11 seconds 00000100:00100000:1.0:1788619932.178396:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571715200 00000100:00100000:1.0:1788619932.178401:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571715200 00000100:00100000:1.0:1788619932.178443:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1279 00000100:00100000:1.0:1788619932.178451:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d01f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571715200:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619932.178461:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619932.178472:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001391204f/0x1cd7fe5368736ad3 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1db3a expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619932.178509:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c71f1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589258112:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619932.178609:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc840 going to sleep for 11 seconds 00000004:00010000:3.0:1788619932.181825:0:5846:0:(mdt_lvb.c:177:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size: 999842 -> 1048576 00000004:00010000:3.0:1788619932.181836:0:5846:0:(mdt_lvb.c:183:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime: 1788619931 -> 1788619932 00000004:00010000:3.0:1788619932.181840:0:5846:0:(mdt_lvb.c:195:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime: 1788619931 -> 1788619932 00000100:00100000:3.0:1788619932.181868:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c7582680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589257984:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619932.181894:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:3.0:1788619932.181916:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7800/0x1cd7fe5368736acc lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db33 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.181937:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619932.181940:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619932.181946:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7800/0x1cd7fe5368736acc lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db33 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.181961:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d7800, rc 0) 00000100:00100000:3.0:1788619932.182022:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d02d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571715072:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4339us (4548us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619932.182031:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1558 00002000:00010000:1.0:1788619932.182175:0:6698:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime: 1788619931 -> 1788619932 00002000:00010000:1.0:1788619932.182180:0:6698:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime: 1788619931 -> 1788619932 00000100:00100000:1.0:1788619932.182196:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c71f1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589258112:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619932.182217:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001391204f/0x1cd7fe5368736ad3 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db3a expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.182234:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001391204f/0x1cd7fe5368736ad3 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db3a expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.182243:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683da00, rc 0) 00000100:00100000:1.0:1788619932.182265:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d01f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571715200:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3816us (4849us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619932.182272:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1279 00000100:00100000:0.0:1788619932.185889:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619932.186040:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571715328 00000100:00100000:1.0:1788619932.186049:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571715328 00000100:00100000:1.0:1788619932.186134:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1559 00000100:00100000:1.0:1788619932.186146:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d01880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571715328:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619932.186168:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619932.186191:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368736ada lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1db41 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.186222:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687369e5 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da6f expref: 19 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.186234:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368736ada lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db41 expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.186271:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687369e5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da6f expref: 19 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.186287:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687369e5 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da6f expref: 19 pid: 5844 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619932.186302:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c71f1500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589258240:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619932.186564:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc840 going to sleep for 11 seconds 00000100:00100000:1.0:1788619932.190054:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619932.190212:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c71f1500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589258240:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619932.190249:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368736ada lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db41 expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.190269:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619932.190272:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619932.190277:0:5844:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368736ada lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db41 expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.190289:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683cc00, rc 0) 00000100:00100000:3.0:1788619932.190375:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d01880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:14908:x1875503571715328:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4230us (4497us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619932.190385:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1559 00000100:00100000:3.0:1788619932.191560:0:13346:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571715456 00000100:00100000:3.0:1788619932.191571:0:13346:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571715456 00002000:00100000:3.0:1788619932.191616:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c71f3800 x1875503571715456/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:28/0 lens 12776/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619932.191626:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c71f3800 with x1875503571715456 00010000:00010000:3.0:1788619932.191634:0:13346:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687369e5 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da6f expref: 20 pid: 5844 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.191649:0:13346:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687369e5 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da6f expref: 20 pid: 5844 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788619932.191656:0:13346:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e77c71f3800 00000100:00100000:3.0:1788619932.191687:0:13346:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 40 00000100:00100000:3.0:1788619932.191697:0:13346:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2354:x1875503571715456:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:3.0:1788619932.192090:0:13346:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1048576 -> 1048576 00000004:00010000:3.0:1788619932.192095:0:13346:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619932 -> 1788619932 00000004:00010000:3.0:1788619932.192098:0:13346:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619931 -> 1788619930 00000004:00010000:3.0:1788619932.192101:0:13346:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619932 -> 1788619932 00000004:00010000:3.0:1788619932.192103:0:13346:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1544 -> 1544 00000100:00100000:3.0:1788619932.192155:0:13346:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2354:x1875503571715456:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 458us (2110us total) trans 4294967839 rc 0/0 00000100:00100000:3.0:1788619932.192164:0:13346:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 40 00002000:00100000:3.0:1788619932.192171:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c71f3800 x1875503571715456/t4294967839(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:28/0 lens 12776/448 e 0 to 0 dl 1788619943 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619932.192179:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c71f3800 with x1875503571715456 00010000:00010000:3.0:1788619932.192184:0:13346:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687369e5 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da6f expref: 22 pid: 5844 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.192195:0:13346:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687369e5 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da6f expref: 22 pid: 5844 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788619932.192202:0:13346:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e77c71f3800 00000100:00100000:1.0:1788619932.197392:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619932.197529:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571715584 00000100:00100000:3.0:1788619932.197539:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571715584 00010000:00010000:3.0:1788619932.197583:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687369e5 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da6f expref: 22 pid: 5844 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619932.197626:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 514 00000100:00100000:3.0:1788619932.197637:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f1180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2354:x1875503571715584:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619932.197652:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619932.197681:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687369e5 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da6f expref: 22 pid: 5844 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.197695:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687369e5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da6f expref: 22 pid: 5844 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.197719:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687369e5 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1da6f expref: 22 pid: 5844 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.197736:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619932.197742:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368736ada lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db41 expref: 9 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.197755:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368736ada lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db41 expref: 9 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.197765:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368736ada lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db41 expref: 9 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.197820:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368736ada lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db41 expref: 9 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619932.197836:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c7582680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589258368:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619932.197913:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48c000 going to sleep for 11 seconds 00000100:00100000:3.0:1788619932.203995:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c7582680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589258368:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619932.204022:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619932.204083:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f1180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2354:x1875503571715584:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 6447us (6701us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619932.204094:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 514 00000100:00100000:0.0:1788619932.206819:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619932.206952:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571715712 00000100:00100000:1.0:1788619932.206963:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571715712 00002000:00100000:1.0:1788619932.207011:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6d03100 x1875503571715712/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619932.207023:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6d03100 with x1875503571715712 00000100:00100000:1.0:1788619932.207059:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 939 00000100:00100000:1.0:1788619932.207069:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d03100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571715712:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619932.207700:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d03100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571715712:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 256us (514us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619932.207712:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 939 00002000:00100000:1.0:1788619932.207720:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6d03100 x1875503571715712/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619932.207729:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6d03100 with x1875503571715712 00000100:00100000:1.0:1788619932.210875:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619932.211003:0:13346:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571715840 00000100:00100000:3.0:1788619932.211012:0:13346:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571715840 00002000:00100000:3.0:1788619932.211061:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c71f3b80 x1875503571715840/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619932.211073:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c71f3b80 with x1875503571715840 00000100:00100000:3.0:1788619932.211109:0:13346:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 940 00000100:00100000:3.0:1788619932.211118:0:13346:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571715840:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619932.211396:0:13346:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571715840:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 278us (529us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619932.211404:0:13346:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 940 00002000:00100000:3.0:1788619932.211411:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c71f3b80 x1875503571715840/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619932.211420:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c71f3b80 with x1875503571715840 00000100:00100000:0.0:1788619932.214809:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619932.214947:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571715968 00000100:00100000:2.0:1788619932.214957:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571715968 00000100:00100000:2.0:1788619932.215015:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1560 00000100:00100000:2.0:1788619932.215025:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d01500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2355:x1875503571715968:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619932.215048:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619932.215068:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe5368736ae1 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1db48 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.215104:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe5368736ae1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db48 expref: 22 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1788619932.215124:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:2.0:1788619932.215131:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe5368736ae1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db48 expref: 22 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.215145:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619932.215148:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619932.215152:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6000, rc 0) 00000100:00100000:2.0:1788619932.215213:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d01500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2355:x1875503571715968:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 189us (425us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619932.215220:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1560 00000100:00100000:1.0:1788619932.218291:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619932.218394:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571716096 00000100:00100000:3.0:1788619932.218403:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571716096 00000100:00100000:3.0:1788619932.218461:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1280 00000100:00100000:3.0:1788619932.218471:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571716096:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619932.218490:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619932.218510:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8dcc7dd/0x1cd7fe5368736ae8 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1db4f expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619932.218587:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c7582680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589258496:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619932.218672:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48ccc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619932.220720:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c7582680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589258496:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619932.220759:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b8dcc7dd/0x1cd7fe5368736ae8 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db4f expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.220783:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8dcc7dd/0x1cd7fe5368736ae8 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db4f expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.220816:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d7800, rc 0) 00000100:00100000:3.0:1788619932.220872:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571716096:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2401us (2589us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619932.220880:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1280 00000100:00100000:0.0:1788619932.223258:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619932.223361:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571716224 00000100:00100000:2.0:1788619932.223370:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571716224 00000100:00100000:2.0:1788619932.223430:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1281 00000100:00100000:2.0:1788619932.223441:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571716224:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619932.223459:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619932.223476:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5ec8983/0x1cd7fe5368736aef lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1db56 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619932.223545:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c08a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589258624:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619932.223628:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164240 going to sleep for 11 seconds 00000100:00100000:2.0:1788619932.225593:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c08a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589258624:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619932.225627:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a5ec8983/0x1cd7fe5368736aef lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db56 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.225649:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5ec8983/0x1cd7fe5368736aef lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db56 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.225663:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6600, rc 0) 00000100:00100000:2.0:1788619932.225714:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571716224:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2274us (2465us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619932.225722:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1281 00000100:00100000:0.0:1788619932.232114:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619932.232198:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571716352 00000100:00100000:2.0:1788619932.232210:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571716352 00000100:00100000:2.0:1788619932.232268:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1282 00000100:00100000:2.0:1788619932.232279:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571716352:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619932.232297:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619932.232313:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1c87370/0x1cd7fe5368736af6 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1db5d expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619932.232401:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c08a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589258752:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619932.232483:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164240 going to sleep for 11 seconds 00000100:00100000:2.0:1788619932.235224:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c08a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589258752:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619932.235262:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b1c87370/0x1cd7fe5368736af6 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db5d expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.235288:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1c87370/0x1cd7fe5368736af6 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db5d expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.235302:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6200, rc 0) 00000100:00100000:2.0:1788619932.235398:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571716352:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3120us (3295us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619932.235408:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1282 00000100:00100000:1.0:1788619932.238378:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619932.238477:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571716480 00000100:00100000:3.0:1788619932.238489:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571716480 00000100:00100000:3.0:1788619932.238554:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1283 00000100:00100000:3.0:1788619932.238564:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571716480:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619932.238582:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619932.238599:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8dcc7dd/0x1cd7fe5368736afd lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1db64 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619932.238659:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c7582680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589258880:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619932.238741:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48ccc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619932.241655:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c7582680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589258880:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619932.241697:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b8dcc7dd/0x1cd7fe5368736afd lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db64 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.241724:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8dcc7dd/0x1cd7fe5368736afd lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db64 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.241738:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d7800, rc 0) 00000100:00100000:2.0:1788619932.241919:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571716480:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3354us (3599us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619932.241929:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1283 00000100:00100000:1.0:1788619932.245181:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619932.245251:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571716608 00000100:00100000:3.0:1788619932.245260:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571716608 00000100:00100000:3.0:1788619932.245372:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1284 00000100:00100000:3.0:1788619932.245382:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571716608:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619932.245398:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619932.245413:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e6a43d0/0x1cd7fe5368736b04 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1db6b expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619932.245466:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c7580380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589259008:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619932.245551:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78dc48ccc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619932.248089:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c7580380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589259008:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619932.248125:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005e6a43d0/0x1cd7fe5368736b04 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db6b expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.248149:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e6a43d0/0x1cd7fe5368736b04 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db6b expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.248162:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d6a00, rc 0) 00000100:00100000:3.0:1788619932.248244:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571716608:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2863us (3071us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619932.248252:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1284 00000100:00100000:1.0:1788619932.251142:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619932.251236:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571716736 00000100:00100000:2.0:1788619932.251246:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571716736 00000100:00100000:2.0:1788619932.251362:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1285 00000100:00100000:2.0:1788619932.251372:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64d3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:14908:x1875503571716736:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619932.251390:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619932.251406:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b1c87370/0x1cd7fe5368736b0b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1db72 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.251437:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000067546236/0x1cd7fe5368736ac5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1060863) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db2c expref: 7 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.251447:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(2 11) 00010000:00010000:2.0:1788619932.251452:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(3 11) 00010000:00010000:2.0:1788619932.251458:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b1c87370/0x1cd7fe5368736b0b lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1064960->1069055] (req 1064960->1069055) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db72 expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.251489:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000067546236/0x1cd7fe5368736ac5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1060863) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db2c expref: 7 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.251503:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000067546236/0x1cd7fe5368736ac5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1060863) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db2c expref: 7 pid: 7654 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619932.251529:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c08a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589259136:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619932.251609:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164240 going to sleep for 11 seconds 00000100:00100000:3.0:1788619932.254900:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619932.254950:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571716864 00000100:00100000:3.0:1788619932.254959:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571716864 00002000:00100000:3.0:1788619932.255008:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1060863) req@ffff8e78c7580380 x1875503571716864/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:28/0 lens 12776/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619932.255020:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7580380 with x1875503571716864 ext(1048576->1060863) 00010000:00010000:3.0:1788619932.255030:0:6701:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 0000000067546236/0x1cd7fe5368736ac5 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1060863) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db2c expref: 8 pid: 7654 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.255049:0:6701:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 0000000067546236/0x1cd7fe5368736ac5 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1060863) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db2c expref: 8 pid: 7654 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788619932.255058:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff8e78c7580380 00000100:00100000:3.0:1788619932.255097:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 26 00000100:00100000:3.0:1788619932.255107:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7580380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571716864:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:3.0:1788619932.255592:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7580380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+10:2354:x1875503571716864:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 484us (699us total) trans 4294967517 rc 0/0 00000100:00100000:3.0:1788619932.255601:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 26 00002000:00100000:3.0:1788619932.255608:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1060863) req@ffff8e78c7580380 x1875503571716864/t4294967517(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:28/0 lens 12776/448 e 0 to 0 dl 1788619943 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619932.255618:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7580380 with x1875503571716864 ext(1048576->1060863) 00010000:00010000:3.0:1788619932.255624:0:6701:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 0000000067546236/0x1cd7fe5368736ac5 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1060863) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db2c expref: 10 pid: 7654 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.255638:0:6701:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 0000000067546236/0x1cd7fe5368736ac5 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1060863) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db2c expref: 10 pid: 7654 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788619932.255646:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff8e78c7580380 00000100:00100000:2.0:1788619932.259111:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c08a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589259136:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619932.259158:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b1c87370/0x1cd7fe5368736b0b lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1064960->1069055] (req 1064960->1069055) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db72 expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.259184:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc6200, rc 0) 00000100:00100000:2.0:1788619932.259264:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64d3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:14908:x1875503571716736:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7892us (8132us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619932.259273:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1285 00000100:00100000:3.0:1788619932.259562:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619932.259883:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571716992 00000100:00100000:0.0:1788619932.259891:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571716992 00010000:00010000:0.0:1788619932.259943:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000067546236/0x1cd7fe5368736ac5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->1060863) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db2c expref: 10 pid: 7654 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619932.259986:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 515 00000100:00100000:0.0:1788619932.259996:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcaf050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+10:2354:x1875503571716992:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619932.260010:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619932.260033:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000067546236/0x1cd7fe5368736ac5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1060863) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db2c expref: 10 pid: 7654 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619932.260046:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000067546236/0x1cd7fe5368736ac5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1060863) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db2c expref: 10 pid: 7654 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619932.260076:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067546236/0x1cd7fe5368736ac5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->1060863) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db2c expref: 10 pid: 7654 timeout: 524 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619932.260091:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(3 11) 00010000:00010000:0.0:1788619932.260096:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(3 11) 00010000:00010000:0.0:1788619932.260103:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b1c87370/0x1cd7fe5368736b0b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1064960->1069055) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db72 expref: 5 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619932.260113:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b1c87370/0x1cd7fe5368736b0b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1064960->1069055) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db72 expref: 5 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619932.260147:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000b1c87370/0x1cd7fe5368736b0b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1064960->1069055) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db72 expref: 5 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619932.260162:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78cdcc0700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589259264:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619932.260229:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5d2de40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619932.264624:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78cdcc0700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589259264:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619932.264652:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619932.264714:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcaf050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571716992:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 4718us (5152us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619932.264727:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 515 00000100:00100000:3.0:1788619932.265700:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619932.266553:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571717120 00000100:00100000:1.0:1788619932.266566:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571717120 00002000:00100000:1.0:1788619932.266618:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1064960->1069055) req@ffff8e78c7581880 x1875503571717120/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619932.266631:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7581880 with x1875503571717120 ext(1064960->1069055) 00002000:00010000:1.0:1788619932.266637:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c7581880 00000100:00100000:1.0:1788619932.266662:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 760 00000100:00100000:1.0:1788619932.266672:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7581880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571717120:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619932.266872:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7581880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571717120:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 200us (1176us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619932.266880:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 760 00002000:00100000:1.0:1788619932.266887:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1064960->1069055) req@ffff8e78c7581880 x1875503571717120/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619932.266896:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7581880 with x1875503571717120 ext(1064960->1069055) 00002000:00010000:1.0:1788619932.266900:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c7581880 00000100:00100000:3.0:1788619932.270700:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619932.270824:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571717248 00000100:00100000:0.0:1788619932.270836:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571717248 00002000:00100000:0.0:1788619932.270886:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1069056->1073151) req@ffff8e78c7582a00 x1875503571717248/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619932.270899:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7582a00 with x1875503571717248 ext(1069056->1073151) 00002000:00010000:0.0:1788619932.270915:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c7582a00 00000100:00100000:0.0:1788619932.270949:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 761 00000100:00100000:0.0:1788619932.270959:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7582a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571717248:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619932.271155:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7582a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571717248:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 196us (464us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619932.271163:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 761 00002000:00100000:0.0:1788619932.271171:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1069056->1073151) req@ffff8e78c7582a00 x1875503571717248/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619932.271179:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7582a00 with x1875503571717248 ext(1069056->1073151) 00002000:00010000:0.0:1788619932.271183:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c7582a00 00000100:00100000:1.0:1788619932.276206:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619932.276309:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571717376 00000100:00100000:2.0:1788619932.276319:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571717376 00002000:00100000:2.0:1788619932.276411:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1073152->1077247) req@ffff8e77c64d1880 x1875503571717376/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619932.276423:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64d1880 with x1875503571717376 ext(1073152->1077247) 00002000:00010000:2.0:1788619932.276429:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64d1880 00000100:00100000:2.0:1788619932.276463:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 762 00000100:00100000:2.0:1788619932.276472:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571717376:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619932.276682:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571717376:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 210us (486us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619932.276690:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 762 00002000:00100000:2.0:1788619932.276696:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1073152->1077247) req@ffff8e77c64d1880 x1875503571717376/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619932.276703:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64d1880 with x1875503571717376 ext(1073152->1077247) 00002000:00010000:2.0:1788619932.276707:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64d1880 00000100:00100000:1.0:1788619932.279836:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619932.279933:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571717504 00000100:00100000:0.0:1788619932.279943:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571717504 00002000:00100000:0.0:1788619932.279991:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1077248->1081343) req@ffff8e77c64d1180 x1875503571717504/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619932.280003:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64d1180 with x1875503571717504 ext(1077248->1081343) 00002000:00010000:0.0:1788619932.280009:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64d1180 00000100:00100000:0.0:1788619932.280044:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 763 00000100:00100000:0.0:1788619932.280054:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64d1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571717504:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619932.280245:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64d1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571717504:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 192us (419us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619932.280254:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 763 00002000:00100000:0.0:1788619932.280262:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1077248->1081343) req@ffff8e77c64d1180 x1875503571717504/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619932.280272:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64d1180 with x1875503571717504 ext(1077248->1081343) 00002000:00010000:0.0:1788619932.280276:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64d1180 00000100:00100000:3.0:1788619932.284220:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619932.284533:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571717632 00000100:00100000:2.0:1788619932.284543:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571717632 00002000:00100000:2.0:1788619932.284593:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1081344->1085439) req@ffff8e78c7580700 x1875503571717632/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619932.284605:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7580700 with x1875503571717632 ext(1081344->1085439) 00002000:00010000:2.0:1788619932.284611:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c7580700 00000100:00100000:2.0:1788619932.284642:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 764 00000100:00100000:2.0:1788619932.284651:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7580700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571717632:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619932.284990:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7580700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571717632:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 338us (778us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619932.284998:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 764 00002000:00100000:2.0:1788619932.285005:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1081344->1085439) req@ffff8e78c7580700 x1875503571717632/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619932.285014:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7580700 with x1875503571717632 ext(1081344->1085439) 00002000:00010000:2.0:1788619932.285018:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c7580700 00000100:00100000:1.0:1788619932.288069:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619932.288153:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571717760 00000100:00100000:0.0:1788619932.288162:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571717760 00002000:00100000:0.0:1788619932.288203:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1085440->1089535) req@ffff8e77c64d2d80 x1875503571717760/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619932.288215:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64d2d80 with x1875503571717760 ext(1085440->1089535) 00002000:00010000:0.0:1788619932.288220:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64d2d80 00000100:00100000:0.0:1788619932.288275:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 765 00000100:00100000:0.0:1788619932.288284:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64d2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571717760:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619932.288527:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64d2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571717760:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 234us (457us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619932.288535:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 765 00002000:00100000:0.0:1788619932.288542:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1085440->1089535) req@ffff8e77c64d2d80 x1875503571717760/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619932.288552:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64d2d80 with x1875503571717760 ext(1085440->1089535) 00002000:00010000:0.0:1788619932.288555:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64d2d80 00000100:00100000:0.0:1788619932.295603:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619932.295686:0:13347:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571717888 00000100:00100000:1.0:1788619932.295695:0:13347:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571717888 00002000:00100000:1.0:1788619932.295746:0:13347:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x4:0x0] (1089536->1093631) req@ffff8e78cdcc0700 x1875503571717888/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619932.295758:0:13347:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78cdcc0700 with x1875503571717888 ext(1089536->1093631) 00002000:00010000:1.0:1788619932.295764:0:13347:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78cdcc0700 00000100:00100000:1.0:1788619932.295809:0:13347:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 766 00000100:00100000:1.0:1788619932.295820:0:13347:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571717888:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619932.296018:0:13347:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571717888:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 199us (427us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619932.296026:0:13347:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 766 00002000:00100000:1.0:1788619932.296033:0:13347:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x4:0x0] (1089536->1093631) req@ffff8e78cdcc0700 x1875503571717888/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619932.296043:0:13347:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78cdcc0700 with x1875503571717888 ext(1089536->1093631) 00002000:00010000:1.0:1788619932.296046:0:13347:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78cdcc0700 00000100:00100000:1.0:1788619932.305614:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619932.305715:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571718016 00000100:00100000:3.0:1788619932.305726:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571718016 00002000:00100000:3.0:1788619932.305781:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1093632->1097727) req@ffff8e77c64d2300 x1875503571718016/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619932.305804:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64d2300 with x1875503571718016 ext(1093632->1097727) 00002000:00010000:3.0:1788619932.305811:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64d2300 00000100:00100000:3.0:1788619932.305849:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 767 00000100:00100000:3.0:1788619932.305859:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64d2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571718016:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619932.306056:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64d2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571718016:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 197us (450us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619932.306063:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 767 00002000:00100000:3.0:1788619932.306070:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1093632->1097727) req@ffff8e77c64d2300 x1875503571718016/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619932.306078:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64d2300 with x1875503571718016 ext(1093632->1097727) 00002000:00010000:3.0:1788619932.306082:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64d2300 00000100:00100000:3.0:1788619932.311685:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619932.312088:0:13347:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571718144 00000100:00100000:1.0:1788619932.312098:0:13347:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571718144 00002000:00100000:1.0:1788619932.312148:0:13347:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x4:0x0] (1097728->1101823) req@ffff8e78c7583800 x1875503571718144/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619932.312160:0:13347:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7583800 with x1875503571718144 ext(1097728->1101823) 00002000:00010000:1.0:1788619932.312165:0:13347:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c7583800 00000100:00100000:1.0:1788619932.312244:0:13347:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 768 00000100:00100000:1.0:1788619932.312253:0:13347:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7583800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571718144:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619932.312476:0:13347:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7583800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571718144:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 224us (800us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619932.312484:0:13347:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 768 00002000:00100000:1.0:1788619932.312491:0:13347:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x4:0x0] (1097728->1101823) req@ffff8e78c7583800 x1875503571718144/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619932.312500:0:13347:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7583800 with x1875503571718144 ext(1097728->1101823) 00002000:00010000:1.0:1788619932.312504:0:13347:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c7583800 00000100:00100000:1.0:1788619932.316817:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619932.316906:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571718272 00000100:00100000:3.0:1788619932.316916:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571718272 00002000:00100000:3.0:1788619932.316969:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1101824->1105919) req@ffff8e77c64d3100 x1875503571718272/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619932.316981:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64d3100 with x1875503571718272 ext(1101824->1105919) 00002000:00010000:3.0:1788619932.316987:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64d3100 00000100:00100000:3.0:1788619932.317022:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 769 00000100:00100000:3.0:1788619932.317032:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64d3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571718272:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619932.317224:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64d3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571718272:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 192us (419us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619932.317232:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 769 00002000:00100000:3.0:1788619932.317239:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1101824->1105919) req@ffff8e77c64d3100 x1875503571718272/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619932.317248:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64d3100 with x1875503571718272 ext(1101824->1105919) 00002000:00010000:3.0:1788619932.317251:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64d3100 00000100:00100000:3.0:1788619932.320490:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619932.320602:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571718400 00000100:00100000:2.0:1788619932.320612:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571718400 00002000:00100000:2.0:1788619932.320659:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1105920->1110015) req@ffff8e78c7581180 x1875503571718400/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619932.320670:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7581180 with x1875503571718400 ext(1105920->1110015) 00002000:00010000:2.0:1788619932.320675:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c7581180 00000100:00100000:2.0:1788619932.320709:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 770 00000100:00100000:2.0:1788619932.320718:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7581180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571718400:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619932.321019:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7581180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571718400:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 302us (538us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619932.321028:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 770 00002000:00100000:2.0:1788619932.321035:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1105920->1110015) req@ffff8e78c7581180 x1875503571718400/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619932.321044:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7581180 with x1875503571718400 ext(1105920->1110015) 00002000:00010000:2.0:1788619932.321047:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c7581180 00000100:00100000:1.0:1788619932.327975:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619932.331554:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571718528 00000100:00100000:3.0:1788619932.331572:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571718528 00002000:00100000:2.0:1788619932.335471:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1110016->1114111) req@ffff8e77c64d0380 x1875503571718528/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619932.335487:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64d0380 with x1875503571718528 ext(1110016->1114111) 00002000:00010000:2.0:1788619932.335494:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64d0380 00000100:00100000:2.0:1788619932.335548:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 771 00000100:00100000:2.0:1788619932.335560:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64d0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571718528:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619932.335920:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64d0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571718528:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 362us (7954us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619932.335929:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 771 00002000:00100000:2.0:1788619932.335937:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1110016->1114111) req@ffff8e77c64d0380 x1875503571718528/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619932.335946:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64d0380 with x1875503571718528 ext(1110016->1114111) 00002000:00010000:2.0:1788619932.335950:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64d0380 00000100:00100000:0.0:1788619932.349608:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619932.352698:0:13347:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571718656 00000100:00100000:1.0:1788619932.352715:0:13347:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571718656 00002000:00100000:1.0:1788619932.353491:0:13347:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x4:0x0] (1114112->1118207) req@ffff8e78cdcc0a80 x1875503571718656/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619932.353508:0:13347:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78cdcc0a80 with x1875503571718656 ext(1114112->1118207) 00002000:00010000:1.0:1788619932.353526:0:13347:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78cdcc0a80 00000100:00100000:1.0:1788619932.353640:0:13347:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 772 00000100:00100000:1.0:1788619932.353655:0:13347:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571718656:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619932.353959:0:13347:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571718656:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 309us (4359us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619932.353969:0:13347:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 772 00002000:00100000:1.0:1788619932.353978:0:13347:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x4:0x0] (1114112->1118207) req@ffff8e78cdcc0a80 x1875503571718656/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619932.353987:0:13347:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78cdcc0a80 with x1875503571718656 ext(1114112->1118207) 00002000:00010000:1.0:1788619932.353991:0:13347:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78cdcc0a80 00000100:00100000:3.0:1788619932.359917:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619932.364203:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571718784 00000100:00100000:2.0:1788619932.364220:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571718784 00002000:00100000:2.0:1788619932.364282:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1064960->1118207) req@ffff8e78c7581f80 x1875503571718784/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619932.364293:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7581f80 with x1875503571718784 ext(1064960->1118207) 00002000:00010000:2.0:1788619932.364300:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c7581f80 00000100:00100000:1.0:1788619932.364442:0:13347:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 773 00000100:00100000:1.0:1788619932.364455:0:13347:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7581f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571718784:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:0.0:1788619932.376011:0:13347:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7581f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+8:2354:x1875503571718784:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 11548us (16093us total) trans 4294967518 rc 0/0 00000100:00100000:0.0:1788619932.376025:0:13347:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 773 00002000:00100000:0.0:1788619932.376040:0:13347:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x4:0x0] (1064960->1118207) req@ffff8e78c7581f80 x1875503571718784/t4294967518(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/448 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619932.376051:0:13347:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7581f80 with x1875503571718784 ext(1064960->1118207) 00002000:00010000:0.0:1788619932.376058:0:13347:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c7581f80 00000100:00100000:3.0:1788619932.380024:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619932.380093:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571718912 00000100:00100000:0.0:1788619932.380104:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571718912 00000100:00100000:0.0:1788619932.380164:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1286 00000100:00100000:0.0:1788619932.380174:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7583100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+8:2352:x1875503571718912:12345-192.168.201.5@tcp:16:fsx.0 00000001:00080000:3.0:1788619932.431830:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967518 is committed 00000001:00080000:3.0:1788619932.431861:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967515 is committed 00000001:00080000:3.0:1788619932.431876:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967517 is committed 00000100:00100000:2.0:1788619932.432133:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7583100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571718912:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 51954us (52113us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619932.432146:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1286 00000100:00100000:2.0:1788619932.436050:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619932.436318:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571719040 00000100:00100000:1.0:1788619932.436369:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571719040 00000100:00100000:1.0:1788619932.436428:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1287 00000100:00100000:1.0:1788619932.436439:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c08a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:14908:x1875503571719040:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619932.436462:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619932.436483:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001fc792ba/0x1cd7fe5368736b12 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1db79 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.436531:0:6698:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000b1c87370/0x1cd7fe5368736b0b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1064960->1069055) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db72 expref: 5 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.436541:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=1s v=(3 11) 00010000:00010000:1.0:1788619932.436546:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(4 11) 00010000:00010000:1.0:1788619932.436551:0:6698:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001fc792ba/0x1cd7fe5368736b12 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1052672->1118207] (req 1052672->1118207) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db79 expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.436586:0:6698:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000b1c87370/0x1cd7fe5368736b0b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1064960->1069055) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db72 expref: 5 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.436600:0:6698:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000b1c87370/0x1cd7fe5368736b0b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1064960->1069055) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db72 expref: 5 pid: 6697 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619932.436615:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64d2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589259392:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619932.436708:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc240 going to sleep for 11 seconds 00000100:00100000:1.0:1788619932.439213:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64d2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589259392:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619932.439252:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001fc792ba/0x1cd7fe5368736b12 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1052672->1118207] (req 1052672->1118207) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db79 expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.439275:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d800, rc 0) 00000100:00100000:1.0:1788619932.439477:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c08a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:14908:x1875503571719040:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3037us (3435us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619932.439486:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1287 00000100:00100000:2.0:1788619932.442776:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619932.443013:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571719168 00000100:00100000:1.0:1788619932.443024:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571719168 00010000:00010000:1.0:1788619932.443065:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000b1c87370/0x1cd7fe5368736b0b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1064960->1069055) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db72 expref: 6 pid: 6697 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619932.443220:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 516 00000100:00100000:1.0:1788619932.443230:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c09880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571719168:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619932.443249:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619932.443272:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000b1c87370/0x1cd7fe5368736b0b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1064960->1069055) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db72 expref: 6 pid: 6697 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.443286:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000b1c87370/0x1cd7fe5368736b0b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1064960->1069055) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db72 expref: 6 pid: 6697 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.443312:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b1c87370/0x1cd7fe5368736b0b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1064960->1069055) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db72 expref: 6 pid: 6697 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.443421:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(4 11) 00010000:00010000:1.0:1788619932.443427:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(4 11) 00010000:00010000:1.0:1788619932.443437:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000001fc792ba/0x1cd7fe5368736b12 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1052672->1118207) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db79 expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.443448:0:5831:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001fc792ba/0x1cd7fe5368736b12 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1052672->1118207) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db79 expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.443488:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000001fc792ba/0x1cd7fe5368736b12 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1052672->1118207) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db79 expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619932.443503:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64d2a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5831:1875503589259520:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619932.443603:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc240 going to sleep for 11 seconds 00000100:00100000:2.0:1788619932.447299:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619932.447458:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571719296 00000100:00100000:3.0:1788619932.447470:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571719296 00002000:00100000:3.0:1788619932.447597:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1052672->1056767) req@ffff8e78c5c09180 x1875503571719296/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619932.447609:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5c09180 with x1875503571719296 ext(1052672->1056767) 00002000:00010000:3.0:1788619932.447615:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5c09180 00000100:00100000:3.0:1788619932.447648:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 774 00000100:00100000:3.0:1788619932.447657:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c09180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571719296:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619932.447887:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c09180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571719296:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 229us (597us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619932.447895:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 774 00002000:00100000:3.0:1788619932.447916:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1052672->1056767) req@ffff8e78c5c09180 x1875503571719296/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619932.447926:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5c09180 with x1875503571719296 ext(1052672->1056767) 00002000:00010000:3.0:1788619932.447930:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5c09180 00010000:00010000:2.0:1788619932.450882:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788619932.451265:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619932.451637:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571719424 00000100:00100000:0.0:1788619932.451648:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571719424 00002000:00100000:0.0:1788619932.451697:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1114112->1118207) req@ffff8e78c5c0bb80 x1875503571719424/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619932.451710:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5c0bb80 with x1875503571719424 ext(1114112->1118207) 00002000:00010000:0.0:1788619932.451715:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5c0bb80 00000100:00100000:0.0:1788619932.451742:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 775 00000100:00100000:0.0:1788619932.451752:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c0bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571719424:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619932.451942:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c0bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571719424:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 191us (683us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619932.451951:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 775 00002000:00100000:0.0:1788619932.451957:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1114112->1118207) req@ffff8e78c5c0bb80 x1875503571719424/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619932.451966:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5c0bb80 with x1875503571719424 ext(1114112->1118207) 00002000:00010000:0.0:1788619932.451969:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5c0bb80 00000100:00100000:1.0:1788619932.452982:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64d2a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5831:1875503589259520:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619932.453006:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619932.453042:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c09880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571719168:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 9814us (10276us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619932.453051:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 516 00000100:00100000:2.0:1788619932.456760:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619932.456952:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571719552 00000100:00100000:1.0:1788619932.456962:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571719552 00000100:00100000:1.0:1788619932.457017:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1288 00000100:00100000:1.0:1788619932.457027:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c0b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571719552:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619932.457046:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619932.457065:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe5368736b19 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1db80 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619932.457124:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64d2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589259648:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619932.457203:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc240 going to sleep for 11 seconds 00000100:00100000:1.0:1788619932.459610:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64d2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589259648:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619932.459646:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe5368736b19 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db80 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.459676:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe5368736b19 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db80 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.459690:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683dc00, rc 0) 00000100:00100000:1.0:1788619932.459743:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c0b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571719552:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2716us (2993us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619932.459752:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1288 00000100:00100000:0.0:1788619932.462262:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619932.462610:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571719680 00000100:00100000:2.0:1788619932.462620:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571719680 00000100:00100000:2.0:1788619932.462676:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1289 00000100:00100000:2.0:1788619932.462687:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571719680:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619932.462704:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619932.462721:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041aa2bf0/0x1cd7fe5368736b20 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1db87 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619932.462793:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c7433480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589259776:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619932.462871:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81640c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619932.465401:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c7433480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589259776:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619932.465440:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000041aa2bf0/0x1cd7fe5368736b20 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db87 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619932.465465:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041aa2bf0/0x1cd7fe5368736b20 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db87 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619932.465479:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78cdcc7a00, rc 0) 00000100:00100000:0.0:1788619932.465541:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571719680:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2855us (3288us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619932.465550:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1289 00000100:00100000:0.0:1788619932.472280:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619932.472382:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571719808 00000100:00100000:1.0:1788619932.472392:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571719808 00002000:00100000:1.0:1788619932.472440:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78cdcc1f80 x1875503571719808/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 504/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619932.472452:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78cdcc1f80 with x1875503571719808 00000100:00100000:1.0:1788619932.472487:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 941 00000100:00100000:1.0:1788619932.472497:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571719808:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619932.494858:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571719808:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 22348us (22575us total) trans 0 rc 1040384/1040384 00000100:00100000:1.0:1788619932.494874:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 941 00002000:00100000:1.0:1788619932.494889:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78cdcc1f80 x1875503571719808/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 504/440 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 1040384/1040384 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619932.494901:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78cdcc1f80 with x1875503571719808 00000100:00100000:2.0:1788619932.502894:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619932.503028:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571719936 00000100:00100000:3.0:1788619932.503038:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571719936 00002000:00100000:3.0:1788619932.503089:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1052672->1118207) req@ffff8e78c7430e00 x1875503571719936/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619932.503107:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7430e00 with x1875503571719936 ext(1052672->1118207) 00002000:00010000:3.0:1788619932.503112:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c7430e00 00000100:00100000:3.0:1788619932.503149:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 776 00000100:00100000:3.0:1788619932.503160:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7430e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571719936:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:3.0:1788619932.508872:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7430e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571719936:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 5703us (5979us total) trans 4294967519 rc 0/0 00000100:00100000:3.0:1788619932.508889:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 776 00002000:00100000:3.0:1788619932.508903:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1052672->1118207) req@ffff8e78c7430e00 x1875503571719936/t4294967519(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:28/0 lens 488/448 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619932.508915:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7430e00 with x1875503571719936 ext(1052672->1118207) 00002000:00010000:3.0:1788619932.508922:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c7430e00 00000100:00100000:0.0:1788619932.511606:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619932.511689:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571720064 00000100:00100000:3.0:1788619932.511700:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571720064 00000100:00100000:3.0:1788619932.511769:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 777 00000100:00100000:3.0:1788619932.511884:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2352:x1875503571720064:12345-192.168.201.5@tcp:22:fsx.0 00002000:00010000:3.0:1788619932.512173:0:6701:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1410173 -> 1410173 00002000:00010000:3.0:1788619932.512177:0:6701:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619932 -> 1788619932 00002000:00010000:3.0:1788619932.512180:0:6701:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619931 -> 1788619931 00002000:00010000:3.0:1788619932.512182:0:6701:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619932 -> 1788619932 00002000:00010000:3.0:1788619932.512185:0:6701:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 519 -> 391 00000100:00100000:3.0:1788619932.512241:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2352:x1875503571720064:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 462us (642us total) trans 4294967520 rc 0/0 00000100:00100000:3.0:1788619932.512249:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 777 00000100:00100000:2.0:1788619932.515643:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619932.515744:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571720192 00000100:00100000:0.0:1788619932.515755:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571720192 00000100:00100000:0.0:1788619932.515919:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1290 00000100:00100000:0.0:1788619932.515930:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7432d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571720192:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619932.515950:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619932.515968:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000064837316/0x1cd7fe5368736b27 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1db8e expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619932.516029:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78cdcc0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589259904:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619932.516154:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5d2d840 going to sleep for 11 seconds 00000100:00100000:0.0:1788619932.518265:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78cdcc0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589259904:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619932.518305:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000064837316/0x1cd7fe5368736b27 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db8e expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619932.518452:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000064837316/0x1cd7fe5368736b27 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db8e expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619932.518467:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c1495a00, rc 0) 00000100:00100000:0.0:1788619932.518536:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7432d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571720192:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2608us (2901us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619932.518545:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1290 00000100:00100000:2.0:1788619932.521268:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619932.521607:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571720320 00000100:00100000:1.0:1788619932.521617:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571720320 00000100:00100000:1.0:1788619932.521706:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1291 00000100:00100000:1.0:1788619932.521718:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7432a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571720320:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619932.521739:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619932.521757:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a72c5200/0x1cd7fe5368736b2e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1db95 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619932.521896:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64d2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589260032:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619932.522105:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc240 going to sleep for 11 seconds 00000100:00100000:1.0:1788619932.524586:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64d2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589260032:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619932.524623:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a72c5200/0x1cd7fe5368736b2e lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db95 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.524648:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a72c5200/0x1cd7fe5368736b2e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db95 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.524662:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c400, rc 0) 00000100:00100000:1.0:1788619932.524755:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7432a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571720320:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3038us (3495us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619932.524765:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1291 00000100:00100000:2.0:1788619932.529013:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619932.529103:0:13346:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571720448 00000100:00100000:3.0:1788619932.529112:0:13346:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571720448 00002000:00100000:3.0:1788619932.529160:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c7431500 x1875503571720448/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619932.529172:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c7431500 with x1875503571720448 00000100:00100000:3.0:1788619932.529247:0:13346:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 942 00000100:00100000:3.0:1788619932.529258:0:13346:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7431500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2352:x1875503571720448:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619932.539718:0:13346:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7431500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2352:x1875503571720448:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 10452us (10705us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619932.539732:0:13346:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 942 00002000:00100000:3.0:1788619932.539743:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c7431500 x1875503571720448/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619932.539754:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c7431500 with x1875503571720448 00000100:00100000:3.0:1788619932.543098:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619932.543545:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571720576 00000100:00100000:0.0:1788619932.543556:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571720576 00002000:00100000:0.0:1788619932.543606:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c7582680 x1875503571720576/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619932.543618:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c7582680 with x1875503571720576 00000100:00100000:0.0:1788619932.543651:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 943 00000100:00100000:0.0:1788619932.543662:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7582680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2353:x1875503571720576:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619932.543890:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7582680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2353:x1875503571720576:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 229us (801us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619932.543899:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 943 00002000:00100000:0.0:1788619932.543906:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c7582680 x1875503571720576/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619932.543915:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c7582680 with x1875503571720576 00000100:00100000:3.0:1788619932.548395:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619932.548470:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571720704 00000100:00100000:0.0:1788619932.548480:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571720704 00002000:00100000:0.0:1788619932.548539:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c7582300 x1875503571720704/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619932.548552:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c7582300 with x1875503571720704 00000100:00100000:0.0:1788619932.548589:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 944 00000100:00100000:0.0:1788619932.548599:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7582300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2352:x1875503571720704:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619932.548835:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7582300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2352:x1875503571720704:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 236us (448us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619932.548844:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 944 00002000:00100000:0.0:1788619932.548851:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c7582300 x1875503571720704/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619932.548859:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c7582300 with x1875503571720704 00000100:00100000:3.0:1788619932.551573:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619932.551757:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571720832 00000100:00100000:0.0:1788619932.551767:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571720832 00002000:00100000:0.0:1788619932.551821:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c8597480 x1875503571720832/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619932.551833:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c8597480 with x1875503571720832 00000100:00100000:0.0:1788619932.551870:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 945 00000100:00100000:0.0:1788619932.551879:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8597480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2352:x1875503571720832:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619932.552076:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8597480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2352:x1875503571720832:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 197us (512us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619932.552084:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 945 00002000:00100000:0.0:1788619932.552090:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c8597480 x1875503571720832/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619932.552099:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c8597480 with x1875503571720832 00000100:00100000:1.0:1788619932.555168:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619932.555257:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571720960 00000100:00100000:0.0:1788619932.555267:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571720960 00002000:00100000:0.0:1788619932.555316:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c64d2a00 x1875503571720960/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619932.555365:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64d2a00 with x1875503571720960 00000100:00100000:0.0:1788619932.555401:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 946 00000100:00100000:0.0:1788619932.555412:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64d2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2353:x1875503571720960:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619932.555648:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64d2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2353:x1875503571720960:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 237us (491us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619932.555657:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 946 00002000:00100000:0.0:1788619932.555665:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c64d2a00 x1875503571720960/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619932.555673:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64d2a00 with x1875503571720960 00000100:00100000:3.0:1788619932.558632:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619932.558877:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571721088 00000100:00100000:0.0:1788619932.558887:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571721088 00002000:00100000:0.0:1788619932.558939:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c8596300 x1875503571721088/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619932.558951:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c8596300 with x1875503571721088 00000100:00100000:0.0:1788619932.558992:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 947 00000100:00100000:0.0:1788619932.559003:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8596300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2352:x1875503571721088:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619932.559220:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8596300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2352:x1875503571721088:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 217us (596us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619932.559227:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 947 00002000:00100000:0.0:1788619932.559235:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c8596300 x1875503571721088/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619932.559243:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c8596300 with x1875503571721088 00000100:00100000:2.0:1788619932.566716:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619932.566905:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571721216 00000100:00100000:0.0:1788619932.566916:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571721216 00002000:00100000:0.0:1788619932.566969:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c7432680 x1875503571721216/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619932.566982:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c7432680 with x1875503571721216 00000100:00100000:0.0:1788619932.567020:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 948 00000100:00100000:0.0:1788619932.567030:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7432680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2352:x1875503571721216:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619932.567378:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7432680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2352:x1875503571721216:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 348us (673us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619932.567387:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 948 00002000:00100000:0.0:1788619932.567395:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c7432680 x1875503571721216/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619932.567403:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c7432680 with x1875503571721216 00000100:00100000:3.0:1788619932.570448:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619932.570605:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571721344 00000100:00100000:0.0:1788619932.570616:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571721344 00002000:00100000:0.0:1788619932.570667:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c8595180 x1875503571721344/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619932.570679:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c8595180 with x1875503571721344 00000100:00100000:0.0:1788619932.570718:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 949 00000100:00100000:0.0:1788619932.570728:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c8595180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2354:x1875503571721344:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619932.570965:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c8595180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2354:x1875503571721344:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 238us (526us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619932.570973:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 949 00002000:00100000:0.0:1788619932.570980:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c8595180 x1875503571721344/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619932.570995:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c8595180 with x1875503571721344 00000100:00100000:2.0:1788619932.574595:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619932.574681:0:13346:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571721472 00000100:00100000:3.0:1788619932.574692:0:13346:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571721472 00002000:00100000:3.0:1788619932.574747:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c7431f80 x1875503571721472/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619932.574760:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c7431f80 with x1875503571721472 00000100:00100000:3.0:1788619932.574809:0:13346:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 950 00000100:00100000:3.0:1788619932.574818:0:13346:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7431f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2355:x1875503571721472:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619932.575056:0:13346:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7431f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2355:x1875503571721472:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 239us (470us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619932.575064:0:13346:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 950 00002000:00100000:3.0:1788619932.575070:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c7431f80 x1875503571721472/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619932.575077:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c7431f80 with x1875503571721472 00000100:00100000:2.0:1788619932.579558:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619932.579660:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571721600 00000100:00100000:0.0:1788619932.579670:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571721600 00002000:00100000:0.0:1788619932.579720:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c7430a80 x1875503571721600/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619932.579732:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c7430a80 with x1875503571721600 00000100:00100000:0.0:1788619932.579769:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 951 00000100:00100000:0.0:1788619932.579785:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7430a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2354:x1875503571721600:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619932.580002:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7430a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2354:x1875503571721600:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 223us (454us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619932.580011:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 951 00002000:00100000:0.0:1788619932.580018:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c7430a80 x1875503571721600/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619932.580026:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c7430a80 with x1875503571721600 00000100:00100000:1.0:1788619932.584217:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619932.584305:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571721728 00000100:00100000:3.0:1788619932.584314:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571721728 00000100:00100000:3.0:1788619932.584407:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1292 00000100:00100000:3.0:1788619932.584417:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64d0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571721728:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619932.584434:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619932.584452:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e4805681/0x1cd7fe5368736b35 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1db9c expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619932.584529:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63fb800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589260160:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619932.584615:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5eafd80 going to sleep for 11 seconds 00002000:00010000:3.0:1788619932.586506:0:6696:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619931 -> 1788619932 00000100:00100000:3.0:1788619932.586545:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63fb800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589260160:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619932.586578:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e4805681/0x1cd7fe5368736b35 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db9c expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.586603:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e4805681/0x1cd7fe5368736b35 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db9c expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.586622:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d6400, rc 0) 00000100:00100000:3.0:1788619932.586681:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64d0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571721728:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2264us (2472us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619932.586691:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1292 00000100:00100000:2.0:1788619932.591319:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619932.591685:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571721856 00000100:00100000:3.0:1788619932.591696:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571721856 00000100:00100000:3.0:1788619932.591760:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1561 00000100:00100000:3.0:1788619932.591770:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7430000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:14908:x1875503571721856:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619932.591802:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619932.591806:0:5846:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1788619932.591857:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368736ada lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db41 expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.591876:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619932.591882:0:5846:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788619932.591895:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7200/0x1cd7fe5368736b3c lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dba3 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.591936:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe5368736ae1 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db48 expref: 21 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.591947:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7200/0x1cd7fe5368736b3c lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dba3 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.591979:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe5368736ae1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db48 expref: 21 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.591993:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe5368736ae1 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db48 expref: 21 pid: 5845 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619932.592007:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c63fb800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589260288:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619932.592135:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5eafd80 going to sleep for 11 seconds 00000100:00100000:3.0:1788619932.597695:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c63fb800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589260288:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619932.597729:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7200/0x1cd7fe5368736b3c lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dba3 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.597749:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619932.597753:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619932.597758:0:5846:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7200/0x1cd7fe5368736b3c lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dba3 expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.597768:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d7200, rc 0) 00000100:00100000:3.0:1788619932.597940:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7430000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:14908:x1875503571721856:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6170us (6629us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619932.597948:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1561 00000100:00100000:2.0:1788619932.604244:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619932.605243:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571721984 00000100:00100000:3.0:1788619932.605254:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571721984 00010000:00010000:3.0:1788619932.605299:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe5368736ae1 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db48 expref: 22 pid: 5845 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619932.605690:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 517 00000100:00100000:1.0:1788619932.605753:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7430380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2354:x1875503571721984:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619932.605771:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619932.605810:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe5368736ae1 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db48 expref: 22 pid: 5845 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.605830:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe5368736ae1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db48 expref: 22 pid: 5845 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.605852:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6000/0x1cd7fe5368736ae1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db48 expref: 22 pid: 5845 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.605869:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619932.605876:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7200/0x1cd7fe5368736b3c lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dba3 expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.605889:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7200/0x1cd7fe5368736b3c lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dba3 expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.605900:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7200/0x1cd7fe5368736b3c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dba3 expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.605951:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7200/0x1cd7fe5368736b3c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dba3 expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619932.605964:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64d1500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589260416:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619932.606053:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc240 going to sleep for 11 seconds 00000100:00100000:1.0:1788619932.608971:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64d1500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589260416:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619932.609000:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619932.609058:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7430380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2354:x1875503571721984:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3358us (4823us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619932.609074:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 517 00000100:00100000:3.0:1788619932.620436:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619932.620595:0:13346:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571722112 00000100:00100000:2.0:1788619932.620605:0:13346:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571722112 00002000:00100000:2.0:1788619932.620656:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c63fb800 x1875503571722112/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619932.620668:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c63fb800 with x1875503571722112 00000100:00100000:2.0:1788619932.620707:0:13346:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 952 00000100:00100000:2.0:1788619932.620718:0:13346:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63fb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571722112:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619932.621027:0:13346:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63fb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571722112:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 310us (603us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619932.621035:0:13346:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 952 00002000:00100000:2.0:1788619932.621043:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c63fb800 x1875503571722112/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619932.621051:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c63fb800 with x1875503571722112 00000100:00100000:3.0:1788619932.627165:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619932.627460:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571722240 00000100:00100000:0.0:1788619932.627471:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571722240 00002000:00100000:0.0:1788619932.627532:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c63fa300 x1875503571722240/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619932.627543:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c63fa300 with x1875503571722240 00000100:00100000:0.0:1788619932.627583:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 953 00000100:00100000:0.0:1788619932.627592:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63fa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571722240:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619932.627871:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63fa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571722240:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 279us (716us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619932.627883:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 953 00002000:00100000:0.0:1788619932.627891:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c63fa300 x1875503571722240/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619932.627900:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c63fa300 with x1875503571722240 00000100:00100000:1.0:1788619932.631126:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619932.631224:0:13346:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571722368 00000100:00100000:2.0:1788619932.631234:0:13346:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571722368 00002000:00100000:2.0:1788619932.631280:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c64d1500 x1875503571722368/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619932.631291:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64d1500 with x1875503571722368 00000100:00100000:2.0:1788619932.631422:0:13346:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 954 00000100:00100000:2.0:1788619932.631433:0:13346:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64d1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571722368:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619932.631662:0:13346:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64d1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571722368:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 229us (543us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619932.631669:0:13346:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 954 00002000:00100000:2.0:1788619932.631676:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c64d1500 x1875503571722368/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619932.631683:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64d1500 with x1875503571722368 00000100:00100000:3.0:1788619932.634569:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619932.635016:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571722496 00000100:00100000:0.0:1788619932.635027:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571722496 00002000:00100000:0.0:1788619932.635075:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c63f9f80 x1875503571722496/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619932.635088:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c63f9f80 with x1875503571722496 00000100:00100000:0.0:1788619932.635124:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 955 00000100:00100000:0.0:1788619932.635133:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63f9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571722496:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619932.635392:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63f9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571722496:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 258us (831us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619932.635400:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 955 00002000:00100000:0.0:1788619932.635407:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c63f9f80 x1875503571722496/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619932.635415:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c63f9f80 with x1875503571722496 00000100:00100000:3.0:1788619932.639091:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619932.639195:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571722624 00000100:00100000:0.0:1788619932.639207:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571722624 00002000:00100000:0.0:1788619932.639259:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c63f9180 x1875503571722624/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619932.639271:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c63f9180 with x1875503571722624 00000100:00100000:0.0:1788619932.639309:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 956 00000100:00100000:0.0:1788619932.639319:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63f9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571722624:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619932.639581:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63f9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571722624:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 262us (498us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619932.639590:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 956 00002000:00100000:0.0:1788619932.639597:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c63f9180 x1875503571722624/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619932.639606:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c63f9180 with x1875503571722624 00000100:00100000:1.0:1788619932.642785:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619932.642886:0:13346:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571722752 00000100:00100000:2.0:1788619932.642900:0:13346:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571722752 00002000:00100000:2.0:1788619932.642955:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c64d3480 x1875503571722752/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619932.642967:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64d3480 with x1875503571722752 00000100:00100000:2.0:1788619932.643005:0:13346:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 957 00000100:00100000:2.0:1788619932.643015:0:13346:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64d3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571722752:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:2.0:1788619932.646264:0:13346:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1048576 -> 1048576 00000004:00010000:2.0:1788619932.646275:0:13346:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619932 -> 1788619932 00000004:00010000:2.0:1788619932.646278:0:13346:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619930 -> 1788619930 00000004:00010000:2.0:1788619932.646280:0:13346:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619932 -> 1788619932 00000004:00010000:2.0:1788619932.646282:0:13346:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1544 -> 1544 00000100:00100000:2.0:1788619932.646411:0:13346:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64d3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571722752:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 3392us (3635us total) trans 4294967840 rc 0/0 00000100:00100000:2.0:1788619932.646424:0:13346:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 957 00002000:00100000:2.0:1788619932.646436:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c64d3480 x1875503571722752/t4294967840(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/448 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619932.646446:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64d3480 with x1875503571722752 00000100:00100000:1.0:1788619932.649092:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619932.649170:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571722880 00000100:00100000:3.0:1788619932.649179:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571722880 00000100:00100000:3.0:1788619932.649238:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1562 00000100:00100000:3.0:1788619932.649249:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64d3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571722880:12345-192.168.201.5@tcp:16:fsx.0 00000001:00080000:0.0:1788619932.682671:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967837 is committed 00000001:00080000:0.0:1788619932.682861:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967840 is committed 00000001:00080000:0.0:1788619932.682871:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967838 is committed 00000001:00080000:0.0:1788619932.682877:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967839 is committed 00000100:00100000:3.0:1788619932.683020:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64d3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571722880:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 33767us (33932us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619932.683034:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1562 00000100:00100000:3.0:1788619932.686209:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619932.686313:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571723008 00000100:00100000:0.0:1788619932.686323:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571723008 00000100:00100000:0.0:1788619932.686533:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1293 00000100:00100000:0.0:1788619932.686545:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63f8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571723008:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619932.686564:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619932.686582:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000064837316/0x1cd7fe5368736b43 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dbaa expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619932.686636:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78cdcc0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589260544:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619932.686716:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5d2d0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619932.688785:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78cdcc0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589260544:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619932.688825:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000064837316/0x1cd7fe5368736b43 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dbaa expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619932.688850:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000064837316/0x1cd7fe5368736b43 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dbaa expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619932.688864:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c1495a00, rc 0) 00000100:00100000:0.0:1788619932.688920:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63f8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571723008:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2376us (2719us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619932.688928:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1293 00000100:00100000:3.0:1788619932.691317:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619932.691754:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571723136 00000100:00100000:2.0:1788619932.691764:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571723136 00000100:00100000:2.0:1788619932.691896:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1294 00000100:00100000:2.0:1788619932.691916:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571723136:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619932.691933:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619932.691950:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a7b141ab/0x1cd7fe5368736b4a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dbb1 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619932.692009:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c7430700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589260672:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619932.692100:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164f00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619932.694580:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c7430700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589260672:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619932.694617:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a7b141ab/0x1cd7fe5368736b4a lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dbb1 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.694640:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a7b141ab/0x1cd7fe5368736b4a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dbb1 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.694654:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed1200, rc 0) 00000100:00100000:2.0:1788619932.694711:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571723136:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2805us (3402us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619932.694721:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1294 00000100:00100000:1.0:1788619932.698073:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619932.698160:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571723264 00000100:00100000:2.0:1788619932.698169:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571723264 00000100:00100000:2.0:1788619932.698226:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1295 00000100:00100000:2.0:1788619932.698236:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64d0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571723264:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619932.698253:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619932.698272:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c5c78e09/0x1cd7fe5368736b51 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dbb8 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619932.698453:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c7430700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589260800:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619932.698544:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164f00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619932.701829:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c7430700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589260800:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619932.701870:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c5c78e09/0x1cd7fe5368736b51 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dbb8 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.701893:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c5c78e09/0x1cd7fe5368736b51 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dbb8 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.701905:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed1400, rc 0) 00000100:00100000:2.0:1788619932.701964:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64d0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571723264:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3730us (3900us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619932.701972:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1295 00000100:00100000:0.0:1788619932.704807:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619932.704941:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571723392 00000100:00100000:2.0:1788619932.704952:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571723392 00000100:00100000:2.0:1788619932.705011:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1296 00000100:00100000:2.0:1788619932.705022:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571723392:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619932.705040:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619932.705057:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a7b141ab/0x1cd7fe5368736b58 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dbbf expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619932.705117:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c7430700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589260928:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619932.705199:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164f00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619932.707970:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c7430700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589260928:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619932.708013:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a7b141ab/0x1cd7fe5368736b58 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dbbf expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.708038:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a7b141ab/0x1cd7fe5368736b58 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dbbf expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.708051:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed1200, rc 0) 00000100:00100000:2.0:1788619932.708108:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571723392:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3089us (3310us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619932.708116:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1296 00000100:00100000:3.0:1788619932.711553:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619932.711671:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571723520 00000100:00100000:1.0:1788619932.711682:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571723520 00000100:00100000:1.0:1788619932.711741:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1297 00000100:00100000:1.0:1788619932.711751:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571723520:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619932.711771:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619932.711802:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000485aa315/0x1cd7fe5368736b5f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dbc6 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619932.711866:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5770000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589261056:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619932.711962:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc300 going to sleep for 11 seconds 00000100:00100000:1.0:1788619932.715040:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5770000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589261056:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619932.715091:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000485aa315/0x1cd7fe5368736b5f lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dbc6 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.715120:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000485aa315/0x1cd7fe5368736b5f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dbc6 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.715136:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683cc00, rc 0) 00000100:00100000:1.0:1788619932.715198:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571723520:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3448us (3655us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619932.715211:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1297 00000100:00100000:3.0:1788619932.721435:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619932.721537:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571723648 00000100:00100000:0.0:1788619932.721547:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571723648 00002000:00100000:0.0:1788619932.721598:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c63f8a80 x1875503571723648/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619932.721609:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c63f8a80 with x1875503571723648 00000100:00100000:0.0:1788619932.721647:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 958 00000100:00100000:0.0:1788619932.721657:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63f8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571723648:12345-192.168.201.5@tcp:3:fsx.0 00000004:00010000:0.0:1788619932.721807:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1544 -> 1800 00000100:00100000:0.0:1788619932.721899:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63f8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571723648:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 243us (473us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619932.721906:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 958 00002000:00100000:0.0:1788619932.721913:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c63f8a80 x1875503571723648/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619932.721921:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c63f8a80 with x1875503571723648 00000100:00100000:3.0:1788619932.725191:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619932.725618:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571723776 00000100:00100000:1.0:1788619932.725629:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571723776 00002000:00100000:1.0:1788619932.725684:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69fad80 x1875503571723776/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619932.725694:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69fad80 with x1875503571723776 00000100:00100000:1.0:1788619932.725737:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 959 00000100:00100000:1.0:1788619932.725748:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571723776:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619932.725990:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571723776:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 243us (807us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619932.725997:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 959 00002000:00100000:1.0:1788619932.726006:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69fad80 x1875503571723776/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619932.726014:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69fad80 with x1875503571723776 00000100:00100000:1.0:1788619932.729991:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619932.730144:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571723904 00000100:00100000:0.0:1788619932.730154:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571723904 00002000:00100000:0.0:1788619932.730206:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5770000 x1875503571723904/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619932.730223:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5770000 with x1875503571723904 00000100:00100000:0.0:1788619932.730262:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 960 00000100:00100000:0.0:1788619932.730272:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5770000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571723904:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619932.730559:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5770000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571723904:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 287us (575us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619932.730567:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 960 00002000:00100000:0.0:1788619932.730574:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5770000 x1875503571723904/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619932.730583:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5770000 with x1875503571723904 00000100:00100000:3.0:1788619932.733719:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619932.734084:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571724032 00000100:00100000:1.0:1788619932.734095:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571724032 00002000:00100000:1.0:1788619932.734146:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69fa300 x1875503571724032/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619932.734159:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69fa300 with x1875503571724032 00000100:00100000:1.0:1788619932.734201:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 961 00000100:00100000:1.0:1788619932.734211:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571724032:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619932.734494:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571724032:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 283us (785us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619932.734504:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 961 00002000:00100000:1.0:1788619932.734524:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69fa300 x1875503571724032/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619932.734532:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69fa300 with x1875503571724032 00000100:00100000:1.0:1788619932.738448:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619932.738645:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571724160 00000100:00100000:0.0:1788619932.738655:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571724160 00002000:00100000:0.0:1788619932.738707:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5770a80 x1875503571724160/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619932.738718:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5770a80 with x1875503571724160 00000100:00100000:0.0:1788619932.738805:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 962 00000100:00100000:0.0:1788619932.738815:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5770a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571724160:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619932.739053:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5770a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571724160:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 238us (615us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619932.739061:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 962 00002000:00100000:0.0:1788619932.739067:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5770a80 x1875503571724160/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619932.739075:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5770a80 with x1875503571724160 00000100:00100000:0.0:1788619932.742875:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619932.743264:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571724288 00000100:00100000:1.0:1788619932.743275:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571724288 00002000:00100000:1.0:1788619932.743466:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78cdcc1c00 x1875503571724288/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619932.743477:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78cdcc1c00 with x1875503571724288 00000100:00100000:1.0:1788619932.743570:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 963 00000100:00100000:1.0:1788619932.743579:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571724288:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619932.743826:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571724288:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 247us (961us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619932.743834:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 963 00002000:00100000:1.0:1788619932.743842:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78cdcc1c00 x1875503571724288/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619932.743850:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78cdcc1c00 with x1875503571724288 00000100:00100000:3.0:1788619932.746915:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619932.747032:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571724416 00000100:00100000:0.0:1788619932.747041:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571724416 00002000:00100000:0.0:1788619932.747090:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69f9f80 x1875503571724416/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619932.747102:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69f9f80 with x1875503571724416 00000100:00100000:0.0:1788619932.747140:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 964 00000100:00100000:0.0:1788619932.747149:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69f9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571724416:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619932.747407:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69f9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571724416:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 258us (505us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619932.747416:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 964 00002000:00100000:0.0:1788619932.747423:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69f9f80 x1875503571724416/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619932.747431:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69f9f80 with x1875503571724416 00000100:00100000:3.0:1788619932.751094:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619932.751440:0:13346:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571724544 00000100:00100000:2.0:1788619932.751450:0:13346:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571724544 00002000:00100000:2.0:1788619932.751500:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69f8380 x1875503571724544/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619932.751523:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69f8380 with x1875503571724544 00000100:00100000:2.0:1788619932.751563:0:13346:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 965 00000100:00100000:2.0:1788619932.751573:0:13346:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69f8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571724544:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619932.751811:0:13346:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69f8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571724544:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 238us (726us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619932.751819:0:13346:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 965 00002000:00100000:2.0:1788619932.751826:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69f8380 x1875503571724544/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619932.751834:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69f8380 with x1875503571724544 00000100:00100000:3.0:1788619932.755789:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619932.755937:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571724672 00000100:00100000:2.0:1788619932.755948:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571724672 00002000:00100000:2.0:1788619932.755999:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69f9880 x1875503571724672/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619932.756011:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69f9880 with x1875503571724672 00000100:00100000:2.0:1788619932.756053:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 966 00000100:00100000:2.0:1788619932.756063:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69f9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571724672:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619932.756304:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69f9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571724672:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 242us (534us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619932.756312:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 966 00002000:00100000:2.0:1788619932.756319:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69f9880 x1875503571724672/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619932.757278:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69f9880 with x1875503571724672 00000100:00100000:3.0:1788619932.759477:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619932.759594:0:13346:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571724800 00000100:00100000:2.0:1788619932.759604:0:13346:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571724800 00002000:00100000:2.0:1788619932.759652:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69faa00 x1875503571724800/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619932.759663:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69faa00 with x1875503571724800 00000100:00100000:2.0:1788619932.759701:0:13346:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 967 00000100:00100000:2.0:1788619932.759711:0:13346:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69faa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571724800:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619932.759956:0:13346:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69faa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571724800:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 246us (487us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619932.759963:0:13346:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 967 00002000:00100000:2.0:1788619932.759970:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69faa00 x1875503571724800/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619932.759977:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69faa00 with x1875503571724800 00000100:00100000:3.0:1788619932.763773:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619932.763889:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571724928 00000100:00100000:1.0:1788619932.763899:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571724928 00002000:00100000:1.0:1788619932.763950:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69fbb80 x1875503571724928/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619932.763960:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69fbb80 with x1875503571724928 00000100:00100000:1.0:1788619932.763999:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 968 00000100:00100000:1.0:1788619932.764009:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571724928:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619932.764230:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571724928:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 222us (468us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619932.764238:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 968 00002000:00100000:1.0:1788619932.764245:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69fbb80 x1875503571724928/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619932.764253:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69fbb80 with x1875503571724928 00000100:00100000:3.0:1788619932.767606:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619932.767713:0:13346:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571725056 00000100:00100000:2.0:1788619932.767723:0:13346:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571725056 00002000:00100000:2.0:1788619932.767772:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69f8700 x1875503571725056/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619932.767790:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69f8700 with x1875503571725056 00000100:00100000:2.0:1788619932.767830:0:13346:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 969 00000100:00100000:2.0:1788619932.767840:0:13346:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69f8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571725056:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619932.768079:0:13346:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69f8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571725056:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 239us (480us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619932.768087:0:13346:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 969 00002000:00100000:2.0:1788619932.768094:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69f8700 x1875503571725056/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619932.768102:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69f8700 with x1875503571725056 00000100:00100000:3.0:1788619932.771693:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619932.771814:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571725184 00000100:00100000:1.0:1788619932.771826:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571725184 00002000:00100000:1.0:1788619932.771880:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69fa680 x1875503571725184/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619932.771890:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69fa680 with x1875503571725184 00000100:00100000:1.0:1788619932.771931:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 970 00000100:00100000:1.0:1788619932.771942:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571725184:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619932.772175:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571725184:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 234us (493us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619932.772183:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 970 00002000:00100000:1.0:1788619932.772190:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69fa680 x1875503571725184/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619932.772198:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69fa680 with x1875503571725184 00000100:00100000:3.0:1788619932.776020:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619932.776131:0:13346:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571725312 00000100:00100000:2.0:1788619932.776142:0:13346:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571725312 00002000:00100000:2.0:1788619932.776196:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69fb100 x1875503571725312/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619932.776211:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69fb100 with x1875503571725312 00000100:00100000:2.0:1788619932.776251:0:13346:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 971 00000100:00100000:2.0:1788619932.776261:0:13346:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571725312:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619932.776538:0:13346:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571725312:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 277us (526us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619932.776546:0:13346:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 971 00002000:00100000:2.0:1788619932.776553:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c69fb100 x1875503571725312/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619932.776561:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69fb100 with x1875503571725312 00000100:00100000:3.0:1788619932.780178:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619932.780289:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571725440 00000100:00100000:1.0:1788619932.780299:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571725440 00002000:00100000:1.0:1788619932.780400:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5025f80 x1875503571725440/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619932.780411:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5025f80 with x1875503571725440 00000100:00100000:1.0:1788619932.780444:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 972 00000100:00100000:1.0:1788619932.780455:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5025f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571725440:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619932.780940:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5025f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2352:x1875503571725440:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 483us (771us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619932.780949:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 972 00002000:00100000:1.0:1788619932.780957:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5025f80 x1875503571725440/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619932.780965:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5025f80 with x1875503571725440 00000100:00100000:3.0:1788619932.784073:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619932.784155:0:13346:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571725568 00000100:00100000:2.0:1788619932.784166:0:13346:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571725568 00002000:00100000:2.0:1788619932.784215:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5025c00 x1875503571725568/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619932.784225:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5025c00 with x1875503571725568 00000100:00100000:2.0:1788619932.784266:0:13346:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 973 00000100:00100000:2.0:1788619932.784276:0:13346:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5025c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571725568:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619932.785174:0:13346:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5025c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571725568:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 894us (1108us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619932.785185:0:13346:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 973 00002000:00100000:2.0:1788619932.785194:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5025c00 x1875503571725568/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619932.785202:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5025c00 with x1875503571725568 00000100:00100000:3.0:1788619932.788350:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619932.788467:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571725696 00000100:00100000:0.0:1788619932.788477:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571725696 00002000:00100000:0.0:1788619932.788539:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5025500 x1875503571725696/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619932.788551:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5025500 with x1875503571725696 00000100:00100000:0.0:1788619932.788595:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 974 00000100:00100000:0.0:1788619932.788605:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5025500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571725696:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:2.0:1788619932.792139:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1048576 -> 1048576 00000004:00010000:2.0:1788619932.792150:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619932 -> 1788619932 00000004:00010000:2.0:1788619932.792152:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619930 -> 1788619930 00000004:00010000:2.0:1788619932.792154:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619932 -> 1788619932 00000004:00010000:2.0:1788619932.792156:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1800 -> 1800 00000100:00100000:2.0:1788619932.792235:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5025500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2353:x1875503571725696:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 3625us (3913us total) trans 4294967841 rc 0/0 00000100:00100000:2.0:1788619932.792246:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 974 00002000:00100000:2.0:1788619932.792260:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5025500 x1875503571725696/t4294967841(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/448 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619932.792269:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5025500 with x1875503571725696 00000100:00100000:3.0:1788619932.795988:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619932.796069:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571725824 00000100:00100000:2.0:1788619932.796079:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571725824 00000100:00100000:2.0:1788619932.796141:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1563 00000100:00100000:2.0:1788619932.796152:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5026680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571725824:12345-192.168.201.5@tcp:16:fsx.0 00000001:00080000:0.0:1788619932.825035:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967841 is committed 00000100:00100000:2.0:1788619932.825205:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5026680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571725824:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 29049us (29222us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619932.825218:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1563 00000100:00100000:1.0:1788619932.827844:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619932.827939:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571725952 00000100:00100000:3.0:1788619932.827949:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571725952 00000100:00100000:3.0:1788619932.828057:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1564 00000100:00100000:3.0:1788619932.828067:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5773480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571725952:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619932.828088:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619932.828107:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6400/0x1cd7fe5368736b66 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dbcd expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.828145:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7200/0x1cd7fe5368736b3c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dba3 expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619932.828185:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5024000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589261184:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619932.828280:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeba80 going to sleep for 11 seconds 00000004:00010000:3.0:1788619932.830303:0:5846:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619930 -> 1788619932 00000100:00100000:3.0:1788619932.830421:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5024000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589261184:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619932.830445:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:3.0:1788619932.830461:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6400/0x1cd7fe5368736b66 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dbcd expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.830482:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619932.830485:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619932.830490:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6400/0x1cd7fe5368736b66 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dbcd expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.830504:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c64d6400, rc 0) 00000100:00100000:3.0:1788619932.830620:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5773480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571725952:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2554us (2846us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619932.830628:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1564 00000100:00100000:1.0:1788619932.833475:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619932.833662:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571726080 00000100:00100000:2.0:1788619932.833672:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571726080 00000100:00100000:2.0:1788619932.833767:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1565 00000100:00100000:2.0:1788619932.833787:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5771c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571726080:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619932.833808:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619932.833832:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1200/0x1cd7fe5368736b6d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dbd4 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.833872:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7200/0x1cd7fe5368736b3c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dba3 expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619932.833925:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c7430700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589261312:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619932.834005:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81640c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619932.835998:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c7430700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589261312:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619932.836023:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:2.0:1788619932.836039:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1200/0x1cd7fe5368736b6d lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dbd4 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.836059:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619932.836062:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619932.836068:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1200/0x1cd7fe5368736b6d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dbd4 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.836083:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed1200, rc 0) 00000100:00100000:2.0:1788619932.836188:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5771c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571726080:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2411us (2723us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619932.836197:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1565 00000100:00100000:3.0:1788619932.843270:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619932.843367:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571726208 00000100:00100000:1.0:1788619932.843377:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571726208 00002000:00100000:1.0:1788619932.843423:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5024000 x1875503571726208/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 520/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619932.843435:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5024000 with x1875503571726208 00000100:00100000:1.0:1788619932.843516:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 975 00000100:00100000:1.0:1788619932.843526:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5024000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571726208:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619932.856092:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5024000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571726208:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 12555us (12817us total) trans 0 rc 962560/962560 00000100:00100000:1.0:1788619932.856109:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 975 00002000:00100000:1.0:1788619932.856123:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5024000 x1875503571726208/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 520/440 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 962560/962560 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619932.856134:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5024000 with x1875503571726208 00000100:00100000:1.0:1788619932.859540:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619932.859676:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571726336 00000100:00100000:2.0:1788619932.859687:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571726336 00000100:00100000:2.0:1788619932.859743:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1566 00000100:00100000:2.0:1788619932.859753:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5772300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571726336:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619932.859786:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619932.859808:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1200/0x1cd7fe5368736b74 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dbdb expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.859847:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7200/0x1cd7fe5368736b3c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dba3 expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619932.859889:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c7430700 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589261440:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619932.860097:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81640c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619932.862625:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c7430700 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589261440:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619932.862651:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:2.0:1788619932.862671:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1200/0x1cd7fe5368736b74 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dbdb expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.862691:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619932.862694:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619932.862700:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1200/0x1cd7fe5368736b74 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dbdb expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.862714:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed1200, rc 0) 00000100:00100000:2.0:1788619932.863299:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5772300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571726336:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3021us (3243us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619932.863312:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1566 00000100:00100000:2.0:1788619932.865633:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619932.865707:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571726464 00000100:00100000:3.0:1788619932.865716:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571726464 00000100:00100000:3.0:1788619932.865770:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1567 00000100:00100000:3.0:1788619932.865881:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7430700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571726464:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619932.865913:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619932.865936:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6200/0x1cd7fe5368736b7b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dbe2 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.865974:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7200/0x1cd7fe5368736b3c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dba3 expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619932.866016:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5026a00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589261568:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619932.866109:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeba80 going to sleep for 11 seconds 00000100:00100000:3.0:1788619932.869831:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5026a00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589261568:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619932.869855:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:3.0:1788619932.869872:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6200/0x1cd7fe5368736b7b lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dbe2 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.869891:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619932.869894:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619932.869900:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6200/0x1cd7fe5368736b7b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dbe2 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.870070:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78ccca6200, rc 0) 00000100:00100000:3.0:1788619932.870168:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7430700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571726464:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4289us (4544us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619932.870179:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1567 00000100:00100000:1.0:1788619932.873715:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619932.873878:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571726592 00000100:00100000:2.0:1788619932.873888:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571726592 00000100:00100000:2.0:1788619932.874042:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1568 00000100:00100000:2.0:1788619932.874053:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5771880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571726592:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619932.874074:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619932.874093:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1200/0x1cd7fe5368736b82 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dbe9 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.874136:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7200/0x1cd7fe5368736b3c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dba3 expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619932.874168:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c7432300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589261696:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619932.874250:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81640c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619932.876425:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c7432300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589261696:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619932.876456:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:3.0:1788619932.876478:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1200/0x1cd7fe5368736b82 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dbe9 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.876501:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619932.876504:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619932.876523:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1200/0x1cd7fe5368736b82 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dbe9 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.876537:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed1200, rc 0) 00000100:00100000:3.0:1788619932.876596:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5771880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571726592:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2545us (2893us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619932.876604:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1568 00000100:00100000:1.0:1788619932.879842:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619932.880035:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571726720 00000100:00100000:2.0:1788619932.880046:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571726720 00000100:00100000:2.0:1788619932.880108:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1569 00000100:00100000:2.0:1788619932.880118:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5770700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571726720:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619932.880138:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619932.880158:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0a00/0x1cd7fe5368736b89 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dbf0 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.880195:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7200/0x1cd7fe5368736b3c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dba3 expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619932.880235:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c7433b80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589261824:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619932.880322:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164d80 going to sleep for 11 seconds 00000100:00100000:2.0:1788619932.884766:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c7433b80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589261824:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619932.884804:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:2.0:1788619932.884824:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0a00/0x1cd7fe5368736b89 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dbf0 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.884847:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619932.884850:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619932.884856:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0a00/0x1cd7fe5368736b89 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dbf0 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.884871:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed0a00, rc 0) 00000100:00100000:2.0:1788619932.885050:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5770700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571726720:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4932us (5216us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619932.885059:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1569 00000100:00100000:3.0:1788619932.888144:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619932.888725:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571726848 00000100:00100000:2.0:1788619932.888738:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571726848 00000100:00100000:2.0:1788619932.888849:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1570 00000100:00100000:2.0:1788619932.888859:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5026a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571726848:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619932.888879:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619932.888898:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1e00/0x1cd7fe5368736b90 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dbf7 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.888949:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7200/0x1cd7fe5368736b3c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dba3 expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619932.888989:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c7433b80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589261952:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619932.889073:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164d80 going to sleep for 11 seconds 00000100:00100000:2.0:1788619932.891942:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c7433b80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589261952:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619932.891973:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:2.0:1788619932.891990:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1e00/0x1cd7fe5368736b90 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dbf7 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.892011:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619932.892014:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619932.892021:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1e00/0x1cd7fe5368736b90 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dbf7 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.892038:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed1e00, rc 0) 00000100:00100000:2.0:1788619932.892104:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5026a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571726848:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3246us (3969us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619932.892113:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1570 00000100:00100000:1.0:1788619932.895105:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619932.895226:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571726976 00000100:00100000:3.0:1788619932.895236:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571726976 00000100:00100000:3.0:1788619932.895296:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1571 00000100:00100000:3.0:1788619932.895307:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5770380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571726976:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619932.895360:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619932.895381:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6200/0x1cd7fe5368736b97 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dbfe expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.895417:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7200/0x1cd7fe5368736b3c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dba3 expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619932.895459:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5027480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589262080:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619932.895549:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeba80 going to sleep for 11 seconds 00000100:00100000:1.0:1788619932.897422:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5027480 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589262080:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619932.897447:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:1.0:1788619932.897464:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6200/0x1cd7fe5368736b97 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dbfe expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.897485:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619932.897488:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619932.897494:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6200/0x1cd7fe5368736b97 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dbfe expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.897520:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78ccca6200, rc 0) 00000100:00100000:1.0:1788619932.897570:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5770380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571726976:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2264us (2475us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619932.897578:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1571 00000100:00100000:3.0:1788619932.900113:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619932.900260:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571727104 00000100:00100000:0.0:1788619932.900269:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571727104 00000100:00100000:0.0:1788619932.900373:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1572 00000100:00100000:0.0:1788619932.900384:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5026300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571727104:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619932.900408:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619932.900429:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1495000/0x1cd7fe5368736b9e lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dc05 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619932.900461:0:9417:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7200/0x1cd7fe5368736b3c lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dba3 expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619932.900472:0:9417:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1495000/0x1cd7fe5368736b9e lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc05 expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619932.900520:0:9417:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7200/0x1cd7fe5368736b3c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dba3 expref: 9 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619932.900534:0:9417:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7200/0x1cd7fe5368736b3c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dba3 expref: 9 pid: 5846 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619932.900547:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78cdcc3480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589262208:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619932.900621:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5d2d900 going to sleep for 11 seconds 00000100:00100000:0.0:1788619932.903036:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78cdcc3480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589262208:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619932.903071:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1495000/0x1cd7fe5368736b9e lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc05 expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619932.903088:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619932.903090:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619932.903094:0:9417:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1495000/0x1cd7fe5368736b9e lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc05 expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619932.903109:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c1495000, rc 0) 00000100:00100000:0.0:1788619932.903155:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5026300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571727104:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2773us (3051us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619932.903163:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1572 00000100:00100000:3.0:1788619932.908866:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619932.908980:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571727232 00000100:00100000:0.0:1788619932.908991:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571727232 00010000:00010000:0.0:1788619932.909036:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7200/0x1cd7fe5368736b3c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dba3 expref: 10 pid: 5846 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619932.909082:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 518 00000100:00100000:0.0:1788619932.909092:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5027480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571727232:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619932.909107:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619932.909135:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7200/0x1cd7fe5368736b3c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dba3 expref: 10 pid: 5846 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619932.909148:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7200/0x1cd7fe5368736b3c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dba3 expref: 10 pid: 5846 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619932.909170:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d7200/0x1cd7fe5368736b3c lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dba3 expref: 10 pid: 5846 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619932.909186:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619932.909193:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1495000/0x1cd7fe5368736b9e lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc05 expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619932.909210:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1495000/0x1cd7fe5368736b9e lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc05 expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619932.909219:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1495000/0x1cd7fe5368736b9e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc05 expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619932.909257:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1495000/0x1cd7fe5368736b9e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc05 expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619932.909271:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78cdcc3480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589262336:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619932.909708:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5d2d900 going to sleep for 11 seconds 00000100:00100000:3.0:1788619932.913374:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78cdcc3480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589262336:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619932.913404:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619932.913611:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5027480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571727232:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 4519us (4753us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619932.913622:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 518 00000100:00100000:0.0:1788619932.914283:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619932.914426:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571727360 00000100:00100000:2.0:1788619932.914435:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571727360 00002000:00100000:2.0:1788619932.914488:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78cdcc1500 x1875503571727360/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619932.914499:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78cdcc1500 with x1875503571727360 00000100:00100000:2.0:1788619932.914555:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 976 00000100:00100000:2.0:1788619932.914565:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571727360:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619932.914816:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571727360:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 252us (534us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619932.914824:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 976 00002000:00100000:2.0:1788619932.914830:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78cdcc1500 x1875503571727360/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619932.914839:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78cdcc1500 with x1875503571727360 00000100:00100000:1.0:1788619932.918653:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619932.918760:0:13346:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571727488 00000100:00100000:2.0:1788619932.918827:0:13346:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571727488 00002000:00100000:2.0:1788619932.918878:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5773800 x1875503571727488/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619932.918889:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5773800 with x1875503571727488 00000100:00100000:2.0:1788619932.918959:0:13346:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 977 00000100:00100000:2.0:1788619932.918969:0:13346:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5773800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571727488:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619932.919173:0:13346:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5773800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571727488:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 205us (531us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619932.919181:0:13346:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 977 00002000:00100000:2.0:1788619932.919188:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5773800 x1875503571727488/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619932.919195:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5773800 with x1875503571727488 00000100:00100000:3.0:1788619932.923103:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619932.923936:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571727616 00000100:00100000:1.0:1788619932.923949:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571727616 00000100:00100000:1.0:1788619932.924019:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 978 00000100:00100000:1.0:1788619932.924028:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571727616:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:1.0:1788619932.924320:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1048576 -> 1048576 00000004:00010000:1.0:1788619932.924354:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619932 -> 1788619932 00000004:00010000:1.0:1788619932.924357:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619932 -> 1788619932 00000004:00010000:1.0:1788619932.924360:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619932 -> 1788619932 00000004:00010000:1.0:1788619932.924362:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1800 -> 1800 00000100:00100000:1.0:1788619932.924431:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571727616:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 404us (1336us total) trans 4294967842 rc 0/0 00000100:00100000:1.0:1788619932.924438:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 978 00000100:00100000:3.0:1788619932.928087:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619932.928128:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619932.928205:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571727872 00000100:00100000:1.0:1788619932.928215:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571727872 00000100:00100000:1.0:1788619932.928274:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1298 00000100:00100000:1.0:1788619932.928283:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571727872:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619932.928301:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619932.928320:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe5368736ba5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dc13 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619932.928413:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5772d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589262464:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619932.928495:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc840 going to sleep for 11 seconds 00000100:00100000:1.0:1788619932.930075:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571727744 00000100:00100000:1.0:1788619932.930086:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571727744 00000100:00100000:1.0:1788619932.930150:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1573 00000100:00100000:1.0:1788619932.930159:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571727744:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619932.930177:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619932.930195:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368736bac lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dc0c expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.930232:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1495000/0x1cd7fe5368736b9e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc05 expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619932.930267:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5773b80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589262592:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619932.930359:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc240 going to sleep for 11 seconds 00000100:00100000:2.0:1788619932.932630:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5773b80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589262592:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619932.932656:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:2.0:1788619932.932676:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368736bac lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc0c expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.932697:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619932.932700:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619932.932705:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368736bac lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc0c expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.932720:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d000, rc 0) 00000100:00100000:2.0:1788619932.932791:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571727744:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2633us (4661us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619932.932800:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1573 00000100:00100000:1.0:1788619932.934211:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5772d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589262464:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619932.934245:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe5368736ba5 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc13 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.934268:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe5368736ba5 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc13 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.934281:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683dc00, rc 0) 00000100:00100000:1.0:1788619932.934377:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571727872:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6095us (6299us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619932.934386:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1298 00000100:00100000:3.0:1788619932.937635:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619932.937740:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571728000 00000100:00100000:2.0:1788619932.937751:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571728000 00000100:00100000:2.0:1788619932.937821:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1574 00000100:00100000:2.0:1788619932.937832:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571728000:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619932.937852:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619932.937874:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0a00/0x1cd7fe5368736bb3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dc1a expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.937909:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1495000/0x1cd7fe5368736b9e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc05 expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619932.937948:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c7433b80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589262720:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619932.938032:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164480 going to sleep for 11 seconds 00000100:00100000:2.0:1788619932.942120:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619932.942217:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571728128 00000100:00100000:1.0:1788619932.942228:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571728128 00000100:00100000:1.0:1788619932.942281:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1299 00000100:00100000:1.0:1788619932.942291:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7433100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571728128:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619932.942307:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619932.942359:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe5368736bba lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dc21 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619932.942423:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5772d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589262848:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619932.942521:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc840 going to sleep for 11 seconds 00000100:00100000:2.0:1788619932.946149:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c7433b80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589262720:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619932.946178:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:2.0:1788619932.946198:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0a00/0x1cd7fe5368736bb3 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc1a expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.946219:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619932.946222:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619932.946228:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0a00/0x1cd7fe5368736bb3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc1a expref: 10 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619932.946244:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed0a00, rc 0) 00000100:00100000:2.0:1788619932.946304:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571728000:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8475us (8679us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619932.946313:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1574 00000100:00100000:1.0:1788619932.948212:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5772d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589262848:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619932.948249:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe5368736bba lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc21 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.948280:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ec6a1654/0x1cd7fe5368736bba lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc21 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.948295:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683dc00, rc 0) 00000100:00100000:1.0:1788619932.948399:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7433100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571728128:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6109us (6289us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619932.948408:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1299 00000100:00100000:3.0:1788619932.952396:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619932.952529:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571728256 00000100:00100000:0.0:1788619932.952540:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571728256 00000100:00100000:0.0:1788619932.952602:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1575 00000100:00100000:0.0:1788619932.952612:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79008ae680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571728256:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619932.952632:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619932.952655:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1494200/0x1cd7fe5368736bc1 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dc28 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619932.952685:0:9417:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1495000/0x1cd7fe5368736b9e lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc05 expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619932.952695:0:9417:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1494200/0x1cd7fe5368736bc1 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc28 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619932.952735:0:9417:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1495000/0x1cd7fe5368736b9e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc05 expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619932.952748:0:9417:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1495000/0x1cd7fe5368736b9e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc05 expref: 14 pid: 9417 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619932.952762:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78cdcc3100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589262976:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619932.952858:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5d2d540 going to sleep for 11 seconds 00000100:00100000:0.0:1788619932.955901:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78cdcc3100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589262976:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619932.955952:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1494200/0x1cd7fe5368736bc1 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc28 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619932.955971:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619932.955974:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619932.955979:0:9417:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1494200/0x1cd7fe5368736bc1 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc28 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619932.955991:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c1494200, rc 0) 00000100:00100000:0.0:1788619932.956077:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79008ae680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:14908:x1875503571728256:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3465us (3689us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619932.956085:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1575 00000100:00100000:3.0:1788619932.959120:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619932.959244:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571728384 00000100:00100000:1.0:1788619932.959254:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571728384 00002000:00100000:1.0:1788619932.959298:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79008ad880 x1875503571728384/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:28/0 lens 16872/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619932.959309:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79008ad880 with x1875503571728384 00010000:00010000:1.0:1788619932.959317:0:5859:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1495000/0x1cd7fe5368736b9e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc05 expref: 15 pid: 9417 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.959376:0:5859:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1495000/0x1cd7fe5368736b9e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc05 expref: 15 pid: 9417 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788619932.959385:0:5859:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e79008ad880 00000100:00100000:1.0:1788619932.959410:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 41 00000100:00100000:1.0:1788619932.959419:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79008ad880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571728384:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:1.0:1788619932.959757:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1048576 -> 1048576 00000004:00010000:1.0:1788619932.959761:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619932 -> 1788619932 00000004:00010000:1.0:1788619932.959764:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619932 -> 1788619932 00000004:00010000:1.0:1788619932.959767:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619932 -> 1788619932 00000004:00010000:1.0:1788619932.959770:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1800 -> 1800 00000100:00100000:1.0:1788619932.959825:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79008ad880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2355:x1875503571728384:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 407us (714us total) trans 4294967843 rc 0/0 00000100:00100000:1.0:1788619932.959833:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 41 00002000:00100000:1.0:1788619932.959840:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79008ad880 x1875503571728384/t4294967843(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:28/0 lens 16872/448 e 0 to 0 dl 1788619943 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619932.959849:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79008ad880 with x1875503571728384 00010000:00010000:1.0:1788619932.959854:0:5859:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1495000/0x1cd7fe5368736b9e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc05 expref: 17 pid: 9417 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.959865:0:5859:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1495000/0x1cd7fe5368736b9e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc05 expref: 17 pid: 9417 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788619932.959872:0:5859:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e79008ad880 00000100:00100000:3.0:1788619932.963591:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619932.963797:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571728512 00000100:00100000:1.0:1788619932.963808:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571728512 00010000:00010000:1.0:1788619932.963854:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1495000/0x1cd7fe5368736b9e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc05 expref: 17 pid: 9417 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619932.963898:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 519 00000100:00100000:1.0:1788619932.964089:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79008aed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2353:x1875503571728512:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619932.964106:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619932.964132:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1495000/0x1cd7fe5368736b9e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc05 expref: 17 pid: 9417 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.964146:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1495000/0x1cd7fe5368736b9e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc05 expref: 17 pid: 9417 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.964169:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1495000/0x1cd7fe5368736b9e lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc05 expref: 17 pid: 9417 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.964489:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619932.964496:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1494200/0x1cd7fe5368736bc1 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc28 expref: 9 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.964518:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1494200/0x1cd7fe5368736bc1 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc28 expref: 9 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.964528:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1494200/0x1cd7fe5368736bc1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc28 expref: 9 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.964570:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1494200/0x1cd7fe5368736bc1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc28 expref: 9 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619932.964583:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5772d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589263104:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619932.964799:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc840 going to sleep for 11 seconds 00000100:00100000:0.0:1788619932.970028:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619932.970194:0:13346:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571728640 00000100:00100000:2.0:1788619932.970204:0:13346:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571728640 00002000:00100000:2.0:1788619932.970252:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78cdcc3100 x1875503571728640/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619932.970263:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78cdcc3100 with x1875503571728640 00000100:00100000:2.0:1788619932.970300:0:13346:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 979 00000100:00100000:2.0:1788619932.970309:0:13346:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571728640:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619932.970574:0:13346:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571728640:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 265us (557us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619932.970582:0:13346:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 979 00002000:00100000:2.0:1788619932.970589:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78cdcc3100 x1875503571728640/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619932.970597:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78cdcc3100 with x1875503571728640 00000100:00100000:3.0:1788619932.973733:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619932.973956:0:13346:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571728768 00000100:00100000:2.0:1788619932.973966:0:13346:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571728768 00002000:00100000:2.0:1788619932.974008:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79008ae300 x1875503571728768/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/0 e 0 to 0 dl 1788619943 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619932.974018:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79008ae300 with x1875503571728768 00000100:00100000:2.0:1788619932.974039:0:13346:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 980 00000100:00100000:2.0:1788619932.974047:0:13346:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79008ae300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571728768:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619932.974533:0:13346:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79008ae300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571728768:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 484us (806us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619932.974541:0:13346:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 980 00002000:00100000:2.0:1788619932.974548:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79008ae300 x1875503571728768/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:28/0 lens 488/4536 e 0 to 0 dl 1788619943 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619932.974556:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79008ae300 with x1875503571728768 00000100:00100000:2.0:1788619932.978596:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619932.979009:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571728896 00000100:00100000:3.0:1788619932.979020:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571728896 00000100:00100000:3.0:1788619932.979083:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1300 00000100:00100000:3.0:1788619932.979093:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7433b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571728896:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619932.979111:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619932.979127:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7278240/0x1cd7fe5368736bc8 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dc2f expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619932.979199:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79008aea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589263232:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619932.979283:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebc00 going to sleep for 11 seconds 00000100:00100000:1.0:1788619932.980302:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5772d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589263104:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619932.980323:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619932.980392:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79008aed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2353:x1875503571728512:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 16307us (16809us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619932.980403:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 519 00000100:00100000:3.0:1788619932.983439:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79008aea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589263232:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619932.983480:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d7278240/0x1cd7fe5368736bc8 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc2f expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.983518:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7278240/0x1cd7fe5368736bc8 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc2f expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.983532:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78ccca7400, rc 0) 00000100:00100000:3.0:1788619932.983596:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7433b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571728896:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4504us (5009us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619932.983604:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1300 00000100:00100000:0.0:1788619932.986970:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619932.987048:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571729024 00000100:00100000:3.0:1788619932.987059:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571729024 00000100:00100000:3.0:1788619932.987124:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1301 00000100:00100000:3.0:1788619932.987135:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571729024:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619932.987154:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619932.987172:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006d21ef6a/0x1cd7fe5368736bcf lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dc36 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619932.987248:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79008aea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589263360:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619932.987373:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebc00 going to sleep for 11 seconds 00000100:00100000:3.0:1788619932.990629:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79008aea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589263360:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619932.990667:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000006d21ef6a/0x1cd7fe5368736bcf lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc36 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.990695:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006d21ef6a/0x1cd7fe5368736bcf lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc36 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619932.990707:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78ccca6400, rc 0) 00000100:00100000:3.0:1788619932.990770:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571729024:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3635us (3810us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619932.990787:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1301 00000100:00100000:2.0:1788619932.997613:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619932.997750:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571729152 00000100:00100000:1.0:1788619932.997761:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571729152 00000100:00100000:1.0:1788619932.997832:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1576 00000100:00100000:1.0:1788619932.997843:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7433800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:14908:x1875503571729152:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619932.997867:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619932.997887:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368736bd6 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dc3d expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.997934:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1494200/0x1cd7fe5368736bc1 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc28 expref: 9 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.997945:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368736bd6 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc3d expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.997978:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1494200/0x1cd7fe5368736bc1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc28 expref: 9 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619932.997991:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1494200/0x1cd7fe5368736bc1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc28 expref: 9 pid: 9417 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619932.998004:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5772d80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589263488:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619932.998081:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc600 going to sleep for 11 seconds 00000100:00100000:1.0:1788619933.003611:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5772d80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589263488:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619933.003653:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368736bd6 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc3d expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.003674:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619933.003677:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619933.003683:0:5845:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368736bd6 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc3d expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.003694:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683dc00, rc 0) 00000100:00100000:1.0:1788619933.003782:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7433800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+17:14908:x1875503571729152:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5940us (6179us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619933.003792:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1576 00000100:00100000:2.0:1788619933.005628:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619933.005714:0:13346:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571729280 00000100:00100000:3.0:1788619933.005723:0:13346:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571729280 00002000:00100000:3.0:1788619933.005782:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c7431880 x1875503571729280/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619933.005794:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c7431880 with x1875503571729280 00010000:00010000:3.0:1788619933.005802:0:13346:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1494200/0x1cd7fe5368736bc1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc28 expref: 10 pid: 9417 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.005817:0:13346:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1494200/0x1cd7fe5368736bc1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc28 expref: 10 pid: 9417 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788619933.005825:0:13346:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e78c7431880 00000100:00100000:3.0:1788619933.005859:0:13346:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 42 00000100:00100000:3.0:1788619933.005869:0:13346:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7431880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571729280:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:0.0:1788619933.011091:0:13346:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1048576 -> 1048576 00000004:00010000:0.0:1788619933.011100:0:13346:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619932 -> 1788619932 00000004:00010000:0.0:1788619933.011103:0:13346:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619932 -> 1788619932 00000004:00010000:0.0:1788619933.011106:0:13346:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619932 -> 1788619932 00000004:00010000:0.0:1788619933.011108:0:13346:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1800 -> 1800 00000100:00100000:0.0:1788619933.011196:0:13346:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7431880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571729280:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 5314us (5562us total) trans 4294967844 rc 0/0 00000100:00100000:0.0:1788619933.011208:0:13346:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 42 00002000:00100000:0.0:1788619933.011220:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c7431880 x1875503571729280/t4294967844(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:29/0 lens 488/448 e 0 to 0 dl 1788619944 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619933.011230:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c7431880 with x1875503571729280 00010000:00010000:0.0:1788619933.011240:0:13346:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1494200/0x1cd7fe5368736bc1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc28 expref: 12 pid: 9417 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.011258:0:13346:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1494200/0x1cd7fe5368736bc1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc28 expref: 12 pid: 9417 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788619933.011266:0:13346:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e78c7431880 00000100:00100000:2.0:1788619933.016129:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619933.016299:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571729408 00000100:00100000:1.0:1788619933.016309:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571729408 00010000:00010000:1.0:1788619933.016439:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1494200/0x1cd7fe5368736bc1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc28 expref: 12 pid: 9417 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619933.016547:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 520 00000100:00100000:1.0:1788619933.016559:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7431c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571729408:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619933.016575:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619933.016601:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1494200/0x1cd7fe5368736bc1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc28 expref: 12 pid: 9417 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.016615:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1494200/0x1cd7fe5368736bc1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc28 expref: 12 pid: 9417 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.016636:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78c1494200/0x1cd7fe5368736bc1 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc28 expref: 12 pid: 9417 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.016653:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619933.016659:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368736bd6 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc3d expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.016679:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368736bd6 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc3d expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.016689:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368736bd6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc3d expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.016728:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368736bd6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc3d expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619933.016742:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5772d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589263616:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619933.017047:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc600 going to sleep for 11 seconds 00000100:00100000:1.0:1788619933.020743:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619933.021075:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571729536 00000100:00100000:2.0:1788619933.021086:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571729536 00000100:00100000:2.0:1788619933.021162:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 981 00000100:00100000:2.0:1788619933.021174:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5772680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2354:x1875503571729536:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:2.0:1788619933.021552:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1048576 -> 1048576 00000004:00010000:2.0:1788619933.021557:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619932 -> 1788619932 00000004:00010000:2.0:1788619933.021559:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619932 -> 1788619932 00000004:00010000:2.0:1788619933.021561:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619932 -> 1788619932 00000004:00010000:2.0:1788619933.021563:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1800 -> 1800 00000100:00100000:2.0:1788619933.021625:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5772680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2354:x1875503571729536:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 453us (890us total) trans 4294967845 rc 0/0 00000100:00100000:2.0:1788619933.021632:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 981 00000100:00100000:0.0:1788619933.022037:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5772d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589263616:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619933.022058:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619933.022082:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7431c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571729408:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 5527us (5964us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619933.022090:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 520 00000100:00100000:2.0:1788619933.026687:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619933.026852:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571729664 00000100:00100000:0.0:1788619933.026862:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571729664 00000100:00100000:0.0:1788619933.027070:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1302 00000100:00100000:0.0:1788619933.027081:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7432300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:14908:x1875503571729664:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619933.027101:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619933.027119:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006aec32ce/0x1cd7fe5368736bdd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dc44 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.027149:0:7654:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000001fc792ba/0x1cd7fe5368736b12 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1052672->1118207) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db79 expref: 8 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.027158:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(4 11) 00010000:00010000:0.0:1788619933.027162:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(5 11) 00010000:00010000:0.0:1788619933.027167:0:7654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006aec32ce/0x1cd7fe5368736bdd lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1269760->1290239] (req 1269760->1290239) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc44 expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.027290:0:7654:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000001fc792ba/0x1cd7fe5368736b12 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1052672->1118207) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db79 expref: 8 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.027305:0:7654:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000001fc792ba/0x1cd7fe5368736b12 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1052672->1118207) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db79 expref: 8 pid: 6698 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619933.027318:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78cdcc1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589263744:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619933.027610:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5d2d540 going to sleep for 11 seconds 00000100:00100000:2.0:1788619933.031673:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619933.031757:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571729792 00000100:00100000:3.0:1788619933.031769:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571729792 00010000:00010000:3.0:1788619933.031816:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000001fc792ba/0x1cd7fe5368736b12 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1052672->1118207) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db79 expref: 9 pid: 6698 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619933.031860:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 521 00000100:00100000:3.0:1788619933.031870:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7433480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2355:x1875503571729792:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619933.031885:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619933.031922:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000001fc792ba/0x1cd7fe5368736b12 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1052672->1118207) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db79 expref: 9 pid: 6698 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.031934:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000001fc792ba/0x1cd7fe5368736b12 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1052672->1118207) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db79 expref: 9 pid: 6698 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.031959:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(5 11) 00010000:00010000:3.0:1788619933.031963:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(5 11) 00010000:00010000:3.0:1788619933.031971:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000006aec32ce/0x1cd7fe5368736bdd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1269760->1290239) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc44 expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.031980:0:13330:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006aec32ce/0x1cd7fe5368736bdd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1269760->1290239) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc44 expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.032014:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000006aec32ce/0x1cd7fe5368736bdd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1269760->1290239) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc44 expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619933.032027:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79008aea00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589263872:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619933.032104:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebc00 going to sleep for 11 seconds 00010000:00010000:0.0:1788619933.034720:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001fc792ba/0x1cd7fe5368736b12 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1052672->1118207) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1db79 expref: 9 pid: 6698 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619933.034745:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78cdcc1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589263744:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619933.034786:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000006aec32ce/0x1cd7fe5368736bdd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1269760->1290239) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc44 expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.034806:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78c1494200, rc 0) 00000100:00100000:0.0:1788619933.034867:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7432300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:14908:x1875503571729664:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7787us (8190us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619933.034875:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1302 00000100:00100000:2.0:1788619933.037055:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79008aea00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589263872:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619933.037080:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619933.037148:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7433480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571729792:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 5279us (5484us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619933.037161:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 521 00000100:00100000:1.0:1788619933.043192:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619933.043377:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571729920 00000100:00100000:2.0:1788619933.043387:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571729920 00002000:00100000:2.0:1788619933.043440:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1413119) req@ffff8e78c5771500 x1875503571729920/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619933.043451:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5771500 with x1875503571729920 ext(1048576->1413119) 00002000:00010000:2.0:1788619933.043457:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5771500 00000100:00100000:2.0:1788619933.043492:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 778 00000100:00100000:2.0:1788619933.043517:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5771500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571729920:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619933.050691:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5771500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571729920:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 7178us (7509us total) trans 0 rc 364544/364544 00000100:00100000:2.0:1788619933.050704:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 778 00002000:00100000:2.0:1788619933.050717:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1413119) req@ffff8e78c5771500 x1875503571729920/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:29/0 lens 488/440 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 364544/364544 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619933.050728:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c5771500 with x1875503571729920 ext(1048576->1413119) 00002000:00010000:2.0:1788619933.050734:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c5771500 00000100:00100000:3.0:1788619933.054446:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619933.054610:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571730048 00000100:00100000:1.0:1788619933.054622:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571730048 00000100:00100000:1.0:1788619933.054690:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1303 00000100:00100000:1.0:1788619933.054699:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79008ac000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571730048:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619933.054719:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619933.054738:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe5368736be4 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dc4b expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619933.054869:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5770e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589264000:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619933.055007:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc840 going to sleep for 11 seconds 00000100:00100000:1.0:1788619933.058009:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5770e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589264000:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619933.058047:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe5368736be4 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc4b expref: 9 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.058073:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe5368736be4 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc4b expref: 9 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.058085:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683ca00, rc 0) 00000100:00100000:1.0:1788619933.058142:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79008ac000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571730048:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3444us (3706us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619933.058150:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1303 00000100:00100000:2.0:1788619933.061642:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619933.061749:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571730176 00000100:00100000:3.0:1788619933.061759:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571730176 00000100:00100000:3.0:1788619933.061825:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1304 00000100:00100000:3.0:1788619933.061835:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64a9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571730176:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619933.061854:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619933.061872:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006d21ef6a/0x1cd7fe5368736beb lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dc52 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619933.061926:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79008af800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589264128:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619933.062018:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb840 going to sleep for 11 seconds 00000100:00100000:2.0:1788619933.065053:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79008af800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589264128:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619933.065093:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000006d21ef6a/0x1cd7fe5368736beb lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc52 expref: 9 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.065118:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006d21ef6a/0x1cd7fe5368736beb lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc52 expref: 9 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.065132:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78ccca6400, rc 0) 00000100:00100000:2.0:1788619933.065201:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64a9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571730176:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3367us (3566us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619933.065209:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1304 00000100:00100000:1.0:1788619933.068479:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619933.068892:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571730304 00000100:00100000:2.0:1788619933.068909:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571730304 00000100:00100000:0.0:1788619933.069046:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1577 00000100:00100000:0.0:1788619933.069056:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5770e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571730304:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619933.069081:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619933.069100:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe5368736bf2 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dc59 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.069129:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368736bd6 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc3d expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.069139:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe5368736bf2 lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc59 expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.069173:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368736bd6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc3d expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.069309:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368736bd6 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc3d expref: 17 pid: 5845 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619933.069354:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78cdcc1880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589264256:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619933.069439:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5d2dc00 going to sleep for 11 seconds 00000100:00100000:0.0:1788619933.072094:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78cdcc1880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589264256:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619933.072127:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe5368736bf2 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc59 expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.072144:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619933.072147:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619933.072152:0:5846:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe5368736bf2 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc59 expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.072162:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa07a00, rc 0) 00000100:00100000:0.0:1788619933.072291:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5770e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:14908:x1875503571730304:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3236us (3821us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619933.072300:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1577 00000100:00100000:1.0:1788619933.074687:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619933.074781:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571730432 00000100:00100000:3.0:1788619933.074792:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571730432 00010000:00010000:3.0:1788619933.074835:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368736bd6 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc3d expref: 18 pid: 5845 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619933.074873:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 522 00000100:00100000:3.0:1788619933.074884:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcae050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2354:x1875503571730432:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619933.074898:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619933.074942:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368736bd6 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc3d expref: 18 pid: 5845 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.074956:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368736bd6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc3d expref: 18 pid: 5845 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.074978:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368736bd6 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc3d expref: 18 pid: 5845 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.074998:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619933.075005:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe5368736bf2 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc59 expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.075017:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe5368736bf2 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc59 expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.075027:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe5368736bf2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc59 expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.075065:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe5368736bf2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc59 expref: 11 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619933.075079:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79008ad500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589264384:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619933.075461:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb540 going to sleep for 11 seconds 00000100:00100000:1.0:1788619933.084529:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619933.084604:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571730560 00000100:00100000:0.0:1788619933.084615:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571730560 00002000:00100000:0.0:1788619933.084666:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5771180 x1875503571730560/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619933.084677:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5771180 with x1875503571730560 00000100:00100000:0.0:1788619933.084717:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 982 00000100:00100000:0.0:1788619933.084727:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5771180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571730560:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619933.086538:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79008ad500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589264384:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619933.086563:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619933.086604:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcae050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2354:x1875503571730432:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 11722us (11925us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619933.086614:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 522 00000100:00100000:2.0:1788619933.115416:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5771180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571730560:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 30679us (30885us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1788619933.115435:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 982 00002000:00100000:2.0:1788619933.115448:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5771180 x1875503571730560/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:29/0 lens 488/440 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619933.115458:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5771180 with x1875503571730560 00000100:00100000:1.0:1788619933.120289:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619933.120569:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571730816 00000100:00100000:0.0:1788619933.120579:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571730816 00000100:00100000:0.0:1788619933.120646:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1578 00000100:00100000:0.0:1788619933.120657:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5773100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+12:14908:x1875503571730816:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:0.0:1788619933.120709:0:9417:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619933.120735:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c64d6e00/0x1cd7fe5368736a4e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dabc expref: 12 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.120756:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619933.120761:0:9417:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788619933.120977:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe5368736bf9 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.120992:0:9417:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368736a78 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dadf expref: 16 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.121000:0:9417:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe5368736bf9 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.121032:0:9417:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368736a78 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dadf expref: 16 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.121046:0:9417:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368736a78 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dadf expref: 16 pid: 11397 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619933.121059:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78cdcc1880 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589264512:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619933.121146:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5d2dc00 going to sleep for 11 seconds 00000100:00100000:1.0:1788619933.123172:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78cdcc1880 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589264512:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619933.123211:0:9417:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe5368736bf9 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619933.124893:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619933.124981:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571730944 00000100:00100000:0.0:1788619933.124989:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571730944 00010000:00010000:0.0:1788619933.125026:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368736a78 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dadf expref: 17 pid: 11397 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619933.125069:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 523 00000100:00100000:0.0:1788619933.125079:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5772d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2355:x1875503571730944:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619933.125094:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619933.125101:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368736a78 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dadf expref: 17 pid: 11397 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.125114:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368736a78 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dadf expref: 17 pid: 11397 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.125136:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368736a78 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dadf expref: 17 pid: 11397 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.125152:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619933.125158:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe5368736bf9 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.125168:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe5368736bf9 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.125176:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe5368736bf9 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.125218:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619933.125268:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5772d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571730944:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 190us (383us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619933.125276:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 523 00010000:00010000:1.0:1788619933.125447:0:9417:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe5368736bf9 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.125456:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe5368736bf9 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788619933.125591:0:9417:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967846 00000004:00010000:1.0:1788619933.125641:0:9417:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1048576 -> 1048576 00000004:00010000:1.0:1788619933.125644:0:9417:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619932 -> 1788619933 00000004:00010000:1.0:1788619933.125647:0:9417:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619932 -> 1788619932 00000004:00010000:1.0:1788619933.125650:0:9417:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619932 -> 1788619933 00000004:00010000:1.0:1788619933.125653:0:9417:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1800 -> 1800 00000004:00010000:1.0:1788619933.125659:0:9417:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000001765a52c reply state 000000008031994a transno 4294967847 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe5368736bf9 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619933.125667:0:9417:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788619933.125734:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5773100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+14:14908:x1875503571730816:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 5079us (5457us total) trans 4294967847 rc 0/0 00000100:00100000:1.0:1788619933.125742:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1578 00000020:00080000:1.0:1788619933.125745:0:9417:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:2.0:1788619933.127869:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe5368736bf9 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.127882:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe5368736bf9 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.127889:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe5368736bf9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.127894:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78e6f65e00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:2.0:1788619933.127898:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe5368736bf9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.127912:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe5368736bf9 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.127928:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619933.127931:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788619933.127935:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe5368736bf9 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.127940:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe5368736bf9 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619933.132095:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619933.132188:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571731072 00000100:00100000:2.0:1788619933.132198:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571731072 00000100:00100000:2.0:1788619933.132260:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1579 00000100:00100000:2.0:1788619933.132271:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5773b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+13:14908:x1875503571731072:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619933.132295:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619933.132298:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1788619933.132384:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe5368736bf2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc59 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.132402:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619933.132407:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788619933.132421:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0a00/0x1cd7fe5368736c00 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dc60 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.132446:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0a00/0x1cd7fe5368736c00 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc60 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.132459:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0a00/0x1cd7fe5368736c00 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc60 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.132469:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619933.132472:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619933.132474:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed0a00, rc 0) 00000100:00100000:2.0:1788619933.132550:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5773b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+13:14908:x1875503571731072:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 281us (465us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619933.132557:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1579 00000100:00100000:0.0:1788619933.136656:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619933.136819:0:13347:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571731200 00000100:00100000:2.0:1788619933.136832:0:13347:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571731200 00002000:00010000:2.0:1788619933.136877:0:13347:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1900123->18446744073709551615) 00002000:00100000:2.0:1788619933.136882:0:13347:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78cdcc0000 with x1875503571731200 ext(1900123->18446744073709551615) 00002000:00010000:2.0:1788619933.136896:0:13347:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78cdcc0000. 00000100:00100000:2.0:1788619933.136928:0:13347:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 779 00000100:00100000:2.0:1788619933.136938:0:13347:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571731200:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:2.0:1788619933.137153:0:13347:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1410173 -> 1900123 00002000:00010000:2.0:1788619933.137157:0:13347:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619932 -> 1788619933 00002000:00010000:2.0:1788619933.137160:0:13347:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619932 -> 1788619931 00002000:00010000:2.0:1788619933.137162:0:13347:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619932 -> 1788619933 00002000:00010000:2.0:1788619933.137165:0:13347:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 391 -> 391 00000100:00100000:2.0:1788619933.137547:0:13347:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571731200:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 609us (900us total) trans 4294967521 rc 0/0 00000100:00100000:2.0:1788619933.137556:0:13347:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 779 00002000:00010000:2.0:1788619933.137560:0:13347:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1900123->18446744073709551615) 00002000:00100000:2.0:1788619933.137563:0:13347:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78cdcc0000 with x1875503571731200 ext(1900123->18446744073709551615) 00002000:00010000:2.0:1788619933.137569:0:13347:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78cdcc0000. 00000100:00100000:3.0:1788619933.140856:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619933.140984:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571731328 00000100:00100000:0.0:1788619933.140994:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571731328 00000100:00100000:0.0:1788619933.141056:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1580 00000100:00100000:0.0:1788619933.141065:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79008aed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:14908:x1875503571731328:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619933.141087:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619933.141107:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64200/0x1cd7fe5368736c07 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dc67 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.141201:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65200/0x1cd7fe5368736c0e lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.141215:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65200/0x1cd7fe5368736c0e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.141227:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65200/0x1cd7fe5368736c0e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619933.141252:0:5846:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1900123 00000004:00200000:0.0:1788619933.141256:0:5846:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619933.141272:0:5846:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65200/0x1cd7fe5368736c0e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.141296:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64200/0x1cd7fe5368736c07 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc67 expref: 18 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.141311:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65200/0x1cd7fe5368736c0e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc67 expref: 17 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.141319:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619933.141321:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619933.141379:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e6f65200, rc 0) 00000100:00100000:0.0:1788619933.141452:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79008aed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+17:14908:x1875503571731328:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 387us (606us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619933.141459:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1580 00000100:00100000:1.0:1788619933.144631:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619933.144670:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619933.144750:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571731584 00000100:00100000:3.0:1788619933.144759:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571731584 00000100:00100000:3.0:1788619933.144826:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1305 00000100:00100000:3.0:1788619933.144836:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cdbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571731584:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619933.144855:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619933.144870:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7278240/0x1cd7fe5368736c15 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dc75 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619933.144928:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79008ad500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589264640:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619933.145019:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619933.145569:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571731456 00000100:00100000:0.0:1788619933.145575:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571731456 00000100:00100000:0.0:1788619933.145620:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1581 00000100:00100000:0.0:1788619933.145628:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2355:x1875503571731456:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619933.145640:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619933.145653:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64400/0x1cd7fe5368736c1c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dc6e expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.145682:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0a00/0x1cd7fe5368736c00 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc60 expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619933.145712:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78cdcc0380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589264768:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619933.145759:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5d2d780 going to sleep for 11 seconds 00000100:00100000:0.0:1788619933.148623:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78cdcc0380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589264768:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619933.148651:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:0.0:1788619933.148671:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64400/0x1cd7fe5368736c1c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc6e expref: 18 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.148691:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619933.148694:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619933.148700:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64400/0x1cd7fe5368736c1c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc6e expref: 18 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.148714:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e6f64400, rc 0) 00000100:00100000:0.0:1788619933.148780:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2355:x1875503571731456:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3151us (4156us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619933.148789:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1581 00002000:00010000:3.0:1788619933.149582:0:6696:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619931 -> 1788619932 00000100:00100000:3.0:1788619933.149603:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79008ad500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589264640:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619933.149628:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d7278240/0x1cd7fe5368736c15 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc75 expref: 9 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.149649:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7278240/0x1cd7fe5368736c15 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc75 expref: 9 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.149661:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78ccca7400, rc 0) 00000100:00100000:3.0:1788619933.149702:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cdbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571731584:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4867us (5031us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619933.149709:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1305 00000100:00100000:1.0:1788619933.154413:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619933.154450:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619933.154594:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571731840 00000100:00100000:0.0:1788619933.154614:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571731840 00000100:00100000:0.0:1788619933.154673:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1306 00000100:00100000:0.0:1788619933.154684:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cdb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571731840:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619933.154701:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619933.154718:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab469d71/0x1cd7fe5368736c23 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dc83 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619933.154786:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78cdcc0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589264896:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619933.154858:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5d2d780 going to sleep for 11 seconds 00000100:00100000:2.0:1788619933.155098:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571731712 00000100:00100000:2.0:1788619933.155103:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571731712 00000100:00100000:2.0:1788619933.155143:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1582 00000100:00100000:2.0:1788619933.155152:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2355:x1875503571731712:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619933.155162:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619933.155177:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0800/0x1cd7fe5368736c2a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dc7c expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.155204:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0a00/0x1cd7fe5368736c00 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc60 expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619933.155227:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64aaa00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589265024:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619933.155254:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164240 going to sleep for 11 seconds 00000100:00100000:2.0:1788619933.158676:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64aaa00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589265024:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619933.158702:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:2.0:1788619933.158721:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0800/0x1cd7fe5368736c2a lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc7c expref: 18 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.158743:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619933.158746:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619933.158752:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0800/0x1cd7fe5368736c2a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc7c expref: 18 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.158767:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed0800, rc 0) 00000100:00100000:2.0:1788619933.158838:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2355:x1875503571731712:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3687us (4433us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619933.158847:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1582 00000100:00100000:1.0:1788619933.160201:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78cdcc0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589264896:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619933.160233:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ab469d71/0x1cd7fe5368736c23 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc83 expref: 9 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.160254:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab469d71/0x1cd7fe5368736c23 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc83 expref: 9 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.160266:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e6f64400, rc 0) 00000100:00100000:1.0:1788619933.160318:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cdb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571731840:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5637us (5866us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619933.160351:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1306 00000100:00100000:0.0:1788619933.169622:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619933.169710:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571731968 00000100:00100000:3.0:1788619933.169720:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571731968 00002000:00100000:3.0:1788619933.169782:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1413120->1900543) req@ffff8e78cdcc2300 x1875503571731968/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619933.169794:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78cdcc2300 with x1875503571731968 ext(1413120->1900543) 00002000:00010000:3.0:1788619933.169800:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78cdcc2300 00000100:00100000:3.0:1788619933.169834:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 780 00000100:00100000:3.0:1788619933.169844:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571731968:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619933.176822:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571731968:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 6967us (7197us total) trans 0 rc 487424/487424 00000100:00100000:3.0:1788619933.176836:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 780 00002000:00100000:3.0:1788619933.176850:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1413120->1900543) req@ffff8e78cdcc2300 x1875503571731968/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:29/0 lens 488/440 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 487424/487424 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619933.176862:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78cdcc2300 with x1875503571731968 ext(1413120->1900543) 00002000:00010000:3.0:1788619933.176870:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78cdcc2300 00000100:00100000:2.0:1788619933.185631:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619933.185734:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571732096 00000100:00100000:0.0:1788619933.185744:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571732096 00000100:00100000:0.0:1788619933.185818:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1583 00000100:00100000:0.0:1788619933.185830:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64aaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+13:14908:x1875503571732096:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619933.185851:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619933.185872:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65800/0x1cd7fe5368736c31 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dc8a expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.185977:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64600/0x1cd7fe5368736c38 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.185994:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64600/0x1cd7fe5368736c38 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.186006:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64600/0x1cd7fe5368736c38 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619933.186034:0:11397:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1900123 00000004:00200000:0.0:1788619933.186039:0:11397:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619933.186058:0:11397:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64600/0x1cd7fe5368736c38 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.186086:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65800/0x1cd7fe5368736c31 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc8a expref: 15 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.186104:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64600/0x1cd7fe5368736c38 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc8a expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.186113:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619933.186116:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619933.186120:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e6f64600, rc 0) 00000100:00100000:0.0:1788619933.186182:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64aaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+14:14908:x1875503571732096:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 355us (561us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619933.186190:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1583 00000100:00100000:0.0:1788619933.189578:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619933.189610:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619933.189699:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571732224 00000100:00100000:1.0:1788619933.189708:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571732224 00000100:00100000:1.0:1788619933.189764:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1584 00000100:00100000:1.0:1788619933.189783:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2355:x1875503571732224:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619933.189806:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619933.189827:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe5368736c3f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dc91 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.189868:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0a00/0x1cd7fe5368736c00 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc60 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619933.189908:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cd9880 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589265152:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619933.189993:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc840 going to sleep for 11 seconds 00000100:00100000:2.0:1788619933.190069:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571732352 00000100:00100000:2.0:1788619933.190074:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571732352 00000100:00100000:2.0:1788619933.190113:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1307 00000100:00100000:2.0:1788619933.190122:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571732352:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619933.190131:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619933.190142:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090281211/0x1cd7fe5368736c46 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dc98 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619933.190190:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64abb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589265280:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619933.190216:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164240 going to sleep for 11 seconds 00000004:00010000:1.0:1788619933.192924:0:9417:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619932 -> 1788619933 00000100:00100000:1.0:1788619933.192958:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cd9880 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589265152:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619933.192985:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:1.0:1788619933.193004:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe5368736c3f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc91 expref: 18 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.193024:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619933.193026:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619933.193031:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe5368736c3f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc91 expref: 18 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.193045:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683da00, rc 0) 00000100:00100000:1.0:1788619933.193098:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2355:x1875503571732224:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3317us (3530us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619933.193106:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1584 00002000:00010000:2.0:1788619933.195828:0:6698:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619932 -> 1788619933 00000100:00100000:2.0:1788619933.195859:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64abb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589265280:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619933.195892:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000090281211/0x1cd7fe5368736c46 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc98 expref: 9 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.196700:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090281211/0x1cd7fe5368736c46 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc98 expref: 9 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.196716:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed0800, rc 0) 00000100:00100000:2.0:1788619933.196846:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571732352:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6725us (7234us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619933.196855:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1307 00000100:00100000:0.0:1788619933.201731:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619933.201805:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619933.201882:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571732608 00000100:00100000:3.0:1788619933.201892:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571732608 00000100:00100000:3.0:1788619933.201959:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1308 00000100:00100000:3.0:1788619933.201970:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2352:x1875503571732608:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619933.201987:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619933.202006:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7278240/0x1cd7fe5368736c4d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dca6 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619933.202067:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79008ad500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589265408:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619933.202150:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebcc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619933.204230:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571732480 00000100:00100000:2.0:1788619933.204244:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571732480 00000100:00100000:2.0:1788619933.204304:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1585 00000100:00100000:2.0:1788619933.204313:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2352:x1875503571732480:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619933.204361:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619933.204381:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1600/0x1cd7fe5368736c54 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dc9f expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.204419:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0a00/0x1cd7fe5368736c00 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc60 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619933.204454:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64abb80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589265536:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619933.204537:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164240 going to sleep for 11 seconds 00000100:00100000:3.0:1788619933.206090:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79008ad500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589265408:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619933.206128:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d7278240/0x1cd7fe5368736c4d lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dca6 expref: 9 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.206153:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7278240/0x1cd7fe5368736c4d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dca6 expref: 9 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.206173:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78ccca7400, rc 0) 00000100:00100000:3.0:1788619933.206245:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2352:x1875503571732608:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4276us (4438us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619933.206254:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1308 00000100:00100000:2.0:1788619933.209642:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64abb80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589265536:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619933.209671:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:2.0:1788619933.209692:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1600/0x1cd7fe5368736c54 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc9f expref: 18 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.209712:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619933.209715:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619933.209720:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1600/0x1cd7fe5368736c54 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc9f expref: 18 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.209735:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed1600, rc 0) 00000100:00100000:2.0:1788619933.209894:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2352:x1875503571732480:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5581us (8172us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619933.209913:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1585 00000100:00100000:1.0:1788619933.214642:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619933.214731:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571732864 00000100:00100000:2.0:1788619933.214741:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571732864 00000100:00100000:2.0:1788619933.214898:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1586 00000100:00100000:2.0:1788619933.214923:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+17:14908:x1875503571732864:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:2.0:1788619933.214979:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619933.215007:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65200/0x1cd7fe5368736c0e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc67 expref: 17 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.215026:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619933.215032:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788619933.215556:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0800/0x1cd7fe5368736c5b lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.215571:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64600/0x1cd7fe5368736c38 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc8a expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.215585:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0800/0x1cd7fe5368736c5b lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.215618:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64600/0x1cd7fe5368736c38 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc8a expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.215632:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64600/0x1cd7fe5368736c38 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc8a expref: 13 pid: 11397 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619933.215646:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64abb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589265664:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619933.215866:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164240 going to sleep for 11 seconds 00000100:00100000:2.0:1788619933.218687:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64abb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589265664:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619933.218715:0:5845:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0800/0x1cd7fe5368736c5b lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619933.220256:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619933.220671:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571732992 00000100:00100000:0.0:1788619933.220681:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571732992 00010000:00010000:0.0:1788619933.220722:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64600/0x1cd7fe5368736c38 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc8a expref: 14 pid: 11397 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619933.220761:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 524 00000100:00100000:0.0:1788619933.220781:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cdad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+14:2354:x1875503571732992:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619933.220796:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619933.220803:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64600/0x1cd7fe5368736c38 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc8a expref: 14 pid: 11397 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.220817:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64600/0x1cd7fe5368736c38 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc8a expref: 14 pid: 11397 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.220837:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64600/0x1cd7fe5368736c38 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc8a expref: 14 pid: 11397 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.220854:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619933.220860:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0800/0x1cd7fe5368736c5b lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.220871:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0800/0x1cd7fe5368736c5b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.220878:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0800/0x1cd7fe5368736c5b lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.221096:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619933.221158:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cdad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571732992:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 378us (909us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619933.221165:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 524 00010000:00010000:2.0:1788619933.222757:0:5845:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0800/0x1cd7fe5368736c5b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.222774:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0800/0x1cd7fe5368736c5b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788619933.222901:0:5845:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967848 00000004:00010000:2.0:1788619933.222961:0:5845:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1048576 -> 1048576 00000004:00010000:2.0:1788619933.222964:0:5845:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619933 -> 1788619933 00000004:00010000:2.0:1788619933.222967:0:5845:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619933 -> 1788619932 00000004:00010000:2.0:1788619933.222969:0:5845:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619933 -> 1788619933 00000004:00010000:2.0:1788619933.222971:0:5845:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1800 -> 1800 00000004:00010000:2.0:1788619933.222978:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000fd4a1578 reply state 00000000a25749be transno 4294967849 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0800/0x1cd7fe5368736c5b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619933.222987:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788619933.223099:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+19:14908:x1875503571732864:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 8176us (8462us total) trans 4294967849 rc 0/0 00000100:00100000:2.0:1788619933.223107:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1586 00000020:00080000:2.0:1788619933.223111:0:5845:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788619933.225054:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0800/0x1cd7fe5368736c5b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.225071:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0800/0x1cd7fe5368736c5b lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.225078:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0800/0x1cd7fe5368736c5b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.225089:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c5ed0800 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788619933.225094:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0800/0x1cd7fe5368736c5b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.225102:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0800/0x1cd7fe5368736c5b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.225119:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619933.225123:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788619933.225128:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0800/0x1cd7fe5368736c5b lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.225135:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0800/0x1cd7fe5368736c5b lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619933.228553:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619933.228648:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571733120 00000100:00100000:1.0:1788619933.228660:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571733120 00000100:00100000:1.0:1788619933.228726:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1587 00000100:00100000:1.0:1788619933.228738:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64abb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+18:14908:x1875503571733120:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619933.228763:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619933.228796:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368736c62 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dcad expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.228828:0:9417:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0a00/0x1cd7fe5368736c00 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc60 expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.228839:0:9417:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368736c62 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcad expref: 19 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.228875:0:9417:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0a00/0x1cd7fe5368736c00 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc60 expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.228888:0:9417:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0a00/0x1cd7fe5368736c00 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc60 expref: 12 pid: 5845 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619933.228902:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cdb100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589265792:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619933.228992:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc840 going to sleep for 11 seconds 00000100:00100000:2.0:1788619933.235597:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619933.236128:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571733248 00000100:00100000:0.0:1788619933.236139:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571733248 00010000:00010000:0.0:1788619933.236254:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0a00/0x1cd7fe5368736c00 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc60 expref: 13 pid: 5845 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619933.236299:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 525 00000100:00100000:0.0:1788619933.236310:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64ab480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571733248:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619933.236362:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619933.236390:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0a00/0x1cd7fe5368736c00 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc60 expref: 13 pid: 5845 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.236404:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0a00/0x1cd7fe5368736c00 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc60 expref: 13 pid: 5845 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.236432:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619933.236438:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368736c62 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcad expref: 19 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.236451:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368736c62 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcad expref: 19 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.236461:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368736c62 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcad expref: 19 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.236521:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368736c62 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcad expref: 19 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619933.236535:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6d02300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589265920:192.168.201.5@tcp:105: 00000100:00100000:0.0:1788619933.236611:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5d2d240 going to sleep for 11 seconds 00000100:00100000:0.0:1788619933.239041:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6d02300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589265920:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619933.239065:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619933.239120:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64ab480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571733248:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 2810us (3532us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619933.239131:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 525 00010000:00010000:1.0:1788619933.240237:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0a00/0x1cd7fe5368736c00 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc60 expref: 12 pid: 5845 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619933.240258:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cdb100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589265792:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619933.240283:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368736c62 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcad expref: 19 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.240299:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619933.240301:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619933.240304:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c400, rc 0) 00000100:00100000:1.0:1788619933.240385:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64abb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+19:14908:x1875503571733120:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 11650us (11843us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619933.240394:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1587 00000100:00100000:3.0:1788619933.243253:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619933.243361:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571733376 00000100:00100000:2.0:1788619933.243370:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571733376 00000100:00100000:2.0:1788619933.243430:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1309 00000100:00100000:2.0:1788619933.243440:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79008ad500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:14908:x1875503571733376:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619933.243458:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619933.243477:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a7b141ab/0x1cd7fe5368736c69 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dcb4 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.243665:0:6696:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000006aec32ce/0x1cd7fe5368736bdd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1269760->1290239) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc44 expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.243675:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(5 11) 00010000:00010000:2.0:1788619933.243679:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(6 11) 00010000:00010000:2.0:1788619933.243685:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a7b141ab/0x1cd7fe5368736c69 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcb4 expref: 9 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.243722:0:6696:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000006aec32ce/0x1cd7fe5368736bdd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1269760->1290239) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc44 expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.243742:0:6696:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000006aec32ce/0x1cd7fe5368736bdd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1269760->1290239) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc44 expref: 6 pid: 7654 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619933.243757:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64ab100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589266048:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619933.244096:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164240 going to sleep for 11 seconds 00000100:00100000:3.0:1788619933.247633:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619933.247707:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571733504 00000100:00100000:1.0:1788619933.247717:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571733504 00002000:00100000:1.0:1788619933.247775:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1269760->1290239) req@ffff8e79008afb80 x1875503571733504/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619933.247787:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e79008afb80 with x1875503571733504 ext(1269760->1290239) 00010000:00010000:1.0:1788619933.247797:0:6701:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000006aec32ce/0x1cd7fe5368736bdd lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1269760->1290239) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc44 expref: 7 pid: 7654 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.247815:0:6701:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000006aec32ce/0x1cd7fe5368736bdd lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1269760->1290239) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc44 expref: 7 pid: 7654 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788619933.247823:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff8e79008afb80 00000100:00100000:1.0:1788619933.247857:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 27 00000100:00100000:1.0:1788619933.247867:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79008afb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571733504:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:2.0:1788619933.252025:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64ab100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589266048:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619933.252061:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a7b141ab/0x1cd7fe5368736c69 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcb4 expref: 9 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.252083:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed1200, rc 0) 00000100:00100000:1.0:1788619933.252633:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79008ad500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+9:14908:x1875503571733376:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 9191us (9386us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619933.252648:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1309 00000100:00100000:1.0:1788619933.255224:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79008afb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571733504:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 7347us (7589us total) trans 4294967522 rc 0/0 00000100:00100000:1.0:1788619933.255237:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 27 00002000:00100000:1.0:1788619933.255249:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1269760->1290239) req@ffff8e79008afb80 x1875503571733504/t4294967522(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:29/0 lens 488/448 e 0 to 0 dl 1788619944 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619933.255260:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e79008afb80 with x1875503571733504 ext(1269760->1290239) 00010000:00010000:1.0:1788619933.255271:0:6701:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000006aec32ce/0x1cd7fe5368736bdd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1269760->1290239) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc44 expref: 9 pid: 7654 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.255289:0:6701:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000006aec32ce/0x1cd7fe5368736bdd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1269760->1290239) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc44 expref: 9 pid: 7654 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1788619933.255297:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff8e79008afb80 00000100:00100000:0.0:1788619933.260654:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619933.260740:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571733632 00000100:00100000:3.0:1788619933.260751:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571733632 00010000:00010000:3.0:1788619933.260806:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000006aec32ce/0x1cd7fe5368736bdd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1269760->1290239) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc44 expref: 9 pid: 7654 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619933.260849:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 526 00000100:00100000:3.0:1788619933.260859:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d02300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571733632:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619933.260876:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619933.260900:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000006aec32ce/0x1cd7fe5368736bdd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1269760->1290239) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc44 expref: 9 pid: 7654 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.260925:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000006aec32ce/0x1cd7fe5368736bdd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1269760->1290239) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc44 expref: 9 pid: 7654 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.260950:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006aec32ce/0x1cd7fe5368736bdd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1269760->1290239) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dc44 expref: 9 pid: 7654 timeout: 525 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.260968:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(6 11) 00010000:00010000:3.0:1788619933.260973:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(6 11) 00010000:00010000:3.0:1788619933.260982:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000a7b141ab/0x1cd7fe5368736c69 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcb4 expref: 8 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.260993:0:5829:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a7b141ab/0x1cd7fe5368736c69 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcb4 expref: 8 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.261032:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000a7b141ab/0x1cd7fe5368736c69 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcb4 expref: 8 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619933.261046:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79008ac380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1875503589266176:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619933.261133:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619933.265118:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79008ac380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1875503589266176:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619933.265150:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619933.265363:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d02300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+8:2355:x1875503571733632:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 4503us (4716us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619933.265378:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 526 00000100:00100000:2.0:1788619933.267795:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619933.267886:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619933.267977:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571733888 00000100:00100000:0.0:1788619933.267987:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571733888 00002000:00010000:0.0:1788619933.268026:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:0.0:1788619933.268031:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64aa680 with x1875503571733888 ext(0->18446744073709551615) 00002000:00010000:0.0:1788619933.268041:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64aa680. 00000100:00100000:0.0:1788619933.268078:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 781 00000100:00100000:0.0:1788619933.268088:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64aa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2355:x1875503571733888:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:0.0:1788619933.268493:0:6703:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1900123 -> 0 00002000:00010000:0.0:1788619933.268507:0:6703:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619933 -> 1788619933 00002000:00010000:0.0:1788619933.268510:0:6703:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619933 -> 1788619932 00002000:00010000:0.0:1788619933.268512:0:6703:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619933 -> 1788619933 00002000:00010000:0.0:1788619933.268514:0:6703:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 391 -> 391 00000100:00100000:0.0:1788619933.268567:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64aa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+10:2355:x1875503571733888:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 480us (678us total) trans 4294967523 rc 0/0 00000100:00100000:0.0:1788619933.268574:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 781 00002000:00010000:0.0:1788619933.268577:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:0.0:1788619933.268580:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64aa680 with x1875503571733888 ext(0->18446744073709551615) 00002000:00010000:0.0:1788619933.268584:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64aa680. 00000100:00100000:3.0:1788619933.269131:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571733760 00000100:00100000:3.0:1788619933.269137:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571733760 00002000:00010000:3.0:1788619933.269163:0:5857:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:3.0:1788619933.269167:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64ab100 with x1875503571733760 00000100:00100000:3.0:1788619933.269262:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 983 00000100:00100000:3.0:1788619933.269272:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64ab100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2355:x1875503571733760:12345-192.168.201.5@tcp:10:fsx.0 00000004:00010000:0.0:1788619933.274286:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 1048576 -> 386252 00000004:00010000:0.0:1788619933.274296:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619933 -> 1788619933 00000004:00010000:0.0:1788619933.274299:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619932 -> 1788619932 00000004:00010000:0.0:1788619933.274301:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619933 -> 1788619933 00000004:00010000:0.0:1788619933.274303:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1800 -> 1800 00000100:00100000:0.0:1788619933.274665:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64ab100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2355:x1875503571733760:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 5390us (6874us total) trans 4294967850 rc 0/0 00000100:00100000:0.0:1788619933.274679:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 983 00002000:00010000:0.0:1788619933.274684:0:5857:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:0.0:1788619933.274688:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64ab100 with x1875503571733760 00000100:00100000:1.0:1788619933.277656:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619933.277745:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571734016 00000100:00100000:2.0:1788619933.277754:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571734016 00000100:00100000:2.0:1788619933.277923:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1588 00000100:00100000:2.0:1788619933.277935:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cdb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+20:14908:x1875503571734016:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619933.277958:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619933.277979:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe5368736c70 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dcbb expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.278068:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe5368736c77 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.278084:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe5368736c77 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.278097:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe5368736c77 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788619933.278125:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 386252 00000004:00200000:2.0:1788619933.278129:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788619933.278147:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe5368736c77 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.278187:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe5368736c70 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcbb expref: 22 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.278206:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe5368736c77 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcbb expref: 21 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.278216:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619933.278219:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788619933.278222:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed1c00, rc 0) 00000100:00100000:2.0:1788619933.278381:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cdb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+21:14908:x1875503571734016:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 448us (732us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619933.278388:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1588 00000100:00100000:2.0:1788619933.283730:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619933.283973:0:13346:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571734144 00000100:00100000:1.0:1788619933.283986:0:13346:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571734144 00002000:00100000:1.0:1788619933.284035:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c64a8380 x1875503571734144/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619933.284047:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64a8380 with x1875503571734144 00000100:00100000:1.0:1788619933.284115:0:13346:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 984 00000100:00100000:1.0:1788619933.284126:0:13346:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64a8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2354:x1875503571734144:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619933.284379:0:13346:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64a8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2354:x1875503571734144:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 254us (659us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619933.284390:0:13346:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 984 00002000:00100000:1.0:1788619933.284398:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c64a8380 x1875503571734144/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/4536 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619933.284406:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64a8380 with x1875503571734144 00000100:00100000:3.0:1788619933.287942:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619933.288752:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571734272 00000100:00100000:2.0:1788619933.288763:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571734272 00002000:00100000:2.0:1788619933.288874:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79008ac700 x1875503571734272/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619933.288886:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79008ac700 with x1875503571734272 00000100:00100000:2.0:1788619933.288926:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 985 00000100:00100000:2.0:1788619933.288935:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79008ac700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2354:x1875503571734272:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619933.289160:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79008ac700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2354:x1875503571734272:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 225us (1226us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619933.289168:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 985 00002000:00100000:2.0:1788619933.289296:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79008ac700 x1875503571734272/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/4536 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619933.289307:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79008ac700 with x1875503571734272 00000100:00100000:1.0:1788619933.293300:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619933.293476:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571734400 00000100:00100000:3.0:1788619933.293488:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571734400 00002000:00100000:3.0:1788619933.293549:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5cd8700 x1875503571734400/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619933.293560:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5cd8700 with x1875503571734400 00000100:00100000:3.0:1788619933.293638:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 986 00000100:00100000:3.0:1788619933.293648:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2354:x1875503571734400:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619933.293880:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2354:x1875503571734400:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 233us (589us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619933.293889:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 986 00002000:00100000:3.0:1788619933.293896:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5cd8700 x1875503571734400/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/4536 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619933.293904:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5cd8700 with x1875503571734400 00000100:00100000:2.0:1788619933.297921:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619933.298029:0:13346:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571734528 00000100:00100000:1.0:1788619933.298039:0:13346:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571734528 00002000:00100000:1.0:1788619933.298089:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c64a9500 x1875503571734528/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619933.298100:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64a9500 with x1875503571734528 00000100:00100000:1.0:1788619933.298249:0:13346:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 987 00000100:00100000:1.0:1788619933.298260:0:13346:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64a9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2352:x1875503571734528:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619933.298592:0:13346:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64a9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2352:x1875503571734528:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 332us (679us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619933.298600:0:13346:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 987 00002000:00100000:1.0:1788619933.298607:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c64a9500 x1875503571734528/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/4536 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619933.298616:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64a9500 with x1875503571734528 00000100:00100000:3.0:1788619933.301992:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619933.302601:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571734656 00000100:00100000:0.0:1788619933.302612:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571734656 00002000:00100000:0.0:1788619933.302664:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79008ad180 x1875503571734656/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619933.302674:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79008ad180 with x1875503571734656 00000100:00100000:0.0:1788619933.302713:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 988 00000100:00100000:0.0:1788619933.302722:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79008ad180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2353:x1875503571734656:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619933.303067:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79008ad180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2353:x1875503571734656:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 344us (1083us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619933.303075:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 988 00002000:00100000:0.0:1788619933.303082:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79008ad180 x1875503571734656/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/4536 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619933.303090:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79008ad180 with x1875503571734656 00000100:00100000:3.0:1788619933.306587:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619933.307434:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571734784 00000100:00100000:2.0:1788619933.307444:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571734784 00002000:00100000:2.0:1788619933.307496:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79008af100 x1875503571734784/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619933.307521:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79008af100 with x1875503571734784 00000100:00100000:0.0:1788619933.307589:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 989 00000100:00100000:0.0:1788619933.307603:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79008af100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2352:x1875503571734784:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619933.307839:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79008af100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2352:x1875503571734784:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 239us (1261us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619933.307847:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 989 00002000:00100000:0.0:1788619933.307854:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79008af100 x1875503571734784/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/4536 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619933.307867:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79008af100 with x1875503571734784 00000100:00100000:3.0:1788619933.310839:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619933.310987:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571734912 00000100:00100000:0.0:1788619933.310997:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571734912 00002000:00100000:0.0:1788619933.311045:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79008ace00 x1875503571734912/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619933.311057:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79008ace00 with x1875503571734912 00000100:00100000:0.0:1788619933.311100:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 990 00000100:00100000:0.0:1788619933.311111:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79008ace00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2353:x1875503571734912:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619933.311653:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79008ace00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2353:x1875503571734912:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 542us (821us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619933.311662:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 990 00002000:00100000:0.0:1788619933.311669:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79008ace00 x1875503571734912/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/4536 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619933.311676:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79008ace00 with x1875503571734912 00000100:00100000:0.0:1788619933.315835:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619933.316005:0:13346:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571735040 00000100:00100000:2.0:1788619933.316014:0:13346:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571735040 00002000:00100000:2.0:1788619933.316064:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6d01180 x1875503571735040/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619933.316076:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6d01180 with x1875503571735040 00000100:00100000:2.0:1788619933.316152:0:13346:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 991 00000100:00100000:2.0:1788619933.316162:0:13346:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d01180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2352:x1875503571735040:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619933.316419:0:13346:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d01180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2352:x1875503571735040:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 257us (595us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619933.316427:0:13346:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 991 00002000:00100000:2.0:1788619933.316434:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c6d01180 x1875503571735040/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/4536 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619933.316443:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c6d01180 with x1875503571735040 00000100:00100000:1.0:1788619933.320155:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619933.320304:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571735168 00000100:00100000:0.0:1788619933.320314:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571735168 00002000:00100000:0.0:1788619933.320385:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5cd9c00 x1875503571735168/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619933.320397:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5cd9c00 with x1875503571735168 00000100:00100000:0.0:1788619933.320438:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 992 00000100:00100000:0.0:1788619933.320447:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2353:x1875503571735168:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619933.320703:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2353:x1875503571735168:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 255us (556us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619933.320715:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 992 00002000:00100000:0.0:1788619933.320722:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5cd9c00 x1875503571735168/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/4536 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619933.320731:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5cd9c00 with x1875503571735168 00000100:00100000:3.0:1788619933.324091:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619933.324521:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571735296 00000100:00100000:0.0:1788619933.324531:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571735296 00002000:00100000:0.0:1788619933.324578:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79008ac380 x1875503571735296/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619933.324590:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79008ac380 with x1875503571735296 00000100:00100000:0.0:1788619933.324628:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 993 00000100:00100000:0.0:1788619933.324638:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79008ac380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2352:x1875503571735296:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619933.324881:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79008ac380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2352:x1875503571735296:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 244us (799us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619933.324889:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 993 00002000:00100000:0.0:1788619933.324897:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79008ac380 x1875503571735296/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/4536 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619933.324905:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79008ac380 with x1875503571735296 00000100:00100000:3.0:1788619933.328430:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619933.328607:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571735424 00000100:00100000:1.0:1788619933.328617:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571735424 00002000:00100000:1.0:1788619933.328662:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79008af800 x1875503571735424/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619933.328674:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79008af800 with x1875503571735424 00000100:00100000:1.0:1788619933.328711:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 994 00000100:00100000:1.0:1788619933.328721:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79008af800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2353:x1875503571735424:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619933.328941:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79008af800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2353:x1875503571735424:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 221us (519us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619933.328950:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 994 00002000:00100000:1.0:1788619933.328956:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79008af800 x1875503571735424/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/4536 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619933.328965:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79008af800 with x1875503571735424 00000100:00100000:1.0:1788619933.332845:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619933.332984:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571735552 00000100:00100000:0.0:1788619933.332994:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571735552 00002000:00100000:0.0:1788619933.333040:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5cd8a80 x1875503571735552/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619933.333052:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5cd8a80 with x1875503571735552 00000100:00100000:0.0:1788619933.333092:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 995 00000100:00100000:0.0:1788619933.333101:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2352:x1875503571735552:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619933.333371:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2352:x1875503571735552:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 270us (533us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619933.333379:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 995 00002000:00100000:0.0:1788619933.333386:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5cd8a80 x1875503571735552/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/4536 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619933.333396:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5cd8a80 with x1875503571735552 00000100:00100000:3.0:1788619933.337370:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619933.337574:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571735680 00000100:00100000:1.0:1788619933.337583:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571735680 00002000:00100000:1.0:1788619933.337629:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79008aea00 x1875503571735680/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619933.337641:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79008aea00 with x1875503571735680 00000100:00100000:1.0:1788619933.337681:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 996 00000100:00100000:1.0:1788619933.337690:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79008aea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2353:x1875503571735680:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619933.337913:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79008aea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2353:x1875503571735680:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 223us (590us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619933.337921:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 996 00002000:00100000:1.0:1788619933.337928:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e79008aea00 x1875503571735680/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/4536 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619933.337936:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e79008aea00 with x1875503571735680 00000100:00100000:1.0:1788619933.341581:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619933.341676:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571735808 00000100:00100000:0.0:1788619933.341686:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571735808 00002000:00100000:0.0:1788619933.341733:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5cdaa00 x1875503571735808/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619933.341745:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5cdaa00 with x1875503571735808 00000100:00100000:0.0:1788619933.341787:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 997 00000100:00100000:0.0:1788619933.341797:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cdaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2352:x1875503571735808:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619933.342010:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cdaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2352:x1875503571735808:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 214us (438us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619933.342018:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 997 00002000:00100000:0.0:1788619933.342026:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5cdaa00 x1875503571735808/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/4536 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619933.342034:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5cdaa00 with x1875503571735808 00000100:00100000:3.0:1788619933.345433:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619933.345603:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571735936 00000100:00100000:1.0:1788619933.345613:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571735936 00002000:00100000:1.0:1788619933.345661:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5c09880 x1875503571735936/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619933.345673:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5c09880 with x1875503571735936 00000100:00100000:1.0:1788619933.345711:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 998 00000100:00100000:1.0:1788619933.345720:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c09880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2353:x1875503571735936:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619933.345930:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c09880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2353:x1875503571735936:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 210us (506us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619933.345939:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 998 00002000:00100000:1.0:1788619933.345946:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5c09880 x1875503571735936/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/4536 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619933.345955:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5c09880 with x1875503571735936 00000100:00100000:2.0:1788619933.349756:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619933.349907:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571736064 00000100:00100000:0.0:1788619933.349917:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571736064 00002000:00100000:0.0:1788619933.349965:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c64a8000 x1875503571736064/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619933.349976:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64a8000 with x1875503571736064 00000100:00100000:0.0:1788619933.350008:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 999 00000100:00100000:0.0:1788619933.350017:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64a8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2355:x1875503571736064:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:0.0:1788619933.353933:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 386252 -> 386252 00000004:00010000:0.0:1788619933.353944:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619933 -> 1788619933 00000004:00010000:0.0:1788619933.353948:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619932 -> 1788619932 00000004:00010000:0.0:1788619933.353951:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619933 -> 1788619933 00000004:00010000:0.0:1788619933.353953:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1800 -> 1800 00000100:00100000:0.0:1788619933.354027:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64a8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+23:2355:x1875503571736064:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 4006us (4276us total) trans 4294967851 rc 0/0 00000100:00100000:0.0:1788619933.354039:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 999 00002000:00100000:0.0:1788619933.354052:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c64a8000 x1875503571736064/t4294967851(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/448 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619933.354063:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64a8000 with x1875503571736064 00000100:00100000:2.0:1788619933.356030:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619933.356565:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571736192 00000100:00100000:1.0:1788619933.356575:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571736192 00000100:00100000:1.0:1788619933.356687:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1589 00000100:00100000:1.0:1788619933.356698:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64ab800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+23:2354:x1875503571736192:12345-192.168.201.5@tcp:16:fsx.0 00000001:00080000:0.0:1788619933.377896:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967851 is committed 00000001:00080000:0.0:1788619933.377914:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967846 is committed 00000001:00080000:0.0:1788619933.377922:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967847 is committed 00000100:00100000:2.0:1788619933.378193:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64ab800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571736192:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 21491us (22164us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619933.378204:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1589 00000100:00100000:1.0:1788619933.381983:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619933.382111:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571736320 00000100:00100000:0.0:1788619933.382121:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571736320 00000100:00100000:0.0:1788619933.382217:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1590 00000100:00100000:0.0:1788619933.382228:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+8:14908:x1875503571736320:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619933.382248:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619933.382267:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe5368736c7e lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dcc2 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.382297:0:9417:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368736c62 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcad expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.382314:0:9417:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe5368736c7e lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcc2 expref: 9 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.382390:0:9417:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368736c62 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcad expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.382404:0:9417:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368736c62 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcad expref: 13 pid: 9417 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619933.382418:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c71f1180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589266304:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619933.382514:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5d2dd80 going to sleep for 11 seconds 00000100:00100000:2.0:1788619933.385925:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619933.386007:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571736448 00000100:00100000:3.0:1788619933.386016:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571736448 00010000:00010000:3.0:1788619933.386052:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368736c62 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcad expref: 14 pid: 9417 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619933.386094:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 527 00000100:00100000:3.0:1788619933.386104:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64a8a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571736448:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619933.386118:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619933.386143:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368736c62 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcad expref: 14 pid: 9417 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.386157:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368736c62 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcad expref: 14 pid: 9417 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.386186:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619933.386192:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe5368736c7e lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcc2 expref: 9 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.386204:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe5368736c7e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcc2 expref: 9 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.386214:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe5368736c7e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcc2 expref: 9 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.386250:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe5368736c7e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcc2 expref: 9 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619933.386263:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5027480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589266432:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619933.386364:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeba80 going to sleep for 11 seconds 00010000:00010000:0.0:1788619933.386862:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368736c62 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcad expref: 14 pid: 9417 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619933.386880:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c71f1180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589266304:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619933.386905:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe5368736c7e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcc2 expref: 9 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.386916:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619933.386919:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619933.386922:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e6f65e00, rc 0) 00000100:00100000:0.0:1788619933.387002:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571736320:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4775us (5028us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619933.387010:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1590 00000100:00100000:3.0:1788619933.390097:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5027480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589266432:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619933.390122:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619933.390180:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64a8a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571736448:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 4077us (4263us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619933.390192:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 527 00000100:00100000:2.0:1788619933.391181:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619933.391259:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571736576 00000100:00100000:1.0:1788619933.391268:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571736576 00002000:00100000:1.0:1788619933.391312:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c64a9f80 x1875503571736576/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619933.391349:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64a9f80 with x1875503571736576 00000100:00100000:1.0:1788619933.391383:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1000 00000100:00100000:1.0:1788619933.391393:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64a9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571736576:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619933.391645:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64a9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571736576:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 252us (467us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619933.391657:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1000 00002000:00100000:1.0:1788619933.391664:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c64a9f80 x1875503571736576/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:29/0 lens 488/4536 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619933.391673:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64a9f80 with x1875503571736576 00000100:00100000:2.0:1788619933.398068:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619933.398154:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571736704 00000100:00100000:3.0:1788619933.398164:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571736704 00002000:00100000:3.0:1788619933.398214:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c64a8700 x1875503571736704/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619933.398224:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64a8700 with x1875503571736704 00000100:00100000:3.0:1788619933.398266:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1001 00000100:00100000:3.0:1788619933.398276:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64a8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571736704:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619933.398591:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64a8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571736704:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 314us (531us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619933.398599:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1001 00002000:00100000:3.0:1788619933.398606:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c64a8700 x1875503571736704/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:29/0 lens 488/4536 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619933.398615:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64a8700 with x1875503571736704 00000100:00100000:3.0:1788619933.401311:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619933.401731:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571736832 00000100:00100000:2.0:1788619933.401741:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571736832 00000100:00100000:2.0:1788619933.401894:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1591 00000100:00100000:2.0:1788619933.401905:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5027480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571736832:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619933.401922:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619933.401940:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0800/0x1cd7fe5368736c85 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dcc9 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.402018:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1e00/0x1cd7fe5368736c8c lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.402035:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1e00/0x1cd7fe5368736c8c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.402046:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1e00/0x1cd7fe5368736c8c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788619933.402073:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 386252 00000004:00200000:2.0:1788619933.402077:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788619933.402093:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1e00/0x1cd7fe5368736c8c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.402122:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0800/0x1cd7fe5368736c85 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcc9 expref: 11 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.402140:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1e00/0x1cd7fe5368736c8c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcc9 expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.402149:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619933.402152:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788619933.402155:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed1e00, rc 0) 00000100:00100000:2.0:1788619933.402261:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5027480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:14908:x1875503571736832:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 358us (959us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619933.402268:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1591 00000100:00100000:2.0:1788619933.405127:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619933.405240:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571736960 00000100:00100000:1.0:1788619933.405248:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571736960 00000100:00100000:1.0:1788619933.405292:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1310 00000100:00100000:1.0:1788619933.405300:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900985050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+8:2355:x1875503571736960:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619933.405315:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619933.405356:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe5368736c93 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dcd0 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619933.405414:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cdb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589266560:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619933.405560:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc240 going to sleep for 11 seconds 00002000:00010000:0.0:1788619933.408851:0:7654:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619932 -> 1788619933 00000100:00100000:0.0:1788619933.408882:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cdb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589266560:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619933.408916:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe5368736c93 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcd0 expref: 9 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.408941:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe5368736c93 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcd0 expref: 9 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.408954:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c600, rc 0) 00000100:00100000:0.0:1788619933.409061:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900985050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+8:2355:x1875503571736960:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3759us (3940us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619933.409069:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1310 00000100:00100000:0.0:1788619933.413037:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619933.413155:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571737088 00000100:00100000:2.0:1788619933.413164:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571737088 00000100:00100000:2.0:1788619933.413269:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1311 00000100:00100000:2.0:1788619933.413281:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+8:2354:x1875503571737088:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619933.413299:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619933.413315:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d4b76022/0x1cd7fe5368736c9a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dcd7 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619933.413408:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64a9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589266688:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619933.413491:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164b40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619933.421471:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64a9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589266688:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619933.421526:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d4b76022/0x1cd7fe5368736c9a lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcd7 expref: 9 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.421553:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d4b76022/0x1cd7fe5368736c9a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcd7 expref: 9 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.421567:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed0a00, rc 0) 00000100:00100000:0.0:1788619933.421661:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+8:2354:x1875503571737088:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8382us (8634us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619933.421671:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1311 00000100:00100000:3.0:1788619933.424878:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619933.424969:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571737344 00000100:00100000:2.0:1788619933.424978:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571737344 00000100:00100000:2.0:1788619933.425066:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1592 00000100:00100000:2.0:1788619933.425077:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7432d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:14908:x1875503571737344:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:2.0:1788619933.425126:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619933.425149:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1e00/0x1cd7fe5368736c8c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcc9 expref: 10 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.425165:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619933.425168:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788619933.425427:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe5368736ca1 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.425440:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe5368736c77 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcbb expref: 12 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.425446:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe5368736ca1 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.425471:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe5368736c77 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcbb expref: 12 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.425481:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe5368736c77 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcbb expref: 12 pid: 5844 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619933.425490:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64aa300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589266816:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619933.425567:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81640c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619933.432878:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619933.433045:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571737472 00000100:00100000:0.0:1788619933.433054:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571737472 00010000:00010000:0.0:1788619933.433088:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe5368736c77 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcbb expref: 13 pid: 5844 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619933.433116:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 528 00000100:00100000:0.0:1788619933.433123:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7432a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571737472:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619933.433133:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619933.433137:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe5368736c77 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcbb expref: 13 pid: 5844 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.433146:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe5368736c77 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcbb expref: 13 pid: 5844 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.433164:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619933.433167:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe5368736ca1 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.433175:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe5368736ca1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.433183:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe5368736ca1 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.433201:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619933.433253:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7432a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571737472:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 130us (382us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619933.433258:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 528 00010000:00010000:1.0:1788619933.434630:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe5368736c77 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcbb expref: 12 pid: 5844 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619933.434654:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64aa300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589266816:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619933.434677:0:5845:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe5368736ca1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.434687:0:5845:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe5368736ca1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.434695:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe5368736ca1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788619933.434877:0:5845:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967852 00000004:00010000:1.0:1788619933.434937:0:5845:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 386252 -> 386252 00000004:00010000:1.0:1788619933.434942:0:5845:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619933 -> 1788619933 00000004:00010000:1.0:1788619933.434944:0:5845:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619932 -> 1788619932 00000004:00010000:1.0:1788619933.434947:0:5845:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619933 -> 1788619933 00000004:00010000:1.0:1788619933.434949:0:5845:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1800 -> 774 00000004:00010000:1.0:1788619933.434957:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000d0aaf7a3 reply state 00000000b9999fd3 transno 4294967853 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe5368736ca1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619933.434967:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788619933.435091:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7432d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+12:14908:x1875503571737344:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 10015us (10221us total) trans 4294967853 rc 0/0 00000100:00100000:1.0:1788619933.435099:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1592 00000020:00080000:1.0:1788619933.435103:0:5845:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1788619933.437197:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619933.437273:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571737600 00000100:00100000:0.0:1788619933.437281:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571737600 00000100:00100000:0.0:1788619933.437356:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1312 00000100:00100000:0.0:1788619933.437366:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7433100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+8:14908:x1875503571737600:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619933.437383:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619933.437399:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000675d07ec/0x1cd7fe5368736ca8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dcde expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.437431:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000a7b141ab/0x1cd7fe5368736c69 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcb4 expref: 9 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.437441:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=1s v=(6 11) 00010000:00010000:0.0:1788619933.437445:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(7 11) 00010000:00010000:0.0:1788619933.437451:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000675d07ec/0x1cd7fe5368736ca8 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcde expref: 9 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.437484:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000a7b141ab/0x1cd7fe5368736c69 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcb4 expref: 9 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.437513:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000a7b141ab/0x1cd7fe5368736c69 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcb4 expref: 9 pid: 6696 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619933.437528:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c71f0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589266944:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619933.437602:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5d2dd80 going to sleep for 11 seconds 00000100:00100000:3.0:1788619933.439874:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c71f0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589266944:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619933.439920:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000675d07ec/0x1cd7fe5368736ca8 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcde expref: 9 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.439941:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e6f64a00, rc 0) 00000100:00100000:3.0:1788619933.439998:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7433100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571737600:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2633us (2809us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619933.440005:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1312 00010000:00010000:2.0:1788619933.440827:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe5368736ca1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.440838:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe5368736ca1 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.440849:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe5368736ca1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.440854:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c5ed1400 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:2.0:1788619933.440858:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe5368736ca1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.440865:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe5368736ca1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.440880:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619933.440884:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788619933.440887:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe5368736ca1 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.440894:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe5368736ca1 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619933.443644:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619933.443724:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571737728 00000100:00100000:1.0:1788619933.443735:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571737728 00010000:00010000:1.0:1788619933.443788:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000a7b141ab/0x1cd7fe5368736c69 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcb4 expref: 10 pid: 6696 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619933.443829:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 529 00000100:00100000:1.0:1788619933.443840:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcadc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+10:2354:x1875503571737728:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619933.443854:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619933.443879:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000a7b141ab/0x1cd7fe5368736c69 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcb4 expref: 10 pid: 6696 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.443894:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000a7b141ab/0x1cd7fe5368736c69 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcb4 expref: 10 pid: 6696 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.443919:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a7b141ab/0x1cd7fe5368736c69 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcb4 expref: 10 pid: 6696 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.443939:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(7 11) 00010000:00010000:1.0:1788619933.443943:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(7 11) 00010000:00010000:1.0:1788619933.443953:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000675d07ec/0x1cd7fe5368736ca8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcde expref: 8 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.443964:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000675d07ec/0x1cd7fe5368736ca8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcde expref: 8 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.444006:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000675d07ec/0x1cd7fe5368736ca8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcde expref: 8 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619933.444023:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cd9500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589267072:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619933.444108:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc300 going to sleep for 11 seconds 00000100:00100000:0.0:1788619933.445798:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cd9500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589267072:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619933.445824:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619933.445871:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcadc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571737728:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 2032us (2237us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619933.445881:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 529 00000100:00100000:0.0:1788619933.448937:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619933.449022:0:13347:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571737856 00000100:00100000:3.0:1788619933.449030:0:13347:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571737856 00002000:00010000:3.0:1788619933.449070:0:13347:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1853533->18446744073709551615) 00002000:00100000:3.0:1788619933.449075:0:13347:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c71f1500 with x1875503571737856 ext(1853533->18446744073709551615) 00002000:00010000:3.0:1788619933.449087:0:13347:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c71f1500. 00000100:00100000:3.0:1788619933.449120:0:13347:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 782 00000100:00100000:3.0:1788619933.449130:0:13347:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571737856:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:3.0:1788619933.449386:0:13347:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 0 -> 1853533 00002000:00010000:3.0:1788619933.449390:0:13347:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619933 -> 1788619933 00002000:00010000:3.0:1788619933.449393:0:13347:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619933 -> 1788619932 00002000:00010000:3.0:1788619933.449395:0:13347:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619933 -> 1788619933 00002000:00010000:3.0:1788619933.449397:0:13347:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 391 -> 391 00000100:00100000:3.0:1788619933.449456:0:13347:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571737856:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 327us (530us total) trans 4294967524 rc 0/0 00000100:00100000:3.0:1788619933.449465:0:13347:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 782 00002000:00010000:3.0:1788619933.449468:0:13347:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1853533->18446744073709551615) 00002000:00100000:3.0:1788619933.449471:0:13347:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c71f1500 with x1875503571737856 ext(1853533->18446744073709551615) 00002000:00010000:3.0:1788619933.449476:0:13347:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c71f1500. 00000100:00100000:3.0:1788619933.453165:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619933.453363:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571737984 00000100:00100000:0.0:1788619933.453373:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571737984 00002000:00100000:0.0:1788619933.453421:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1810432->1814527) req@ffff8e78c7433b80 x1875503571737984/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619933.453429:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7433b80 with x1875503571737984 ext(1810432->1814527) 00002000:00010000:0.0:1788619933.453434:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c7433b80 00000100:00100000:0.0:1788619933.453464:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 783 00000100:00100000:0.0:1788619933.453471:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7433b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571737984:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619933.453752:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7433b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571737984:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 279us (593us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619933.453760:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 783 00002000:00100000:0.0:1788619933.453776:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1810432->1814527) req@ffff8e78c7433b80 x1875503571737984/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:29/0 lens 488/4536 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619933.453784:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7433b80 with x1875503571737984 ext(1810432->1814527) 00002000:00010000:0.0:1788619933.453788:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c7433b80 00000100:00100000:2.0:1788619933.456810:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619933.456891:0:13347:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571738112 00000100:00100000:3.0:1788619933.456901:0:13347:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571738112 00002000:00100000:3.0:1788619933.456944:0:13347:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x4:0x0] (1814528->1818623) req@ffff8e77c64a8e00 x1875503571738112/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619933.456956:0:13347:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64a8e00 with x1875503571738112 ext(1814528->1818623) 00002000:00010000:3.0:1788619933.456961:0:13347:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64a8e00 00000100:00100000:3.0:1788619933.456994:0:13347:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 784 00000100:00100000:3.0:1788619933.457003:0:13347:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64a8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571738112:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619933.457228:0:13347:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64a8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571738112:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 225us (426us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619933.457240:0:13347:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 784 00002000:00100000:3.0:1788619933.457246:0:13347:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x4:0x0] (1814528->1818623) req@ffff8e77c64a8e00 x1875503571738112/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:29/0 lens 488/4536 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619933.457253:0:13347:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64a8e00 with x1875503571738112 ext(1814528->1818623) 00002000:00010000:3.0:1788619933.457256:0:13347:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64a8e00 00000100:00100000:1.0:1788619933.461488:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619933.461643:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571738240 00000100:00100000:0.0:1788619933.461654:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571738240 00002000:00100000:0.0:1788619933.461699:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1818624->1822719) req@ffff8e77c5cd9f80 x1875503571738240/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619933.461709:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5cd9f80 with x1875503571738240 ext(1818624->1822719) 00002000:00010000:0.0:1788619933.461715:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5cd9f80 00000100:00100000:0.0:1788619933.461855:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 785 00000100:00100000:0.0:1788619933.461865:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571738240:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619933.462131:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571738240:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 266us (810us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619933.462139:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 785 00002000:00100000:0.0:1788619933.462146:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1818624->1822719) req@ffff8e77c5cd9f80 x1875503571738240/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:29/0 lens 488/4536 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619933.462154:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5cd9f80 with x1875503571738240 ext(1818624->1822719) 00002000:00010000:0.0:1788619933.462157:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5cd9f80 00000100:00100000:3.0:1788619933.464856:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619933.465709:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571738368 00000100:00100000:2.0:1788619933.465722:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571738368 00002000:00100000:2.0:1788619933.465861:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1822720->1826815) req@ffff8e78c7432300 x1875503571738368/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619933.465872:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7432300 with x1875503571738368 ext(1822720->1826815) 00002000:00010000:2.0:1788619933.465879:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c7432300 00000100:00100000:2.0:1788619933.465947:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 786 00000100:00100000:2.0:1788619933.465958:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7432300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571738368:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619933.466217:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7432300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571738368:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 260us (1369us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619933.466225:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 786 00002000:00100000:2.0:1788619933.466232:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1822720->1826815) req@ffff8e78c7432300 x1875503571738368/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:29/0 lens 488/4536 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619933.466241:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7432300 with x1875503571738368 ext(1822720->1826815) 00002000:00010000:2.0:1788619933.466245:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c7432300 00000100:00100000:1.0:1788619933.469942:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619933.470044:0:13347:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571738496 00000100:00100000:3.0:1788619933.470055:0:13347:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571738496 00002000:00100000:3.0:1788619933.470102:0:13347:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x4:0x0] (1826816->1830911) req@ffff8e77c5cda300 x1875503571738496/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619933.470114:0:13347:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5cda300 with x1875503571738496 ext(1826816->1830911) 00002000:00010000:3.0:1788619933.470120:0:13347:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5cda300 00000100:00100000:3.0:1788619933.470152:0:13347:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 787 00000100:00100000:3.0:1788619933.470162:0:13347:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cda300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571738496:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619933.470553:0:13347:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cda300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571738496:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 391us (618us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619933.470561:0:13347:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 787 00002000:00100000:3.0:1788619933.470568:0:13347:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x4:0x0] (1826816->1830911) req@ffff8e77c5cda300 x1875503571738496/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:29/0 lens 488/4536 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619933.470577:0:13347:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5cda300 with x1875503571738496 ext(1826816->1830911) 00002000:00010000:3.0:1788619933.470580:0:13347:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5cda300 00000100:00100000:1.0:1788619933.473669:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619933.473768:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571738624 00000100:00100000:2.0:1788619933.473779:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571738624 00002000:00100000:2.0:1788619933.473827:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1830912->1835007) req@ffff8e77c5cdad80 x1875503571738624/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619933.473838:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5cdad80 with x1875503571738624 ext(1830912->1835007) 00002000:00010000:2.0:1788619933.473843:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5cdad80 00000100:00100000:2.0:1788619933.473881:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 788 00000100:00100000:2.0:1788619933.473890:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cdad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571738624:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619933.474138:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cdad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571738624:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 247us (476us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619933.474145:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 788 00002000:00100000:2.0:1788619933.474152:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1830912->1835007) req@ffff8e77c5cdad80 x1875503571738624/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:29/0 lens 488/4536 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619933.474161:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5cdad80 with x1875503571738624 ext(1830912->1835007) 00002000:00010000:2.0:1788619933.474164:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5cdad80 00010000:00010000:3.0:1788619933.475074:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788619933.478085:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619933.478191:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571738752 00000100:00100000:1.0:1788619933.478200:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571738752 00002000:00100000:1.0:1788619933.478251:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1835008->1839103) req@ffff8e77c71f2d80 x1875503571738752/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619933.478263:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c71f2d80 with x1875503571738752 ext(1835008->1839103) 00002000:00010000:1.0:1788619933.478270:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c71f2d80 00000100:00100000:1.0:1788619933.478306:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 789 00000100:00100000:1.0:1788619933.478316:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571738752:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619933.478682:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571738752:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 365us (604us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619933.478691:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 789 00002000:00100000:1.0:1788619933.478699:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1835008->1839103) req@ffff8e77c71f2d80 x1875503571738752/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:29/0 lens 488/4536 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619933.478708:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c71f2d80 with x1875503571738752 ext(1835008->1839103) 00002000:00010000:1.0:1788619933.478712:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c71f2d80 00000100:00100000:1.0:1788619933.482133:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619933.482244:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571738880 00000100:00100000:2.0:1788619933.482254:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571738880 00002000:00100000:2.0:1788619933.482299:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1839104->1843199) req@ffff8e77c5cd9500 x1875503571738880/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619933.482315:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5cd9500 with x1875503571738880 ext(1839104->1843199) 00002000:00010000:2.0:1788619933.482378:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5cd9500 00000100:00100000:2.0:1788619933.482402:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 790 00000100:00100000:2.0:1788619933.482411:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cd9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571738880:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619933.482668:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cd9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571738880:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 257us (542us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619933.482676:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 790 00002000:00100000:2.0:1788619933.482683:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1839104->1843199) req@ffff8e77c5cd9500 x1875503571738880/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:29/0 lens 488/4536 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619933.482691:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5cd9500 with x1875503571738880 ext(1839104->1843199) 00002000:00010000:2.0:1788619933.482695:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5cd9500 00000100:00100000:2.0:1788619933.486141:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619933.486238:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571739008 00000100:00100000:1.0:1788619933.486247:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571739008 00002000:00100000:1.0:1788619933.486291:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1843200->1847295) req@ffff8e77c64ab480 x1875503571739008/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619933.486302:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64ab480 with x1875503571739008 ext(1843200->1847295) 00002000:00010000:1.0:1788619933.486307:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64ab480 00000100:00100000:1.0:1788619933.486374:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 791 00000100:00100000:1.0:1788619933.486384:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64ab480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571739008:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619933.486628:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64ab480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2353:x1875503571739008:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 243us (496us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619933.486635:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 791 00002000:00100000:1.0:1788619933.486642:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1843200->1847295) req@ffff8e77c64ab480 x1875503571739008/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:29/0 lens 488/4536 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619933.486657:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64ab480 with x1875503571739008 ext(1843200->1847295) 00002000:00010000:1.0:1788619933.486661:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64ab480 00000100:00100000:3.0:1788619933.489455:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619933.490031:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571739136 00000100:00100000:2.0:1788619933.490040:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571739136 00002000:00100000:2.0:1788619933.490086:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1847296->1851391) req@ffff8e78c7430380 x1875503571739136/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619933.490097:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7430380 with x1875503571739136 ext(1847296->1851391) 00002000:00010000:2.0:1788619933.490103:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c7430380 00000100:00100000:2.0:1788619933.490142:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 792 00000100:00100000:2.0:1788619933.490152:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7430380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571739136:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619933.490597:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7430380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571739136:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 445us (1150us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619933.490605:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 792 00002000:00100000:2.0:1788619933.490613:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1847296->1851391) req@ffff8e78c7430380 x1875503571739136/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:29/0 lens 488/4536 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619933.490622:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c7430380 with x1875503571739136 ext(1847296->1851391) 00002000:00010000:2.0:1788619933.490626:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c7430380 00000100:00100000:0.0:1788619933.493849:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619933.493959:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571739264 00000100:00100000:1.0:1788619933.493969:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571739264 00002000:00100000:1.0:1788619933.494020:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1851392->1855487) req@ffff8e77c71f1180 x1875503571739264/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619933.494031:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c71f1180 with x1875503571739264 ext(1851392->1855487) 00002000:00010000:1.0:1788619933.494036:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c71f1180 00000100:00100000:1.0:1788619933.494067:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 793 00000100:00100000:1.0:1788619933.494076:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571739264:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619933.494659:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571739264:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 248us (484us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619933.494668:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 793 00002000:00100000:1.0:1788619933.494677:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1851392->1855487) req@ffff8e77c71f1180 x1875503571739264/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:29/0 lens 488/4536 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619933.494686:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c71f1180 with x1875503571739264 ext(1851392->1855487) 00002000:00010000:1.0:1788619933.494689:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c71f1180 00000100:00100000:1.0:1788619933.497849:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619933.497949:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571739392 00000100:00100000:0.0:1788619933.497959:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571739392 00002000:00100000:0.0:1788619933.498006:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1810432->1853532) req@ffff8e77c5cdbb80 x1875503571739392/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619933.498018:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5cdbb80 with x1875503571739392 ext(1810432->1853532) 00002000:00010000:0.0:1788619933.498023:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5cdbb80 00000100:00100000:0.0:1788619933.498059:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 794 00000100:00100000:0.0:1788619933.498069:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cdbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571739392:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:2.0:1788619933.506297:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cdbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:2354:x1875503571739392:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 8218us (8447us total) trans 4294967525 rc 0/0 00000100:00100000:2.0:1788619933.506311:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 794 00002000:00100000:2.0:1788619933.506367:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1810432->1853532) req@ffff8e77c5cdbb80 x1875503571739392/t4294967525(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:29/0 lens 488/448 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619933.506378:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5cdbb80 with x1875503571739392 ext(1810432->1853532) 00002000:00010000:2.0:1788619933.506385:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5cdbb80 00000100:00100000:1.0:1788619933.509319:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619933.509526:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571739520 00000100:00100000:2.0:1788619933.509535:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571739520 00000100:00100000:2.0:1788619933.509633:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1313 00000100:00100000:2.0:1788619933.509644:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cdb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+12:2355:x1875503571739520:12345-192.168.201.5@tcp:16:fsx.0 00000001:00080000:1.0:1788619933.537049:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967523 is committed 00000001:00080000:1.0:1788619933.537069:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967522 is committed 00000001:00080000:1.0:1788619933.537080:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967525 is committed 00000100:00100000:1.0:1788619933.537250:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cdb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571739520:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 27602us (27934us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619933.537267:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1313 00000100:00100000:0.0:1788619933.540062:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619933.540179:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571739648 00000100:00100000:1.0:1788619933.540191:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571739648 00000100:00100000:1.0:1788619933.540283:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1593 00000100:00100000:1.0:1788619933.540294:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+12:14908:x1875503571739648:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619933.540315:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619933.540370:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368736caf lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dce5 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.540455:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368736cb6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.540469:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368736cb6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.540479:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368736cb6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788619933.540523:0:9417:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1853533 00000004:00200000:1.0:1788619933.540527:0:9417:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788619933.540545:0:9417:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368736cb6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.540573:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368736caf lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dce5 expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.540590:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368736cb6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dce5 expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.540603:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619933.540605:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788619933.540609:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683dc00, rc 0) 00000100:00100000:1.0:1788619933.540676:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571739648:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 384us (623us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619933.540682:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1593 00000100:00100000:2.0:1788619933.543097:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619933.543143:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619933.543193:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571739904 00000100:00100000:3.0:1788619933.543202:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571739904 00000100:00100000:3.0:1788619933.543251:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1314 00000100:00100000:3.0:1788619933.543261:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64a8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571739904:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619933.543279:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619933.543295:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7278240/0x1cd7fe5368736cbd lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dcf3 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619933.543397:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c7431c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589267200:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619933.543485:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeba80 going to sleep for 11 seconds 00000100:00100000:0.0:1788619933.544215:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571739776 00000100:00100000:0.0:1788619933.544221:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571739776 00000100:00100000:0.0:1788619933.544277:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1594 00000100:00100000:0.0:1788619933.544286:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64aa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571739776:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619933.544299:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619933.544314:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65a00/0x1cd7fe5368736cc4 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dcec expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.544367:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe5368736c7e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcc2 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619933.544396:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5772d80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589267328:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619933.544468:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5eaf0c0 going to sleep for 11 seconds 00002000:00010000:1.0:1788619933.548548:0:7654:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619932 -> 1788619933 00000100:00100000:1.0:1788619933.548581:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c7431c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589267200:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619933.548622:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d7278240/0x1cd7fe5368736cbd lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcf3 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.548651:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7278240/0x1cd7fe5368736cbd lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcf3 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.548666:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78ccca7400, rc 0) 00000100:00100000:1.0:1788619933.548723:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64a8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571739904:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5462us (5634us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619933.548731:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1314 00000004:00010000:1.0:1788619933.550246:0:11397:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619932 -> 1788619933 00000004:00010000:1.0:1788619933.550254:0:11397:0:(mdt_lvb.c:201:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks: 774 -> 1800 00000100:00100000:1.0:1788619933.550275:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5772d80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589267328:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619933.550294:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 386252 00010000:00010000:1.0:1788619933.550310:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65a00/0x1cd7fe5368736cc4 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcec expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.550366:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619933.550369:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619933.550375:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65a00/0x1cd7fe5368736cc4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcec expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.550387:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e6f65a00, rc 0) 00000100:00100000:1.0:1788619933.550440:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64aa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571739776:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6154us (7295us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619933.550448:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1594 00000100:00100000:2.0:1788619933.554046:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619933.554091:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619933.554155:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571740032 00000100:00100000:1.0:1788619933.554164:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571740032 00000100:00100000:1.0:1788619933.554221:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1595 00000100:00100000:1.0:1788619933.554232:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64a9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571740032:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619933.554253:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619933.554276:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368736ccb lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dcfa expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.554310:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe5368736c7e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcc2 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619933.554392:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cdb480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589267456:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619933.554473:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cce40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619933.554540:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571740160 00000100:00100000:0.0:1788619933.554545:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571740160 00000100:00100000:0.0:1788619933.554583:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1315 00000100:00100000:0.0:1788619933.554590:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64a9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571740160:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619933.554598:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619933.554610:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007fc9f597/0x1cd7fe5368736cd2 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dd01 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619933.554646:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5772d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589267584:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619933.554675:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5eaf900 going to sleep for 11 seconds 00000100:00100000:2.0:1788619933.558941:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5772d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589267584:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619933.558989:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007fc9f597/0x1cd7fe5368736cd2 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd01 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.559016:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007fc9f597/0x1cd7fe5368736cd2 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd01 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.559031:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e6f65c00, rc 0) 00000100:00100000:2.0:1788619933.559097:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64a9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571740160:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4506us (5003us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619933.559105:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1315 00000100:00100000:1.0:1788619933.559299:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cdb480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589267456:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619933.559313:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 386252 00010000:00010000:1.0:1788619933.559351:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368736ccb lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcfa expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.559367:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619933.559370:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619933.559374:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368736ccb lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcfa expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.559385:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d200, rc 0) 00000100:00100000:1.0:1788619933.559409:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64a9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571740032:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5180us (5376us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619933.559416:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1595 00000100:00100000:3.0:1788619933.563413:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619933.563452:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619933.563579:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571740288 00000100:00100000:1.0:1788619933.563589:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571740288 00000100:00100000:1.0:1788619933.563639:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1596 00000100:00100000:1.0:1788619933.563648:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7433480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571740288:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619933.563667:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619933.563687:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368736cd9 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dd08 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.563722:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe5368736c7e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcc2 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619933.563770:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cdb480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589267712:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619933.563850:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cce40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619933.564078:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571740416 00000100:00100000:0.0:1788619933.564082:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571740416 00000100:00100000:0.0:1788619933.564121:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1316 00000100:00100000:0.0:1788619933.564128:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7431c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571740416:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619933.564137:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619933.564149:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1387451/0x1cd7fe5368736ce0 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dd0f expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619933.564186:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5772d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589267840:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619933.564216:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5eaf300 going to sleep for 11 seconds 00000100:00100000:3.0:1788619933.566573:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5772d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589267840:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619933.566610:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e1387451/0x1cd7fe5368736ce0 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd0f expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.566635:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1387451/0x1cd7fe5368736ce0 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd0f expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.566648:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e6f64600, rc 0) 00000100:00100000:3.0:1788619933.566708:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7431c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571740416:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2580us (3255us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619933.566716:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1316 00000100:00100000:1.0:1788619933.566832:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cdb480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589267712:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619933.566846:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 386252 00010000:00010000:1.0:1788619933.566858:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368736cd9 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd08 expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.566871:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619933.566874:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619933.566878:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368736cd9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd08 expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.566887:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c000, rc 0) 00000100:00100000:1.0:1788619933.567037:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7433480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571740288:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3389us (3634us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619933.567044:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1596 00000100:00100000:3.0:1788619933.569936:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619933.570035:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619933.570167:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571740672 00000100:00100000:1.0:1788619933.570177:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571740672 00000100:00100000:1.0:1788619933.570234:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1317 00000100:00100000:1.0:1788619933.570243:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74a8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571740672:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619933.570259:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619933.570275:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe5368736ce7 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dd1d expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619933.570370:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cdb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589267968:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619933.570441:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cce40 going to sleep for 11 seconds 00000100:00100000:2.0:1788619933.571569:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571740544 00000100:00100000:2.0:1788619933.571578:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571740544 00000100:00100000:2.0:1788619933.571675:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1597 00000100:00100000:2.0:1788619933.571685:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7432a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571740544:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619933.571702:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619933.571719:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe5368736cee lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dd16 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.571751:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe5368736c7e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcc2 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619933.571795:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64ad880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589268096:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619933.571853:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164b40 going to sleep for 11 seconds 00000100:00100000:1.0:1788619933.575841:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cdb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589267968:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619933.575883:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe5368736ce7 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd1d expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.575922:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe5368736ce7 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd1d expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.575937:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683ca00, rc 0) 00000100:00100000:1.0:1788619933.576002:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74a8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571740672:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5758us (5965us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619933.576010:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1317 00000100:00100000:2.0:1788619933.577700:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64ad880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589268096:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619933.577723:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 386252 00010000:00010000:2.0:1788619933.577741:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe5368736cee lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd16 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.577826:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619933.577830:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619933.577838:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1400/0x1cd7fe5368736cee lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd16 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.577854:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed1400, rc 0) 00000100:00100000:2.0:1788619933.578036:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7432a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571740544:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 6351us (8108us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619933.578045:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1597 00000100:00100000:0.0:1788619933.581041:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619933.581126:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571740800 00000100:00100000:1.0:1788619933.581136:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571740800 00000100:00100000:1.0:1788619933.581220:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1598 00000100:00100000:1.0:1788619933.581231:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5772d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571740800:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619933.581249:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619933.581269:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368736cf5 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dd24 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.581299:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe5368736c7e lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcc2 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.581309:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368736cf5 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd24 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.581543:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe5368736c7e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcc2 expref: 10 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.581557:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe5368736c7e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcc2 expref: 10 pid: 9417 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619933.581570:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cdb480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589268224:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619933.581654:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cce40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619933.584133:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619933.584434:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571740928 00000100:00100000:3.0:1788619933.584445:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571740928 00002000:00100000:3.0:1788619933.584503:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c501c380 x1875503571740928/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619933.584515:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c501c380 with x1875503571740928 00010000:00010000:3.0:1788619933.584524:0:5858:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe5368736c7e lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcc2 expref: 11 pid: 9417 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.584539:0:5858:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe5368736c7e lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcc2 expref: 11 pid: 9417 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788619933.584547:0:5858:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e77c501c380 00000100:00100000:3.0:1788619933.584586:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 43 00000100:00100000:3.0:1788619933.584596:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571740928:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:1.0:1788619933.585746:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cdb480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589268224:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619933.585789:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368736cf5 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd24 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.585806:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619933.585809:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619933.585813:0:5845:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368736cf5 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd24 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.585822:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c000, rc 0) 00000100:00100000:1.0:1788619933.585871:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5772d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571740800:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4642us (4838us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619933.585878:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1598 00000004:00010000:3.0:1788619933.588174:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 386252 -> 386252 00000004:00010000:3.0:1788619933.588183:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619933 -> 1788619933 00000004:00010000:3.0:1788619933.588186:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619933 -> 1788619932 00000004:00010000:3.0:1788619933.588188:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619933 -> 1788619933 00000004:00010000:3.0:1788619933.588190:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 1800 -> 774 00000100:00100000:3.0:1788619933.588271:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+13:2355:x1875503571740928:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 3672us (4144us total) trans 4294967854 rc 0/0 00000100:00100000:3.0:1788619933.588281:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 43 00002000:00100000:3.0:1788619933.588293:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c501c380 x1875503571740928/t4294967854(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:29/0 lens 488/448 e 0 to 0 dl 1788619944 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619933.588302:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c501c380 with x1875503571740928 00010000:00010000:3.0:1788619933.588311:0:5858:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe5368736c7e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcc2 expref: 13 pid: 9417 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.588364:0:5858:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe5368736c7e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcc2 expref: 13 pid: 9417 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788619933.588372:0:5858:0:(mdt_io.c:157:mdt_rw_hpreq_check()) lustre-MDT0000: refreshed 1 locks timeout for req ffff8e77c501c380 00000100:00100000:0.0:1788619933.591793:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619933.591921:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571741056 00000100:00100000:1.0:1788619933.591930:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571741056 00010000:00010000:1.0:1788619933.591968:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe5368736c7e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcc2 expref: 13 pid: 9417 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619933.592010:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 530 00000100:00100000:1.0:1788619933.592020:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+13:2353:x1875503571741056:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619933.592034:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619933.592061:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe5368736c7e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcc2 expref: 13 pid: 9417 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.592075:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe5368736c7e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcc2 expref: 13 pid: 9417 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.592100:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe5368736c7e lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcc2 expref: 13 pid: 9417 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.592117:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619933.592124:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368736cf5 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd24 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.592135:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368736cf5 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd24 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.592145:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368736cf5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd24 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.592185:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368736cf5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd24 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619933.592199:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cdb480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589268352:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619933.592426:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cce40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619933.600019:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619933.600154:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571741184 00000100:00100000:3.0:1788619933.600166:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571741184 00000100:00100000:3.0:1788619933.600247:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1002 00000100:00100000:3.0:1788619933.600259:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b34000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571741184:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:3.0:1788619933.600595:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 386252 -> 386252 00000004:00010000:3.0:1788619933.600599:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619933 -> 1788619933 00000004:00010000:3.0:1788619933.600602:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619932 -> 1788619932 00000004:00010000:3.0:1788619933.600604:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619933 -> 1788619933 00000004:00010000:3.0:1788619933.600606:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 774 -> 774 00000100:00100000:3.0:1788619933.600672:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b34000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571741184:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 414us (663us total) trans 4294967855 rc 0/0 00000100:00100000:3.0:1788619933.600680:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1002 00000100:00100000:0.0:1788619933.603183:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619933.603270:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571741312 00000100:00100000:3.0:1788619933.603279:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571741312 00000100:00100000:3.0:1788619933.603364:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1599 00000100:00100000:3.0:1788619933.603374:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d02300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+13:14908:x1875503571741312:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619933.603397:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619933.603417:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7e00/0x1cd7fe5368736cfc lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dd2b expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.603511:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7c00/0x1cd7fe5368736d03 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.603528:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7c00/0x1cd7fe5368736d03 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.603540:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7c00/0x1cd7fe5368736d03 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788619933.603567:0:9417:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1853533 00000004:00200000:3.0:1788619933.603571:0:9417:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788619933.603589:0:9417:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7c00/0x1cd7fe5368736d03 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.603612:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7e00/0x1cd7fe5368736cfc lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd2b expref: 15 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.603632:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7c00/0x1cd7fe5368736d03 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd2b expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.603640:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619933.603643:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788619933.603647:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78ccca7c00, rc 0) 00000100:00100000:3.0:1788619933.603696:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d02300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+14:14908:x1875503571741312:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 323us (522us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619933.603704:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1599 00000100:00100000:1.0:1788619933.606082:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cdb480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589268352:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619933.606106:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619933.606158:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+13:2353:x1875503571741056:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 14140us (14374us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619933.606169:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 530 00000100:00100000:0.0:1788619933.607661:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619933.607746:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571741440 00000100:00100000:2.0:1788619933.607754:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571741440 00000100:00100000:2.0:1788619933.607916:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1600 00000100:00100000:2.0:1788619933.607927:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d02680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571741440:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619933.607951:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619933.607972:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe5368736d0a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dd32 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.608012:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368736cf5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd24 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619933.608056:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64ad880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589268480:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619933.608144:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164b40 going to sleep for 11 seconds 00000004:00010000:2.0:1788619933.610134:0:5844:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619932 -> 1788619933 00000100:00100000:2.0:1788619933.610164:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64ad880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589268480:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619933.610185:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 386252 00010000:00010000:2.0:1788619933.610202:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe5368736d0a lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd32 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.610220:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619933.610224:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619933.610230:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe5368736d0a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd32 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.610244:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed1c00, rc 0) 00000100:00100000:2.0:1788619933.610370:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d02680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571741440:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2444us (2714us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619933.610378:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1600 00000100:00100000:0.0:1788619933.613574:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619933.613670:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571741568 00000100:00100000:1.0:1788619933.613680:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571741568 00000100:00100000:1.0:1788619933.613828:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1601 00000100:00100000:1.0:1788619933.613840:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d00000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+13:2355:x1875503571741568:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619933.613861:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619933.613881:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368736d11 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dd39 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.613986:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368736cf5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd24 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619933.614028:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cdb480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589268608:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619933.614147:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cce40 going to sleep for 11 seconds 00000100:00100000:1.0:1788619933.616692:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cdb480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589268608:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619933.616718:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 386252 00010000:00010000:1.0:1788619933.616740:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368736d11 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd39 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.616831:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619933.616835:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619933.616841:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368736d11 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd39 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.616858:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683ca00, rc 0) 00000100:00100000:1.0:1788619933.616960:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d00000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+13:2355:x1875503571741568:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3122us (3396us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619933.616968:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1601 00000100:00100000:0.0:1788619933.619880:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619933.620019:0:13346:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571741696 00000100:00100000:2.0:1788619933.620028:0:13346:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571741696 00000100:00100000:2.0:1788619933.620091:0:13346:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1003 00000100:00100000:2.0:1788619933.620100:0:13346:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6d01f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571741696:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:2.0:1788619933.620835:0:13346:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 386252 -> 386252 00000004:00010000:2.0:1788619933.620840:0:13346:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619933 -> 1788619933 00000004:00010000:2.0:1788619933.620843:0:13346:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619933 -> 1788619932 00000004:00010000:2.0:1788619933.620845:0:13346:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619933 -> 1788619933 00000004:00010000:2.0:1788619933.620848:0:13346:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 774 -> 774 00000100:00100000:2.0:1788619933.620997:0:13346:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6d01f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571741696:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 895us (1123us total) trans 4294967856 rc 0/0 00000100:00100000:2.0:1788619933.621005:0:13346:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1003 00000100:00100000:0.0:1788619933.623680:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619933.623774:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571741824 00000100:00100000:3.0:1788619933.623783:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571741824 00000100:00100000:3.0:1788619933.623838:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1602 00000100:00100000:3.0:1788619933.623847:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7583100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+13:2355:x1875503571741824:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619933.623866:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619933.623886:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7e00/0x1cd7fe5368736d18 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dd40 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.623923:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368736cf5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd24 expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619933.623967:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c74aa680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589268736:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619933.624086:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb480 going to sleep for 11 seconds 00000004:00010000:3.0:1788619933.626298:0:5846:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619932 -> 1788619933 00000100:00100000:3.0:1788619933.626456:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c74aa680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589268736:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619933.626479:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 386252 00010000:00010000:3.0:1788619933.626508:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7e00/0x1cd7fe5368736d18 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd40 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.626530:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619933.626534:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619933.626540:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7e00/0x1cd7fe5368736d18 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd40 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.626559:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78ccca7e00, rc 0) 00000100:00100000:3.0:1788619933.626609:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7583100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+13:2355:x1875503571741824:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2763us (2938us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619933.626619:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1602 00000100:00100000:0.0:1788619933.629282:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619933.629521:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571741952 00000100:00100000:3.0:1788619933.629530:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571741952 00000100:00100000:3.0:1788619933.629588:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1603 00000100:00100000:3.0:1788619933.629597:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c0b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+13:2355:x1875503571741952:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619933.629620:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619933.629640:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6e00/0x1cd7fe5368736d1f lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dd47 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.629677:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368736cf5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd24 expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619933.629715:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c74aa680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589268864:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619933.629847:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb480 going to sleep for 11 seconds 00000100:00100000:2.0:1788619933.632606:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c74aa680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589268864:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619933.632633:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 386252 00010000:00010000:2.0:1788619933.632651:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6e00/0x1cd7fe5368736d1f lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd47 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.632671:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619933.632674:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619933.632680:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6e00/0x1cd7fe5368736d1f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd47 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.632696:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78ccca6e00, rc 0) 00000100:00100000:2.0:1788619933.632865:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c0b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+13:2355:x1875503571741952:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3267us (3591us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619933.632874:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1603 00000100:00100000:0.0:1788619933.636284:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619933.636579:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571742080 00000100:00100000:2.0:1788619933.636588:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571742080 00000100:00100000:2.0:1788619933.636645:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1318 00000100:00100000:2.0:1788619933.636656:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c09500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571742080:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619933.636672:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619933.636689:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063f558c8/0x1cd7fe5368736d26 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dd4e expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619933.636748:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64ad880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589268992:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619933.637020:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164b40 going to sleep for 11 seconds 00000100:00100000:2.0:1788619933.639290:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64ad880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589268992:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619933.639520:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000063f558c8/0x1cd7fe5368736d26 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd4e expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.639546:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063f558c8/0x1cd7fe5368736d26 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd4e expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.639561:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed1c00, rc 0) 00000100:00100000:2.0:1788619933.639665:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c09500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571742080:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3010us (3388us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619933.639674:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1318 00000100:00100000:1.0:1788619933.645744:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619933.645860:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571742208 00000100:00100000:0.0:1788619933.645869:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571742208 00000100:00100000:0.0:1788619933.645928:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1319 00000100:00100000:0.0:1788619933.645938:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cdb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:14908:x1875503571742208:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619933.645957:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619933.645977:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004217bfe6/0x1cd7fe5368736d2d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dd55 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.646010:0:6698:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000675d07ec/0x1cd7fe5368736ca8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcde expref: 5 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.646017:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(7 11) 00010000:00010000:0.0:1788619933.646020:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(8 11) 00010000:00010000:0.0:1788619933.646024:0:6698:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004217bfe6/0x1cd7fe5368736d2d lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1744896->1748991] (req 1744896->1748991) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd55 expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.646053:0:6698:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000675d07ec/0x1cd7fe5368736ca8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcde expref: 5 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.646063:0:6698:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000675d07ec/0x1cd7fe5368736ca8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcde expref: 5 pid: 6697 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619933.646074:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c0aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589269120:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619933.646152:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5eaff00 going to sleep for 11 seconds 00000100:00100000:2.0:1788619933.648401:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c0aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589269120:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619933.648434:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004217bfe6/0x1cd7fe5368736d2d lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1744896->1748991] (req 1744896->1748991) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd55 expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.648454:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e6f64c00, rc 0) 00000100:00100000:2.0:1788619933.648505:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cdb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:14908:x1875503571742208:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2568us (2771us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619933.648513:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1319 00000100:00100000:2.0:1788619933.651040:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619933.651159:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571742336 00000100:00100000:3.0:1788619933.651168:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571742336 00010000:00010000:3.0:1788619933.651210:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000675d07ec/0x1cd7fe5368736ca8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcde expref: 6 pid: 6697 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619933.651254:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 531 00000100:00100000:3.0:1788619933.651264:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64ad880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571742336:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619933.651279:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619933.651300:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000675d07ec/0x1cd7fe5368736ca8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcde expref: 6 pid: 6697 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.651314:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000675d07ec/0x1cd7fe5368736ca8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcde expref: 6 pid: 6697 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.651374:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000675d07ec/0x1cd7fe5368736ca8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dcde expref: 6 pid: 6697 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.651404:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(8 11) 00010000:00010000:3.0:1788619933.651409:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(8 11) 00010000:00010000:3.0:1788619933.651418:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000004217bfe6/0x1cd7fe5368736d2d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1744896->1748991) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd55 expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.651429:0:13330:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004217bfe6/0x1cd7fe5368736d2d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1744896->1748991) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd55 expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.651465:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000004217bfe6/0x1cd7fe5368736d2d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1744896->1748991) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd55 expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619933.651479:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c74a9f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589269248:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619933.651563:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebcc0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619933.655991:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c74a9f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589269248:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619933.656016:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619933.656070:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64ad880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571742336:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 4806us (5038us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619933.656087:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 531 00000100:00100000:1.0:1788619933.658167:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619933.658240:0:13347:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571742464 00000100:00100000:3.0:1788619933.658249:0:13347:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571742464 00002000:00100000:3.0:1788619933.658291:0:13347:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x4:0x0] (1413120->1855487) req@ffff8e78df8b1f80 x1875503571742464/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619933.658302:0:13347:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df8b1f80 with x1875503571742464 ext(1413120->1855487) 00002000:00010000:3.0:1788619933.658307:0:13347:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df8b1f80 00000100:00100000:1.0:1788619933.658561:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 795 00000100:00100000:1.0:1788619933.658573:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571742464:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619933.666180:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571742464:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 7599us (8008us total) trans 0 rc 442368/442368 00000100:00100000:3.0:1788619933.666195:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 795 00002000:00100000:3.0:1788619933.666208:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1413120->1855487) req@ffff8e78df8b1f80 x1875503571742464/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/440 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 442368/442368 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619933.666219:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df8b1f80 with x1875503571742464 ext(1413120->1855487) 00002000:00010000:3.0:1788619933.666226:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df8b1f80 00000100:00100000:1.0:1788619933.674833:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619933.674927:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571742592 00000100:00100000:2.0:1788619933.674937:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571742592 00002000:00100000:2.0:1788619933.674979:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1744896->1810431) req@ffff8e78df8b3b80 x1875503571742592/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619933.674986:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df8b3b80 with x1875503571742592 ext(1744896->1810431) 00002000:00010000:2.0:1788619933.674991:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df8b3b80 00000100:00100000:2.0:1788619933.675023:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 796 00000100:00100000:2.0:1788619933.675030:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571742592:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:2.0:1788619933.677988:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571742592:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 2947us (3153us total) trans 4294967526 rc 0/0 00000100:00100000:2.0:1788619933.678002:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 796 00002000:00100000:2.0:1788619933.678014:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1744896->1810431) req@ffff8e78df8b3b80 x1875503571742592/t4294967526(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/448 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619933.678031:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df8b3b80 with x1875503571742592 ext(1744896->1810431) 00002000:00010000:2.0:1788619933.678037:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df8b3b80 00000100:00100000:0.0:1788619933.679844:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619933.679931:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571742720 00000100:00100000:3.0:1788619933.679940:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571742720 00000100:00100000:3.0:1788619933.679986:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1320 00000100:00100000:3.0:1788619933.679997:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c0a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2355:x1875503571742720:12345-192.168.201.5@tcp:16:fsx.0 00000001:00080000:1.0:1788619933.701801:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967526 is committed 00000100:00100000:0.0:1788619933.701966:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c0a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571742720:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 21964us (22122us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619933.701978:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1320 00000100:00100000:1.0:1788619933.704746:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619933.704827:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571742848 00000100:00100000:2.0:1788619933.704838:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571742848 00000100:00100000:2.0:1788619933.704935:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1321 00000100:00100000:2.0:1788619933.704943:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:14908:x1875503571742848:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619933.704958:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619933.704972:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dd8a44c0/0x1cd7fe5368736d34 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dd5c expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.704997:0:6696:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000004217bfe6/0x1cd7fe5368736d2d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1744896->1748991) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd55 expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.705003:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(8 11) 00010000:00010000:2.0:1788619933.705007:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(9 11) 00010000:00010000:2.0:1788619933.705010:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd8a44c0/0x1cd7fe5368736d34 lrc: 4/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1806336->1818623] (req 1806336->1818623) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd5c expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.705036:0:6696:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000004217bfe6/0x1cd7fe5368736d2d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1744896->1748991) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd55 expref: 5 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.705045:0:6696:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000004217bfe6/0x1cd7fe5368736d2d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1744896->1748991) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd55 expref: 5 pid: 6698 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619933.705055:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64ac000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589269376:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619933.705137:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164540 going to sleep for 11 seconds 00000100:00100000:2.0:1788619933.707354:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64ac000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589269376:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619933.707386:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000dd8a44c0/0x1cd7fe5368736d34 lrc: 3/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1806336->1818623] (req 1806336->1818623) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd5c expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.707407:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed1600, rc 0) 00000100:00100000:2.0:1788619933.707459:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:14908:x1875503571742848:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2515us (2719us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619933.707468:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1321 00000100:00100000:1.0:1788619933.707917:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619933.708011:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571742976 00000100:00100000:3.0:1788619933.708018:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571742976 00010000:00010000:3.0:1788619933.708049:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000004217bfe6/0x1cd7fe5368736d2d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1744896->1748991) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd55 expref: 6 pid: 6698 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619933.708103:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 532 00000100:00100000:1.0:1788619933.708113:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571742976:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619933.708128:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619933.708151:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000004217bfe6/0x1cd7fe5368736d2d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1744896->1748991) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd55 expref: 6 pid: 6698 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.708165:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000004217bfe6/0x1cd7fe5368736d2d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1744896->1748991) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd55 expref: 6 pid: 6698 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.708263:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004217bfe6/0x1cd7fe5368736d2d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1744896->1748991) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd55 expref: 6 pid: 6698 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.708279:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(9 11) 00010000:00010000:1.0:1788619933.708283:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(9 11) 00010000:00010000:1.0:1788619933.708292:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000dd8a44c0/0x1cd7fe5368736d34 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1806336->1818623) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd5c expref: 5 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.708302:0:5831:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dd8a44c0/0x1cd7fe5368736d34 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1806336->1818623) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd5c expref: 5 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.708399:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000dd8a44c0/0x1cd7fe5368736d34 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1806336->1818623) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd5c expref: 5 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619933.708412:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df8b2a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5831:1875503589269504:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619933.708475:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619933.710191:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df8b2a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5831:1875503589269504:192.168.201.5@tcp:105: 00010000:00010000:0.0:1788619933.710212:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619933.710256:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571742976:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 2145us (2340us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619933.710266:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 532 00000100:00100000:3.0:1788619933.711032:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619933.711139:0:13347:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571743104 00000100:00100000:2.0:1788619933.711145:0:13347:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571743104 00002000:00100000:2.0:1788619933.711188:0:13347:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x4:0x0] (1806336->1810431) req@ffff8e77c74a9f80 x1875503571743104/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619933.711198:0:13347:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c74a9f80 with x1875503571743104 ext(1806336->1810431) 00002000:00010000:2.0:1788619933.711203:0:13347:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c74a9f80 00000100:00100000:2.0:1788619933.711265:0:13347:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 797 00000100:00100000:2.0:1788619933.711280:0:13347:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74a9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571743104:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619933.711510:0:13347:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74a9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571743104:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 235us (478us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619933.711516:0:13347:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 797 00002000:00100000:2.0:1788619933.711523:0:13347:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x4:0x0] (1806336->1810431) req@ffff8e77c74a9f80 x1875503571743104/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:29/0 lens 488/4536 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619933.711531:0:13347:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c74a9f80 with x1875503571743104 ext(1806336->1810431) 00002000:00010000:2.0:1788619933.711535:0:13347:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c74a9f80 00000100:00100000:1.0:1788619933.713985:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619933.714104:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571743232 00000100:00100000:0.0:1788619933.714113:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571743232 00002000:00100000:0.0:1788619933.714161:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1810432->1814527) req@ffff8e78df8b1500 x1875503571743232/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619933.714173:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df8b1500 with x1875503571743232 ext(1810432->1814527) 00002000:00010000:0.0:1788619933.714178:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df8b1500 00000100:00100000:0.0:1788619933.714210:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 798 00000100:00100000:0.0:1788619933.714219:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571743232:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619933.714447:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571743232:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 229us (472us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619933.714455:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 798 00002000:00100000:0.0:1788619933.714462:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1810432->1814527) req@ffff8e78df8b1500 x1875503571743232/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:29/0 lens 488/4536 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619933.714471:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df8b1500 with x1875503571743232 ext(1810432->1814527) 00002000:00010000:0.0:1788619933.714475:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df8b1500 00000100:00100000:1.0:1788619933.721906:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619933.721954:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571743360 00000100:00100000:3.0:1788619933.721966:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571743360 00002000:00100000:3.0:1788619933.722026:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1814528->1818623) req@ffff8e78df8b2680 x1875503571743360/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619933.722038:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df8b2680 with x1875503571743360 ext(1814528->1818623) 00002000:00010000:3.0:1788619933.722043:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df8b2680 00000100:00100000:0.0:1788619933.722175:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 799 00000100:00100000:0.0:1788619933.722189:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571743360:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:0.0:1788619933.722605:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571743360:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 418us (709us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1788619933.722613:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 799 00002000:00100000:0.0:1788619933.722621:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1814528->1818623) req@ffff8e78df8b2680 x1875503571743360/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:29/0 lens 488/4536 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619933.722630:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df8b2680 with x1875503571743360 ext(1814528->1818623) 00002000:00010000:0.0:1788619933.722634:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df8b2680 00000100:00100000:1.0:1788619933.730199:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619933.730467:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571743488 00000100:00100000:2.0:1788619933.730479:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571743488 00000100:00100000:2.0:1788619933.730591:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1604 00000100:00100000:2.0:1788619933.730602:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+13:2355:x1875503571743488:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619933.730623:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619933.730643:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0800/0x1cd7fe5368736d3b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dd63 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.730681:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368736cf5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd24 expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619933.730725:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64ac000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589269632:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619933.730885:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164a80 going to sleep for 11 seconds 00000100:00100000:2.0:1788619933.733039:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64ac000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589269632:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619933.733064:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 386252 00010000:00010000:2.0:1788619933.733095:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0800/0x1cd7fe5368736d3b lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd63 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.733117:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619933.733121:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619933.733126:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0800/0x1cd7fe5368736d3b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd63 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.733141:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed0800, rc 0) 00000100:00100000:2.0:1788619933.733198:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+13:2355:x1875503571743488:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2598us (3008us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619933.733207:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1604 00000100:00100000:0.0:1788619933.736196:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619933.736351:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571743616 00000100:00100000:3.0:1788619933.736362:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571743616 00000100:00100000:3.0:1788619933.736420:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1605 00000100:00100000:3.0:1788619933.736430:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c08e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571743616:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619933.736449:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619933.736469:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7e00/0x1cd7fe5368736d42 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dd6a expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.736526:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368736cf5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd24 expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619933.736569:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c74a9500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589269760:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619933.736658:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619933.739003:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c74a9500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589269760:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619933.739033:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 386252 00010000:00010000:0.0:1788619933.739052:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7e00/0x1cd7fe5368736d42 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd6a expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.739090:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619933.739094:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619933.739099:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7e00/0x1cd7fe5368736d42 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd6a expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.739115:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78ccca7e00, rc 0) 00000100:00100000:0.0:1788619933.739179:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c08e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571743616:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2749us (2992us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619933.739188:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1605 00000100:00100000:0.0:1788619933.742518:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619933.742577:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571743744 00000100:00100000:2.0:1788619933.742585:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571743744 00000100:00100000:2.0:1788619933.742636:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1322 00000100:00100000:2.0:1788619933.742644:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c08a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571743744:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619933.742662:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619933.742678:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d4b76022/0x1cd7fe5368736d49 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dd71 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.742703:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(9 11) 00010000:00010000:2.0:1788619933.742707:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(9 11) 00010000:00010000:2.0:1788619933.742716:0:7654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d4b76022/0x1cd7fe5368736d49 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd71 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.742733:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d4b76022/0x1cd7fe5368736d49 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd71 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.742747:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed0a00, rc 0) 00000100:00100000:2.0:1788619933.742809:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c08a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571743744:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 165us (302us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619933.742816:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1322 00000100:00100000:1.0:1788619933.746695:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619933.746790:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571743872 00000100:00100000:3.0:1788619933.746800:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571743872 00000100:00100000:3.0:1788619933.746874:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1004 00000100:00100000:3.0:1788619933.746885:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571743872:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:3.0:1788619933.747488:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 386252 -> 386252 00000004:00010000:3.0:1788619933.747504:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619933 -> 1788619933 00000004:00010000:3.0:1788619933.747507:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619933 -> 1788619932 00000004:00010000:3.0:1788619933.747509:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619933 -> 1788619933 00000004:00010000:3.0:1788619933.747511:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 774 -> 774 00000100:00100000:3.0:1788619933.747578:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2355:x1875503571743872:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 692us (890us total) trans 4294967857 rc 0/0 00000100:00100000:3.0:1788619933.747586:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1004 00000100:00100000:3.0:1788619933.752599:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619933.752749:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571744000 00000100:00100000:1.0:1788619933.752760:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571744000 00002000:00100000:1.0:1788619933.752822:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c74a8380 x1875503571744000/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619933.752832:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c74a8380 with x1875503571744000 00000100:00100000:1.0:1788619933.752870:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1005 00000100:00100000:1.0:1788619933.752880:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74a8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2353:x1875503571744000:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619933.753299:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74a8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2353:x1875503571744000:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 418us (706us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619933.753307:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1005 00002000:00100000:1.0:1788619933.753314:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c74a8380 x1875503571744000/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/440 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619933.753363:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c74a8380 with x1875503571744000 00000100:00100000:0.0:1788619933.756423:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619933.756633:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571744128 00000100:00100000:1.0:1788619933.756643:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571744128 00002000:00100000:1.0:1788619933.756692:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5c0b800 x1875503571744128/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619933.756702:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5c0b800 with x1875503571744128 00000100:00100000:1.0:1788619933.756738:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1006 00000100:00100000:1.0:1788619933.756747:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c0b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2355:x1875503571744128:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619933.757015:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c0b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2355:x1875503571744128:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 269us (601us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619933.757023:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1006 00002000:00100000:1.0:1788619933.757032:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5c0b800 x1875503571744128/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/440 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619933.757038:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5c0b800 with x1875503571744128 00000100:00100000:0.0:1788619933.759860:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619933.759997:0:13346:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571744256 00000100:00100000:2.0:1788619933.760006:0:13346:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571744256 00002000:00100000:2.0:1788619933.760052:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5c0aa00 x1875503571744256/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619933.760062:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5c0aa00 with x1875503571744256 00000100:00100000:2.0:1788619933.760107:0:13346:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1007 00000100:00100000:2.0:1788619933.760118:0:13346:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c0aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2355:x1875503571744256:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619933.760398:0:13346:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c0aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2355:x1875503571744256:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 281us (547us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619933.760405:0:13346:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1007 00002000:00100000:2.0:1788619933.760412:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78c5c0aa00 x1875503571744256/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/440 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619933.760420:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78c5c0aa00 with x1875503571744256 00000100:00100000:2.0:1788619933.764070:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619933.764464:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571744384 00000100:00100000:0.0:1788619933.764475:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571744384 00000100:00100000:0.0:1788619933.764545:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1606 00000100:00100000:0.0:1788619933.764557:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64ac000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571744384:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619933.764576:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619933.764596:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65600/0x1cd7fe5368736d50 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dd78 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.764634:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368736cf5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd24 expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619933.764674:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64d3800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589269888:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619933.764753:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5eafe40 going to sleep for 11 seconds 00000004:00010000:2.0:1788619933.767664:0:5844:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619932 -> 1788619933 00000100:00100000:2.0:1788619933.767695:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64d3800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589269888:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619933.767718:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 386252 00010000:00010000:2.0:1788619933.767735:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65600/0x1cd7fe5368736d50 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd78 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.767767:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619933.767770:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619933.767776:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65600/0x1cd7fe5368736d50 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd78 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.767790:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e6f65600, rc 0) 00000100:00100000:2.0:1788619933.767884:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64ac000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+13:2354:x1875503571744384:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3329us (3823us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619933.767893:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1606 00000100:00100000:3.0:1788619933.770622:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619933.770705:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571744512 00000100:00100000:1.0:1788619933.770715:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571744512 00000100:00100000:1.0:1788619933.770783:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1607 00000100:00100000:1.0:1788619933.770793:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74a8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+13:2353:x1875503571744512:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619933.770813:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619933.770834:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368736d57 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dd7f expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.770870:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368736cf5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd24 expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619933.770922:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df8b3100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589270016:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619933.771299:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc600 going to sleep for 11 seconds 00000100:00100000:1.0:1788619933.774073:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df8b3100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589270016:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619933.774172:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 386252 00010000:00010000:1.0:1788619933.774192:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368736d57 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd7f expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.774214:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619933.774218:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619933.774224:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368736d57 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd7f expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.774238:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c400, rc 0) 00000100:00100000:1.0:1788619933.774472:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74a8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+13:2353:x1875503571744512:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3679us (3857us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619933.774481:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1607 00000100:00100000:0.0:1788619933.784291:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619933.786043:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571744768 00000100:00100000:2.0:1788619933.786057:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571744768 00000100:00100000:2.0:1788619933.786222:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1608 00000100:00100000:2.0:1788619933.786232:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64d0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+13:14908:x1875503571744768:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:2.0:1788619933.786284:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619933.786308:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7c00/0x1cd7fe5368736d03 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd2b expref: 13 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.786354:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619933.786359:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788619933.786559:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1200/0x1cd7fe5368736d5e lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.786573:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368736cb6 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dce5 expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.786583:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1200/0x1cd7fe5368736d5e lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.786611:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368736cb6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dce5 expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.786624:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368736cb6 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dce5 expref: 16 pid: 9417 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619933.786637:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64adc00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589270144:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619933.786724:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164a80 going to sleep for 11 seconds 00000100:00100000:3.0:1788619933.789527:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64adc00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589270144:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619933.789556:0:5845:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1200/0x1cd7fe5368736d5e lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619933.791407:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619933.791578:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571744896 00000100:00100000:1.0:1788619933.791588:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571744896 00010000:00010000:1.0:1788619933.791627:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368736cb6 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dce5 expref: 17 pid: 9417 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619933.791669:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 533 00000100:00100000:1.0:1788619933.791679:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64d0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2355:x1875503571744896:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619933.791694:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619933.791701:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368736cb6 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dce5 expref: 17 pid: 9417 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.791714:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368736cb6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dce5 expref: 17 pid: 9417 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.791737:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368736cb6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dce5 expref: 17 pid: 9417 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.791753:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619933.791767:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1200/0x1cd7fe5368736d5e lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.791778:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1200/0x1cd7fe5368736d5e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.791785:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1200/0x1cd7fe5368736d5e lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.791815:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619933.791872:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64d0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571744896:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 193us (473us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619933.791879:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 533 00010000:00010000:2.0:1788619933.792030:0:5845:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1200/0x1cd7fe5368736d5e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.792037:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1200/0x1cd7fe5368736d5e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788619933.792165:0:5845:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967858 00000004:00010000:2.0:1788619933.792213:0:5845:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 386252 -> 386252 00000004:00010000:2.0:1788619933.792217:0:5845:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619933 -> 1788619933 00000004:00010000:2.0:1788619933.792219:0:5845:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619933 -> 1788619932 00000004:00010000:2.0:1788619933.792221:0:5845:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619933 -> 1788619933 00000004:00010000:2.0:1788619933.792223:0:5845:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 774 -> 774 00000004:00010000:2.0:1788619933.792230:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000096b90013 reply state 000000005bf41025 transno 4294967859 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1200/0x1cd7fe5368736d5e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619933.792239:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788619933.792359:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64d0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+15:14908:x1875503571744768:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 6128us (8077us total) trans 4294967859 rc 0/0 00000100:00100000:2.0:1788619933.792366:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1608 00000020:00080000:2.0:1788619933.792370:0:5845:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:1.0:1788619933.795457:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1200/0x1cd7fe5368736d5e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.795474:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1200/0x1cd7fe5368736d5e lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.795482:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1200/0x1cd7fe5368736d5e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.795488:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c5ed1200 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:1.0:1788619933.795504:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1200/0x1cd7fe5368736d5e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.795514:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1200/0x1cd7fe5368736d5e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.795536:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619933.795539:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788619933.795543:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1200/0x1cd7fe5368736d5e lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.795549:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1200/0x1cd7fe5368736d5e lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619933.796420:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619933.796513:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571745024 00000100:00100000:2.0:1788619933.796524:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571745024 00000100:00100000:2.0:1788619933.796578:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1609 00000100:00100000:2.0:1788619933.796588:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64d3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+14:14908:x1875503571745024:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619933.796615:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619933.796635:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1e00/0x1cd7fe5368736d65 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dd86 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.796661:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368736cf5 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd24 expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.796672:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1e00/0x1cd7fe5368736d65 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd86 expref: 15 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.796706:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368736cf5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd24 expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.796720:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368736cf5 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd24 expref: 15 pid: 5845 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619933.796735:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64ae680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589270272:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619933.796826:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81649c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619933.798733:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64ae680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589270272:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619933.798775:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1e00/0x1cd7fe5368736d65 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd86 expref: 15 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.798788:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619933.798790:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619933.798793:0:5844:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1e00/0x1cd7fe5368736d65 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd86 expref: 15 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.798800:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed1e00, rc 0) 00000100:00100000:2.0:1788619933.798844:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64d3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+15:14908:x1875503571745024:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2258us (2431us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619933.798850:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1609 00000100:00100000:1.0:1788619933.800404:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619933.800519:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571745152 00000100:00100000:3.0:1788619933.800527:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571745152 00010000:00010000:3.0:1788619933.800563:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368736cf5 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd24 expref: 16 pid: 5845 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619933.800602:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 534 00000100:00100000:3.0:1788619933.800612:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b3100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571745152:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619933.800627:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619933.800653:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368736cf5 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd24 expref: 16 pid: 5845 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.800666:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368736cf5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd24 expref: 16 pid: 5845 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.800688:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368736cf5 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd24 expref: 16 pid: 5845 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.800704:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619933.800709:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1e00/0x1cd7fe5368736d65 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd86 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.800721:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1e00/0x1cd7fe5368736d65 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd86 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.800730:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1e00/0x1cd7fe5368736d65 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd86 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.800788:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1e00/0x1cd7fe5368736d65 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd86 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619933.800803:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c74aad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589270400:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619933.800881:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebc00 going to sleep for 11 seconds 00000100:00100000:0.0:1788619933.809195:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619933.809360:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571745280 00000100:00100000:2.0:1788619933.809369:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571745280 00000100:00100000:2.0:1788619933.809424:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1323 00000100:00100000:2.0:1788619933.809434:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63f8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:14908:x1875503571745280:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619933.809453:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619933.809471:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000043e62d96/0x1cd7fe5368736d6c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dd8d expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.809515:0:6698:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000dd8a44c0/0x1cd7fe5368736d34 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1806336->1818623) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd5c expref: 7 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.809526:0:6698:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000d4b76022/0x1cd7fe5368736d49 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd71 expref: 5 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.809534:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(9 11) 00010000:00010000:2.0:1788619933.809538:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(9 11) 00010000:00010000:2.0:1788619933.809544:0:6698:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000043e62d96/0x1cd7fe5368736d6c lrc: 5/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd8d expref: 7 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.809576:0:6698:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000d4b76022/0x1cd7fe5368736d49 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd71 expref: 5 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.809590:0:6698:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000d4b76022/0x1cd7fe5368736d49 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd71 expref: 5 pid: 7654 timeout: 526 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619933.809603:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64ae680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589270528:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619933.809699:0:6698:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000dd8a44c0/0x1cd7fe5368736d34 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1806336->1818623) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd5c expref: 7 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.809714:0:6698:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000dd8a44c0/0x1cd7fe5368736d34 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1806336->1818623) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd5c expref: 7 pid: 6696 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619933.809723:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64aca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589270656:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619933.809744:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81649c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619933.809945:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c74aad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589270400:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619933.809964:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619933.809989:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b3100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571745152:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 9378us (9592us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619933.809999:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 534 00000100:00100000:2.0:1788619933.814276:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64ae680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589270528:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619933.814291:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64aca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589270656:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619933.814357:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000043e62d96/0x1cd7fe5368736d6c lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd8d expref: 7 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.814381:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed0400, rc 0) 00000100:00100000:2.0:1788619933.814444:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63f8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:14908:x1875503571745280:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5011us (5258us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619933.814454:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1323 00000100:00100000:0.0:1788619933.816009:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619933.816095:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619933.816235:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571745408 00000100:00100000:1.0:1788619933.816245:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571745408 00010000:00010000:1.0:1788619933.816302:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000d4b76022/0x1cd7fe5368736d49 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd71 expref: 6 pid: 7654 timeout: 526 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619933.816390:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571745536 00000100:00100000:1.0:1788619933.816394:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571745536 00010000:00010000:1.0:1788619933.816408:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000dd8a44c0/0x1cd7fe5368736d34 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1806336->1818623) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd5c expref: 7 pid: 6696 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619933.816552:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 535 00000100:00100000:3.0:1788619933.816580:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63fa680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571745408:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619933.816598:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619933.816628:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d4b76022/0x1cd7fe5368736d49 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd71 expref: 6 pid: 7654 timeout: 526 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.816643:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d4b76022/0x1cd7fe5368736d49 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd71 expref: 6 pid: 7654 timeout: 526 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.816669:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d4b76022/0x1cd7fe5368736d49 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd71 expref: 6 pid: 7654 timeout: 526 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.816688:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619933.816749:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63fa680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571745408:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 185us (747us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619933.816757:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 535 00000100:00100000:3.0:1788619933.816775:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 536 00000100:00100000:3.0:1788619933.816782:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcaf850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571745536:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619933.816791:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619933.816800:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000dd8a44c0/0x1cd7fe5368736d34 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1806336->1818623) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd5c expref: 7 pid: 6696 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.816811:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000dd8a44c0/0x1cd7fe5368736d34 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1806336->1818623) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd5c expref: 7 pid: 6696 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.816824:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dd8a44c0/0x1cd7fe5368736d34 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1806336->1818623) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd5c expref: 7 pid: 6696 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.816835:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(9 11) 00010000:00010000:3.0:1788619933.816840:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(9 11) 00010000:00010000:3.0:1788619933.816850:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000043e62d96/0x1cd7fe5368736d6c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd8d expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.816860:0:13330:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000043e62d96/0x1cd7fe5368736d6c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd8d expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.816897:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000043e62d96/0x1cd7fe5368736d6c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd8d expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619933.816929:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c74aad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589270784:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619933.816980:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebc00 going to sleep for 11 seconds 00000100:00100000:3.0:1788619933.819928:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c74aad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589270784:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619933.819954:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619933.820054:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcaf850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571745536:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3271us (3957us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619933.820066:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 536 00000100:00100000:2.0:1788619933.823587:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619933.823696:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571745664 00000100:00100000:3.0:1788619933.823706:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571745664 00002000:00010000:3.0:1788619933.823751:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1666024->18446744073709551615) 00002000:00100000:3.0:1788619933.823814:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64aca80 with x1875503571745664 ext(1666024->18446744073709551615) 00002000:00010000:3.0:1788619933.823829:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64aca80. 00000100:00100000:3.0:1788619933.823899:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 800 00000100:00100000:3.0:1788619933.823911:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64aca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571745664:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:3.0:1788619933.824265:0:6702:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1853533 -> 1666024 00002000:00010000:3.0:1788619933.824269:0:6702:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619933 -> 1788619933 00002000:00010000:3.0:1788619933.824271:0:6702:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619933 -> 1788619933 00002000:00010000:3.0:1788619933.824273:0:6702:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619933 -> 1788619933 00002000:00010000:3.0:1788619933.824276:0:6702:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 391 -> 391 00000100:00100000:3.0:1788619933.824528:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64aca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571745664:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 617us (949us total) trans 4294967527 rc 0/0 00000100:00100000:3.0:1788619933.824537:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 800 00002000:00010000:3.0:1788619933.824541:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1666024->18446744073709551615) 00002000:00100000:3.0:1788619933.824545:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64aca80 with x1875503571745664 ext(1666024->18446744073709551615) 00002000:00010000:3.0:1788619933.824550:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64aca80. 00000100:00100000:0.0:1788619933.827546:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619933.827659:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571745792 00000100:00100000:1.0:1788619933.827669:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571745792 00000100:00100000:1.0:1788619933.827786:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1610 00000100:00100000:1.0:1788619933.827798:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63f8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+15:14908:x1875503571745792:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619933.827820:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619933.827842:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368736d73 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dd94 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.827943:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368736d7a lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.827959:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368736d7a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.827970:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368736d7a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788619933.828000:0:9417:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1666024 00000004:00200000:1.0:1788619933.828004:0:9417:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788619933.828023:0:9417:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368736d7a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.828049:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368736d73 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd94 expref: 17 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.828067:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368736d7a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd94 expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.828075:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619933.828077:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788619933.828079:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c000, rc 0) 00000100:00100000:1.0:1788619933.828132:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63f8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+16:14908:x1875503571745792:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 338us (601us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619933.828137:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1610 00000100:00100000:0.0:1788619933.831124:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619933.831179:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571745920 00000100:00100000:2.0:1788619933.831187:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571745920 00000100:00100000:2.0:1788619933.831233:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1324 00000100:00100000:2.0:1788619933.831244:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:14908:x1875503571745920:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619933.831262:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619933.831280:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c5c78e09/0x1cd7fe5368736d81 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dd9b expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.831320:0:7654:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000043e62d96/0x1cd7fe5368736d6c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd8d expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.831522:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(9 11) 00010000:00010000:2.0:1788619933.831527:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(10 11) 00010000:00010000:2.0:1788619933.831535:0:7654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c5c78e09/0x1cd7fe5368736d81 lrc: 4/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1433600->1478655] (req 1433600->1478655) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd9b expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.831577:0:7654:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000043e62d96/0x1cd7fe5368736d6c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd8d expref: 6 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.831592:0:7654:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000043e62d96/0x1cd7fe5368736d6c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd8d expref: 6 pid: 6698 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619933.831606:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64ae680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589270912:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619933.831703:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81649c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619933.837755:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619933.838018:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571746048 00000100:00100000:0.0:1788619933.838030:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571746048 00010000:00010000:0.0:1788619933.838073:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000043e62d96/0x1cd7fe5368736d6c lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd8d expref: 7 pid: 6698 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619933.838112:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 537 00000100:00100000:0.0:1788619933.838119:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74aad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571746048:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619933.838131:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619933.838150:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000043e62d96/0x1cd7fe5368736d6c lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd8d expref: 7 pid: 6698 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.838161:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000043e62d96/0x1cd7fe5368736d6c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd8d expref: 7 pid: 6698 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.838187:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(10 11) 00010000:00010000:0.0:1788619933.838191:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(10 11) 00010000:00010000:0.0:1788619933.838201:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000c5c78e09/0x1cd7fe5368736d81 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1433600->1478655) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd9b expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.838218:0:5831:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c5c78e09/0x1cd7fe5368736d81 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1433600->1478655) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd9b expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.838258:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000c5c78e09/0x1cd7fe5368736d81 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1433600->1478655) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd9b expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619933.838270:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be4700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5831:1875503589271040:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619933.838580:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000043e62d96/0x1cd7fe5368736d6c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd8d expref: 7 pid: 6698 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619933.838596:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64ae680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589270912:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619933.838604:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5eafe40 going to sleep for 11 seconds 00010000:00010000:2.0:1788619933.838623:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c5c78e09/0x1cd7fe5368736d81 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1433600->1478655) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd9b expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.838639:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed1400, rc 0) 00000100:00100000:2.0:1788619933.838666:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+6:14908:x1875503571745920:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7424us (7551us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619933.838672:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1324 00000100:00100000:1.0:1788619933.841554:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be4700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5831:1875503589271040:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619933.841583:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619933.841701:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74aad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571746048:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3579us (3956us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619933.841713:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 537 00000100:00100000:3.0:1788619933.843302:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619933.843645:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571746176 00000100:00100000:1.0:1788619933.843656:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571746176 00002000:00100000:1.0:1788619933.843703:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1433600->1437695) req@ffff8e77c74ab100 x1875503571746176/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619933.843714:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c74ab100 with x1875503571746176 ext(1433600->1437695) 00002000:00010000:1.0:1788619933.843719:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c74ab100 00000100:00100000:1.0:1788619933.843795:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 801 00000100:00100000:1.0:1788619933.843805:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74ab100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571746176:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619933.844121:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74ab100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571746176:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 317us (825us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619933.844129:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 801 00002000:00100000:1.0:1788619933.844136:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1433600->1437695) req@ffff8e77c74ab100 x1875503571746176/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/4536 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619933.844145:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c74ab100 with x1875503571746176 ext(1433600->1437695) 00002000:00010000:1.0:1788619933.844148:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c74ab100 00000100:00100000:0.0:1788619933.848729:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619933.848849:0:13347:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571746304 00000100:00100000:1.0:1788619933.848860:0:13347:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571746304 00002000:00100000:1.0:1788619933.848909:0:13347:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x4:0x0] (1437696->1441791) req@ffff8e77c4be4700 x1875503571746304/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619933.848921:0:13347:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4be4700 with x1875503571746304 ext(1437696->1441791) 00002000:00010000:1.0:1788619933.848927:0:13347:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4be4700 00000100:00100000:1.0:1788619933.848958:0:13347:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 802 00000100:00100000:1.0:1788619933.848967:0:13347:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571746304:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619933.849216:0:13347:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571746304:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 248us (495us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619933.849225:0:13347:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 802 00002000:00100000:1.0:1788619933.849232:0:13347:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x4:0x0] (1437696->1441791) req@ffff8e77c4be4700 x1875503571746304/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/4536 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619933.849241:0:13347:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c4be4700 with x1875503571746304 ext(1437696->1441791) 00002000:00010000:1.0:1788619933.849244:0:13347:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c4be4700 00000100:00100000:0.0:1788619933.852548:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619933.852644:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571746432 00000100:00100000:2.0:1788619933.852653:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571746432 00002000:00100000:2.0:1788619933.852703:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1441792->1445887) req@ffff8e78cdcc1180 x1875503571746432/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619933.852713:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78cdcc1180 with x1875503571746432 ext(1441792->1445887) 00002000:00010000:2.0:1788619933.852718:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78cdcc1180 00000100:00100000:2.0:1788619933.852751:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 803 00000100:00100000:2.0:1788619933.852769:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571746432:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619933.853027:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571746432:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 265us (488us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619933.853035:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 803 00002000:00100000:2.0:1788619933.853042:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1441792->1445887) req@ffff8e78cdcc1180 x1875503571746432/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/4536 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619933.853050:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78cdcc1180 with x1875503571746432 ext(1441792->1445887) 00002000:00010000:2.0:1788619933.853053:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78cdcc1180 00000100:00100000:2.0:1788619933.856420:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619933.856571:0:13347:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571746560 00000100:00100000:1.0:1788619933.856583:0:13347:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571746560 00002000:00100000:1.0:1788619933.856638:0:13347:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x4:0x0] (1445888->1449983) req@ffff8e77c64ae680 x1875503571746560/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619933.856650:0:13347:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64ae680 with x1875503571746560 ext(1445888->1449983) 00002000:00010000:1.0:1788619933.856656:0:13347:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64ae680 00000100:00100000:1.0:1788619933.856691:0:13347:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 804 00000100:00100000:1.0:1788619933.856701:0:13347:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64ae680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571746560:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619933.857057:0:13347:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64ae680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571746560:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 356us (647us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619933.857065:0:13347:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 804 00002000:00100000:1.0:1788619933.857073:0:13347:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x4:0x0] (1445888->1449983) req@ffff8e77c64ae680 x1875503571746560/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/4536 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619933.857081:0:13347:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64ae680 with x1875503571746560 ext(1445888->1449983) 00002000:00010000:1.0:1788619933.857084:0:13347:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64ae680 00000100:00100000:3.0:1788619933.860423:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619933.860531:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571746688 00000100:00100000:2.0:1788619933.860542:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571746688 00002000:00100000:2.0:1788619933.860587:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1449984->1454079) req@ffff8e77c74a9180 x1875503571746688/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619933.860600:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c74a9180 with x1875503571746688 ext(1449984->1454079) 00002000:00010000:2.0:1788619933.860606:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c74a9180 00000100:00100000:2.0:1788619933.860644:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 805 00000100:00100000:2.0:1788619933.860654:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74a9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571746688:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619933.860934:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74a9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571746688:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 281us (521us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619933.860942:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 805 00002000:00100000:2.0:1788619933.860948:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1449984->1454079) req@ffff8e77c74a9180 x1875503571746688/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/4536 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619933.860956:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c74a9180 with x1875503571746688 ext(1449984->1454079) 00002000:00010000:2.0:1788619933.860960:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c74a9180 00000100:00100000:0.0:1788619933.865746:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619933.866004:0:13347:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571746816 00000100:00100000:1.0:1788619933.866015:0:13347:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571746816 00002000:00100000:1.0:1788619933.866074:0:13347:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x4:0x0] (1454080->1458175) req@ffff8e78cdcc3b80 x1875503571746816/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619933.866091:0:13347:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78cdcc3b80 with x1875503571746816 ext(1454080->1458175) 00002000:00010000:1.0:1788619933.866098:0:13347:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78cdcc3b80 00000100:00100000:1.0:1788619933.866135:0:13347:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 806 00000100:00100000:1.0:1788619933.866146:0:13347:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571746816:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619933.866452:0:13347:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571746816:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 307us (714us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619933.866460:0:13347:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 806 00002000:00100000:1.0:1788619933.866468:0:13347:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x4:0x0] (1454080->1458175) req@ffff8e78cdcc3b80 x1875503571746816/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/4536 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619933.866476:0:13347:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78cdcc3b80 with x1875503571746816 ext(1454080->1458175) 00002000:00010000:1.0:1788619933.866481:0:13347:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78cdcc3b80 00000100:00100000:0.0:1788619933.869485:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619933.869816:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571746944 00000100:00100000:2.0:1788619933.869827:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571746944 00002000:00100000:2.0:1788619933.869879:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1458176->1462271) req@ffff8e78cdcc2680 x1875503571746944/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619933.869891:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78cdcc2680 with x1875503571746944 ext(1458176->1462271) 00002000:00010000:2.0:1788619933.869897:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78cdcc2680 00000100:00100000:2.0:1788619933.869932:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 807 00000100:00100000:2.0:1788619933.869941:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571746944:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619933.870196:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571746944:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 254us (719us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619933.870203:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 807 00002000:00100000:2.0:1788619933.870211:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1458176->1462271) req@ffff8e78cdcc2680 x1875503571746944/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/4536 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619933.870218:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78cdcc2680 with x1875503571746944 ext(1458176->1462271) 00002000:00010000:2.0:1788619933.870221:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78cdcc2680 00000100:00100000:0.0:1788619933.872721:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619933.872989:0:13347:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571747072 00000100:00100000:1.0:1788619933.872999:0:13347:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571747072 00002000:00100000:1.0:1788619933.873044:0:13347:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x4:0x0] (1462272->1466367) req@ffff8e78cdcc0e00 x1875503571747072/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619933.873054:0:13347:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78cdcc0e00 with x1875503571747072 ext(1462272->1466367) 00002000:00010000:1.0:1788619933.873058:0:13347:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78cdcc0e00 00000100:00100000:1.0:1788619933.873092:0:13347:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 808 00000100:00100000:1.0:1788619933.873101:0:13347:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571747072:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619933.873360:0:13347:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571747072:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 260us (648us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1788619933.873368:0:13347:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 808 00002000:00100000:1.0:1788619933.873375:0:13347:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x4:0x0] (1462272->1466367) req@ffff8e78cdcc0e00 x1875503571747072/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/4536 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619933.873383:0:13347:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78cdcc0e00 with x1875503571747072 ext(1462272->1466367) 00002000:00010000:1.0:1788619933.873387:0:13347:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78cdcc0e00 00000100:00100000:0.0:1788619933.876835:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619933.877060:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571747200 00000100:00100000:2.0:1788619933.877070:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571747200 00002000:00100000:2.0:1788619933.877120:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1466368->1470463) req@ffff8e78cdcc3800 x1875503571747200/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619933.877130:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78cdcc3800 with x1875503571747200 ext(1466368->1470463) 00002000:00010000:2.0:1788619933.877136:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78cdcc3800 00000100:00100000:2.0:1788619933.877173:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 809 00000100:00100000:2.0:1788619933.877183:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571747200:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619933.877475:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571747200:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 292us (658us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619933.877483:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 809 00002000:00100000:2.0:1788619933.877501:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1466368->1470463) req@ffff8e78cdcc3800 x1875503571747200/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/4536 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619933.877509:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78cdcc3800 with x1875503571747200 ext(1466368->1470463) 00002000:00010000:2.0:1788619933.877512:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78cdcc3800 00000100:00100000:0.0:1788619933.880280:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619933.880391:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571747328 00000100:00100000:3.0:1788619933.880402:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571747328 00002000:00100000:3.0:1788619933.880453:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1470464->1474559) req@ffff8e78cdcc1880 x1875503571747328/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619933.880465:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78cdcc1880 with x1875503571747328 ext(1470464->1474559) 00002000:00010000:3.0:1788619933.880471:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78cdcc1880 00000100:00100000:3.0:1788619933.880513:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 810 00000100:00100000:3.0:1788619933.880522:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78cdcc1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571747328:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619933.880792:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78cdcc1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571747328:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 270us (518us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619933.880801:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 810 00002000:00100000:3.0:1788619933.880808:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1470464->1474559) req@ffff8e78cdcc1880 x1875503571747328/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/4536 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619933.880815:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78cdcc1880 with x1875503571747328 ext(1470464->1474559) 00002000:00010000:3.0:1788619933.880818:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78cdcc1880 00000100:00100000:0.0:1788619933.884823:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619933.884954:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571747456 00000100:00100000:2.0:1788619933.884964:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571747456 00002000:00100000:2.0:1788619933.885008:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1474560->1478655) req@ffff8e78c4a4f800 x1875503571747456/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619933.885020:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c4a4f800 with x1875503571747456 ext(1474560->1478655) 00002000:00010000:2.0:1788619933.885026:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c4a4f800 00000100:00100000:2.0:1788619933.885058:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 811 00000100:00100000:2.0:1788619933.885067:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571747456:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619933.885368:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571747456:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 300us (553us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619933.885376:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 811 00002000:00100000:2.0:1788619933.885383:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1474560->1478655) req@ffff8e78c4a4f800 x1875503571747456/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/4536 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619933.885391:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78c4a4f800 with x1875503571747456 ext(1474560->1478655) 00002000:00010000:2.0:1788619933.885395:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78c4a4f800 00000100:00100000:0.0:1788619933.888542:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619933.888649:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571747584 00000100:00100000:2.0:1788619933.888659:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571747584 00000100:00100000:2.0:1788619933.888721:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1611 00000100:00100000:2.0:1788619933.888732:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:14908:x1875503571747584:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619933.888855:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619933.888879:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe5368736d88 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dda2 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.888963:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe5368736d8f lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.888976:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe5368736d8f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.888985:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe5368736d8f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788619933.889011:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1666024 00000004:00200000:2.0:1788619933.889014:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788619933.889027:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe5368736d8f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.889046:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe5368736d88 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dda2 expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.889060:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe5368736d8f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dda2 expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.889065:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619933.889067:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788619933.889070:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed1c00, rc 0) 00000100:00100000:2.0:1788619933.889152:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:14908:x1875503571747584:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 424us (622us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619933.889157:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1611 00000100:00100000:0.0:1788619933.891532:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619933.891575:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571747712 00000100:00100000:2.0:1788619933.891586:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571747712 00000100:00100000:2.0:1788619933.891633:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1612 00000100:00100000:2.0:1788619933.891640:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571747712:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619933.891655:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619933.891669:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0800/0x1cd7fe5368736d96 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dda9 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.891701:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1e00/0x1cd7fe5368736d65 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd86 expref: 15 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619933.891745:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64ad500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589271168:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619933.891831:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164d80 going to sleep for 11 seconds 00000004:00010000:2.0:1788619933.896575:0:5844:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619932 -> 1788619933 00000100:00100000:2.0:1788619933.896610:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64ad500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589271168:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619933.896634:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 386252 00010000:00010000:2.0:1788619933.896654:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0800/0x1cd7fe5368736d96 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dda9 expref: 17 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.896676:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619933.896680:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619933.896687:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0800/0x1cd7fe5368736d96 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dda9 expref: 17 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.896702:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed0800, rc 0) 00000100:00100000:2.0:1788619933.896751:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571747712:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5111us (5229us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619933.896771:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1612 00000100:00100000:0.0:1788619933.901966:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619933.902120:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571747840 00000100:00100000:1.0:1788619933.902129:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571747840 00000100:00100000:1.0:1788619933.902580:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1613 00000100:00100000:1.0:1788619933.902594:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571747840:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619933.902614:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619933.902635:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368736d9d lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ddb0 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.902672:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1e00/0x1cd7fe5368736d65 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd86 expref: 15 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619933.902712:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df8b1880 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589271296:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619933.902809:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc600 going to sleep for 11 seconds 00000100:00100000:1.0:1788619933.906238:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df8b1880 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589271296:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619933.906627:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 386252 00010000:00010000:1.0:1788619933.906648:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368736d9d lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ddb0 expref: 17 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.906670:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619933.906673:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619933.906678:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368736d9d lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ddb0 expref: 17 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.906697:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d600, rc 0) 00000100:00100000:1.0:1788619933.906802:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571747840:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4210us (4853us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619933.906811:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1613 00000100:00100000:0.0:1788619933.912566:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619933.912678:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571747968 00000100:00100000:2.0:1788619933.912688:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571747968 00000100:00100000:2.0:1788619933.912790:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1614 00000100:00100000:2.0:1788619933.912802:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c4a4df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2352:x1875503571747968:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619933.912823:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619933.912846:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0800/0x1cd7fe5368736da4 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ddb7 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.912895:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1e00/0x1cd7fe5368736d65 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd86 expref: 15 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619933.912950:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64ad500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589271424:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619933.913043:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164d80 going to sleep for 11 seconds 00000100:00100000:2.0:1788619933.916132:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64ad500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589271424:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619933.916173:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 386252 00010000:00010000:2.0:1788619933.916194:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0800/0x1cd7fe5368736da4 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ddb7 expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.916215:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619933.916218:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619933.916223:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0800/0x1cd7fe5368736da4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ddb7 expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.916238:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed0800, rc 0) 00000100:00100000:2.0:1788619933.916390:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c4a4df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2352:x1875503571747968:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3590us (3833us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619933.916399:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1614 00000100:00100000:0.0:1788619933.922529:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619933.922608:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571748096 00000100:00100000:2.0:1788619933.922618:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571748096 00000100:00100000:2.0:1788619933.922699:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1615 00000100:00100000:2.0:1788619933.922710:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79008ac000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2353:x1875503571748096:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619933.922729:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619933.922748:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe5368736dab lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ddbe expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.922795:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1e00/0x1cd7fe5368736d65 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd86 expref: 15 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619933.922833:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64ad500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589271552:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619933.922919:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164d80 going to sleep for 11 seconds 00000100:00100000:2.0:1788619933.925272:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64ad500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589271552:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619933.925298:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 386252 00010000:00010000:2.0:1788619933.925317:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe5368736dab lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ddbe expref: 17 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.925381:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619933.925384:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619933.925391:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe5368736dab lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ddbe expref: 17 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.925406:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed0400, rc 0) 00000100:00100000:2.0:1788619933.925458:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79008ac000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2353:x1875503571748096:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2750us (2937us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619933.925467:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1615 00000100:00100000:1.0:1788619933.928948:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619933.929099:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571748224 00000100:00100000:3.0:1788619933.929108:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571748224 00000100:00100000:3.0:1788619933.929170:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1325 00000100:00100000:3.0:1788619933.929181:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:14908:x1875503571748224:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619933.929200:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619933.929219:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca5ff342/0x1cd7fe5368736db2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ddc5 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.929252:0:6696:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000c5c78e09/0x1cd7fe5368736d81 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1433600->1478655) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd9b expref: 7 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.929261:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(10 11) 00010000:00010000:3.0:1788619933.929266:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(10 11) 00010000:00010000:3.0:1788619933.929272:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca5ff342/0x1cd7fe5368736db2 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1376256->1433599] (req 1376256->1433599) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ddc5 expref: 7 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.929306:0:6696:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000c5c78e09/0x1cd7fe5368736d81 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1433600->1478655) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd9b expref: 7 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.929366:0:6696:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000c5c78e09/0x1cd7fe5368736d81 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1433600->1478655) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd9b expref: 7 pid: 7654 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619933.929382:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c74abb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589271680:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619933.929456:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb000 going to sleep for 11 seconds 00000100:00100000:2.0:1788619933.932449:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c74abb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589271680:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619933.932499:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ca5ff342/0x1cd7fe5368736db2 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1376256->1433599] (req 1376256->1433599) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ddc5 expref: 7 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.932521:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78ccca7000, rc 0) 00000100:00100000:2.0:1788619933.932579:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:14908:x1875503571748224:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3399us (3641us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619933.932589:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1325 00000100:00100000:1.0:1788619933.936678:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619933.937001:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571748352 00000100:00100000:0.0:1788619933.937011:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571748352 00010000:00010000:0.0:1788619933.937056:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000c5c78e09/0x1cd7fe5368736d81 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1433600->1478655) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd9b expref: 7 pid: 7654 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619933.937106:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 538 00000100:00100000:0.0:1788619933.937116:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b0700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571748352:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619933.937130:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619933.937153:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000c5c78e09/0x1cd7fe5368736d81 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1433600->1478655) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd9b expref: 7 pid: 7654 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.937168:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000c5c78e09/0x1cd7fe5368736d81 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1433600->1478655) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd9b expref: 7 pid: 7654 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.937192:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c5c78e09/0x1cd7fe5368736d81 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1433600->1478655) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd9b expref: 7 pid: 7654 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.937208:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(10 11) 00010000:00010000:0.0:1788619933.937213:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(10 11) 00010000:00010000:0.0:1788619933.937221:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000ca5ff342/0x1cd7fe5368736db2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1376256->1433599) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ddc5 expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.937232:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca5ff342/0x1cd7fe5368736db2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1376256->1433599) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ddc5 expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619933.937270:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000ca5ff342/0x1cd7fe5368736db2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1376256->1433599) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ddc5 expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619933.937285:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79008ad500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589271808:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619933.937804:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5eafa80 going to sleep for 11 seconds 00000100:00100000:2.0:1788619933.943207:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79008ad500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589271808:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619933.943237:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619933.943304:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b0700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571748352:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 6189us (6635us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619933.943674:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 538 00000100:00100000:0.0:1788619933.953700:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619933.953897:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571748480 00000100:00100000:3.0:1788619933.953922:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571748480 00002000:00100000:3.0:1788619933.953990:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1376256->1380351) req@ffff8e79008ad500 x1875503571748480/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619933.954003:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e79008ad500 with x1875503571748480 ext(1376256->1380351) 00002000:00010000:3.0:1788619933.954010:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e79008ad500 00000100:00100000:3.0:1788619933.954045:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 812 00000100:00100000:3.0:1788619933.954056:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79008ad500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571748480:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619933.954382:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79008ad500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571748480:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 327us (691us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1788619933.954390:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 812 00002000:00100000:3.0:1788619933.954397:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1376256->1380351) req@ffff8e79008ad500 x1875503571748480/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/4536 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619933.954405:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e79008ad500 with x1875503571748480 ext(1376256->1380351) 00002000:00010000:3.0:1788619933.954409:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e79008ad500 00000100:00100000:1.0:1788619933.959907:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619933.960166:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571748608 00000100:00100000:2.0:1788619933.960176:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571748608 00002000:00100000:2.0:1788619933.960226:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1429504->1433599) req@ffff8e78df8b0380 x1875503571748608/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619933.960237:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df8b0380 with x1875503571748608 ext(1429504->1433599) 00002000:00010000:2.0:1788619933.960243:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df8b0380 00000100:00100000:2.0:1788619933.960547:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 813 00000100:00100000:2.0:1788619933.960559:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571748608:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619933.960834:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571748608:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 276us (936us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1788619933.960842:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 813 00002000:00100000:2.0:1788619933.960850:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1429504->1433599) req@ffff8e78df8b0380 x1875503571748608/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/4536 e 0 to 0 dl 1788619944 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619933.960859:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df8b0380 with x1875503571748608 ext(1429504->1433599) 00002000:00010000:2.0:1788619933.960863:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df8b0380 00000100:00100000:0.0:1788619933.965422:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619933.965535:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571748736 00000100:00100000:1.0:1788619933.965545:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571748736 00000100:00100000:1.0:1788619933.965610:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1616 00000100:00100000:1.0:1788619933.965622:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2353:x1875503571748736:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619933.965644:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619933.965667:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368736db9 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ddcc expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.965706:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1e00/0x1cd7fe5368736d65 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd86 expref: 15 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619933.965746:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df8b2300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589271936:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619933.965830:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc600 going to sleep for 11 seconds 00000100:00100000:1.0:1788619933.967800:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df8b2300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589271936:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619933.968054:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 386252 00010000:00010000:1.0:1788619933.968070:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368736db9 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ddcc expref: 17 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.968090:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619933.968092:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619933.968096:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368736db9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ddcc expref: 17 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619933.968106:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c200, rc 0) 00000100:00100000:1.0:1788619933.968153:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2353:x1875503571748736:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2536us (2743us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619933.968160:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1616 00000100:00100000:3.0:1788619933.970224:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619933.970837:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571748864 00000100:00100000:2.0:1788619933.970846:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571748864 00000100:00100000:2.0:1788619933.970942:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1617 00000100:00100000:2.0:1788619933.970954:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74ab480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2352:x1875503571748864:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619933.970974:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619933.970993:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe5368736dc0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ddd3 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.971028:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1e00/0x1cd7fe5368736d65 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd86 expref: 15 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619933.971060:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64ad500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589272064:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619933.971127:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164480 going to sleep for 11 seconds 00000100:00100000:3.0:1788619933.973056:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64ad500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589272064:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619933.973078:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 386252 00010000:00010000:3.0:1788619933.973093:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe5368736dc0 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ddd3 expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.973109:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619933.973111:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619933.973115:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe5368736dc0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ddd3 expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.973125:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed0400, rc 0) 00000100:00100000:3.0:1788619933.973241:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74ab480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2352:x1875503571748864:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2288us (3024us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619933.973248:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1617 00000100:00100000:1.0:1788619933.976302:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619933.976525:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571749120 00000100:00100000:2.0:1788619933.976536:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571749120 00000100:00100000:2.0:1788619933.976602:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1618 00000100:00100000:2.0:1788619933.976612:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+16:14908:x1875503571749120:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:2.0:1788619933.976662:0:5844:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619933.976703:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368736d7a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd94 expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.976721:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619933.976727:0:5844:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788619933.976929:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0800/0x1cd7fe5368736dc7 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.976945:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe5368736d8f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dda2 expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.976953:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0800/0x1cd7fe5368736dc7 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.976986:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe5368736d8f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dda2 expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.976999:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe5368736d8f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dda2 expref: 15 pid: 5845 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619933.977011:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64ac700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589272192:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619933.977091:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81646c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619933.980308:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619933.980769:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64ac700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589272192:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619933.980795:0:5844:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0800/0x1cd7fe5368736dc7 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619933.980961:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571749248 00000100:00100000:3.0:1788619933.980968:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571749248 00010000:00010000:3.0:1788619933.981005:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe5368736d8f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dda2 expref: 16 pid: 5845 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619933.981046:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 539 00000100:00100000:3.0:1788619933.981055:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b3480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571749248:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619933.981068:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619933.981074:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe5368736d8f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dda2 expref: 16 pid: 5845 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.981087:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe5368736d8f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dda2 expref: 16 pid: 5845 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.981108:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1c00/0x1cd7fe5368736d8f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dda2 expref: 16 pid: 5845 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.981124:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619933.981129:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0800/0x1cd7fe5368736dc7 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.981139:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0800/0x1cd7fe5368736dc7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.981146:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0800/0x1cd7fe5368736dc7 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.981179:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619933.981236:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b3480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571749248:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 182us (937us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619933.981243:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 539 00010000:00010000:2.0:1788619933.982004:0:5844:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0800/0x1cd7fe5368736dc7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.982014:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0800/0x1cd7fe5368736dc7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788619933.982125:0:5844:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967860 00000004:00010000:2.0:1788619933.982182:0:5844:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 386252 -> 386252 00000004:00010000:2.0:1788619933.982186:0:5844:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619933 -> 1788619933 00000004:00010000:2.0:1788619933.982189:0:5844:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619933 -> 1788619932 00000004:00010000:2.0:1788619933.982191:0:5844:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619933 -> 1788619933 00000004:00010000:2.0:1788619933.982193:0:5844:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 774 -> 774 00000004:00010000:2.0:1788619933.982200:0:5844:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000b6501b2f reply state 00000000b9e73e1c transno 4294967861 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0800/0x1cd7fe5368736dc7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619933.982213:0:5844:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788619933.982309:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+18:14908:x1875503571749120:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 5696us (6014us total) trans 4294967861 rc 0/0 00000100:00100000:2.0:1788619933.982351:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1618 00000020:00080000:2.0:1788619933.982355:0:5844:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:2.0:1788619933.984197:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0800/0x1cd7fe5368736dc7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.984212:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0800/0x1cd7fe5368736dc7 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.984219:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0800/0x1cd7fe5368736dc7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.984225:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c5ed0800 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:2.0:1788619933.984230:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0800/0x1cd7fe5368736dc7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.984238:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0800/0x1cd7fe5368736dc7 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.984255:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619933.984258:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788619933.984263:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0800/0x1cd7fe5368736dc7 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619933.984268:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0800/0x1cd7fe5368736dc7 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619933.984943:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619933.985014:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571749376 00000100:00100000:3.0:1788619933.985024:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571749376 00000100:00100000:3.0:1788619933.985078:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1326 00000100:00100000:3.0:1788619933.985089:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:14908:x1875503571749376:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619933.985105:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619933.985121:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000989ca2b5/0x1cd7fe5368736dce lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ddda expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.985151:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000ca5ff342/0x1cd7fe5368736db2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1376256->1433599) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ddc5 expref: 5 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.985159:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(10 11) 00010000:00010000:3.0:1788619933.985175:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(11 11) 00010000:00010000:3.0:1788619933.985181:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000989ca2b5/0x1cd7fe5368736dce lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ddda expref: 7 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.985217:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000ca5ff342/0x1cd7fe5368736db2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1376256->1433599) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ddc5 expref: 5 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.985232:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000ca5ff342/0x1cd7fe5368736db2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1376256->1433599) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ddc5 expref: 5 pid: 6696 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619933.985246:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c74a9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589272320:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619933.985355:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebb40 going to sleep for 11 seconds 00000100:00100000:3.0:1788619933.987312:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c74a9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589272320:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619933.987528:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000989ca2b5/0x1cd7fe5368736dce lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ddda expref: 7 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.987555:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78ccca7c00, rc 0) 00000100:00100000:0.0:1788619933.988477:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619933.988568:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571749504 00000100:00100000:3.0:1788619933.988575:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571749504 00002000:00100000:3.0:1788619933.988619:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1376256->1433599) req@ffff8e77c69fd500 x1875503571749504/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/0 e 0 to 0 dl 1788619944 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619933.988629:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c69fd500 with x1875503571749504 ext(1376256->1433599) 00010000:00010000:3.0:1788619933.988636:0:6702:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000ca5ff342/0x1cd7fe5368736db2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1376256->1433599) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ddc5 expref: 6 pid: 6696 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.988650:0:6702:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000ca5ff342/0x1cd7fe5368736db2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1376256->1433599) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ddc5 expref: 6 pid: 6696 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788619933.988657:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff8e77c69fd500 00000100:00100000:3.0:1788619933.988689:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 28 00000100:00100000:3.0:1788619933.988699:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571749504:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:2.0:1788619933.989814:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:14908:x1875503571749376:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4719us (4867us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619933.989824:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1326 00000100:00100000:3.0:1788619933.995239:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571749504:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 6529us (6754us total) trans 4294967528 rc 0/0 00000100:00100000:3.0:1788619933.995253:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 28 00002000:00100000:3.0:1788619933.995267:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1376256->1433599) req@ffff8e77c69fd500 x1875503571749504/t4294967528(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:29/0 lens 488/448 e 0 to 0 dl 1788619944 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619933.995278:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c69fd500 with x1875503571749504 ext(1376256->1433599) 00010000:00010000:3.0:1788619933.995288:0:6702:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000ca5ff342/0x1cd7fe5368736db2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1376256->1433599) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ddc5 expref: 8 pid: 6696 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.995312:0:6702:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000ca5ff342/0x1cd7fe5368736db2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1376256->1433599) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ddc5 expref: 8 pid: 6696 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788619933.995320:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff8e77c69fd500 00000100:00100000:0.0:1788619933.997270:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619933.997348:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571749632 00000100:00100000:3.0:1788619933.997356:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571749632 00010000:00010000:3.0:1788619933.997397:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000ca5ff342/0x1cd7fe5368736db2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1376256->1433599) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ddc5 expref: 8 pid: 6696 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619933.997436:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 540 00000100:00100000:3.0:1788619933.997445:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fd180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571749632:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619933.997460:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619933.997482:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000ca5ff342/0x1cd7fe5368736db2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1376256->1433599) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ddc5 expref: 8 pid: 6696 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.997533:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000ca5ff342/0x1cd7fe5368736db2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1376256->1433599) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ddc5 expref: 8 pid: 6696 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.997558:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca5ff342/0x1cd7fe5368736db2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1376256->1433599) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ddc5 expref: 8 pid: 6696 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.997578:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(11 11) 00010000:00010000:3.0:1788619933.997582:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(11 11) 00010000:00010000:3.0:1788619933.997589:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000989ca2b5/0x1cd7fe5368736dce lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ddda expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.997597:0:13330:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000989ca2b5/0x1cd7fe5368736dce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ddda expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619933.997629:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000989ca2b5/0x1cd7fe5368736dce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ddda expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619933.997639:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c74a9c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589272448:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619933.997836:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebb40 going to sleep for 11 seconds 00000100:00100000:1.0:1788619934.000174:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619934.000246:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c74a9c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:13330:1875503589272448:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619934.000268:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619934.000349:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fd180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571749632:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 2882us (3068us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619934.000356:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 540 00000100:00100000:2.0:1788619934.001626:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571749760 00000100:00100000:2.0:1788619934.001635:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571749760 00002000:00010000:2.0:1788619934.001688:0:6701:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1261898->18446744073709551615) 00002000:00100000:2.0:1788619934.001694:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df8b1180 with x1875503571749760 ext(1261898->18446744073709551615) 00002000:00010000:2.0:1788619934.001704:0:6701:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df8b1180. 00000100:00100000:2.0:1788619934.001737:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 814 00000100:00100000:2.0:1788619934.001746:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571749760:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:2.0:1788619934.002036:0:6701:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1666024 -> 1261898 00002000:00010000:2.0:1788619934.002040:0:6701:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619933 -> 1788619933 00002000:00010000:2.0:1788619934.002043:0:6701:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619933 -> 1788619933 00002000:00010000:2.0:1788619934.002045:0:6701:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619933 -> 1788619933 00002000:00010000:2.0:1788619934.002047:0:6701:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 391 -> 391 00000100:00100000:2.0:1788619934.002095:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+8:2355:x1875503571749760:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 351us (1934us total) trans 4294967529 rc 0/0 00000100:00100000:2.0:1788619934.002101:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 814 00002000:00010000:2.0:1788619934.002104:0:6701:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1261898->18446744073709551615) 00002000:00100000:2.0:1788619934.002107:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e78df8b1180 with x1875503571749760 ext(1261898->18446744073709551615) 00002000:00010000:2.0:1788619934.002111:0:6701:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e78df8b1180. 00000100:00080000:3.0:1788619934.002172:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1788619934.004137:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619934.004199:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571749888 00000100:00100000:3.0:1788619934.004207:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571749888 00000100:00100000:3.0:1788619934.004259:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1619 00000100:00100000:3.0:1788619934.004266:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+17:14908:x1875503571749888:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619934.004284:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619934.004298:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7e00/0x1cd7fe5368736dd5 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dde1 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1788619934.004445:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:3.0:1788619934.004504:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6e00/0x1cd7fe5368736ddc lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.004528:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6e00/0x1cd7fe5368736ddc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.004541:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6e00/0x1cd7fe5368736ddc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788619934.004568:0:9417:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1261898 00000004:00200000:3.0:1788619934.004572:0:9417:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788619934.004590:0:9417:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6e00/0x1cd7fe5368736ddc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.004617:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7e00/0x1cd7fe5368736dd5 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dde1 expref: 19 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.004637:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6e00/0x1cd7fe5368736ddc lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dde1 expref: 18 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.004645:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619934.004647:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788619934.004651:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78ccca6e00, rc 0) 00000100:00100000:3.0:1788619934.004750:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+18:14908:x1875503571749888:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 483us (618us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619934.004767:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1619 00000100:00100000:0.0:1788619934.009220:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619934.009362:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571750016 00000100:00100000:3.0:1788619934.009443:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571750016 00000100:00100000:3.0:1788619934.009595:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1620 00000100:00100000:3.0:1788619934.009607:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:14908:x1875503571750016:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619934.009630:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619934.009655:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7000/0x1cd7fe5368736de3 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dde8 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.009771:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7400/0x1cd7fe5368736dea lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.009792:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7400/0x1cd7fe5368736dea lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.009807:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7400/0x1cd7fe5368736dea lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788619934.009842:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1261898 00000004:00200000:3.0:1788619934.009846:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788619934.009868:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7400/0x1cd7fe5368736dea lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.009901:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7000/0x1cd7fe5368736de3 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dde8 expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.009921:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7400/0x1cd7fe5368736dea lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dde8 expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.009930:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619934.009933:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788619934.009937:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78ccca7400, rc 0) 00000100:00100000:3.0:1788619934.009998:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:14908:x1875503571750016:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 393us (788us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619934.010005:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1620 00000100:00080000:2.0:1788619934.013114:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1788619934.013763:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619934.013840:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571750144 00000100:00100000:3.0:1788619934.013851:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571750144 00000100:00100000:3.0:1788619934.013906:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1621 00000100:00100000:3.0:1788619934.013915:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fe300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571750144:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619934.013940:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619934.013962:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6400/0x1cd7fe5368736df1 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ddef expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.013998:0:5844:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1e00/0x1cd7fe5368736d65 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd86 expref: 17 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619934.014039:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c74a9c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589272576:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619934.014134:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebb40 going to sleep for 11 seconds 00000100:00100000:0.0:1788619934.018593:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619934.018701:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571750272 00000100:00100000:2.0:1788619934.018711:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571750272 00000100:00100000:2.0:1788619934.018794:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1327 00000100:00100000:2.0:1788619934.018806:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69ff100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571750272:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619934.018835:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619934.018860:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090281211/0x1cd7fe5368736df8 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ddf6 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619934.018933:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64ac700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589272704:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619934.019023:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81646c0 going to sleep for 11 seconds 00000100:00080000:3.0:1788619934.019705:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000004:00010000:3.0:1788619934.021152:0:5844:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619932 -> 1788619933 00000100:00100000:3.0:1788619934.021183:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c74a9c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589272576:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619934.021206:0:5844:0:(mdt_io.c:1550:mdt_lvb2reply()) size 386252 00010000:00010000:3.0:1788619934.021225:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6400/0x1cd7fe5368736df1 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ddef expref: 17 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.021245:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619934.021258:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619934.021265:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6400/0x1cd7fe5368736df1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ddef expref: 17 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.021282:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78ccca6400, rc 0) 00000100:00100000:3.0:1788619934.021717:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fe300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571750144:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7801us (7969us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619934.021730:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1621 00000100:00100000:2.0:1788619934.023466:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64ac700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589272704:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619934.023513:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000090281211/0x1cd7fe5368736df8 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ddf6 expref: 8 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.023538:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090281211/0x1cd7fe5368736df8 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ddf6 expref: 8 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.023551:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed0800, rc 0) 00000100:00100000:2.0:1788619934.023604:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69ff100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571750272:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4801us (5021us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619934.023613:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1327 00000100:00100000:0.0:1788619934.034425:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619934.034465:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619934.034608:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571750400 00000100:00100000:3.0:1788619934.034620:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571750400 00000100:00100000:3.0:1788619934.034680:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1622 00000100:00100000:3.0:1788619934.034689:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69ff800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571750400:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619934.034710:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619934.034731:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6400/0x1cd7fe5368736dff lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ddfd expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.034773:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1e00/0x1cd7fe5368736d65 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd86 expref: 17 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619934.034814:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c74a9c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589272832:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619934.034909:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebb40 going to sleep for 11 seconds 00000100:00100000:2.0:1788619934.034971:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571750528 00000100:00100000:2.0:1788619934.034976:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571750528 00000100:00100000:2.0:1788619934.035015:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1328 00000100:00100000:2.0:1788619934.035023:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69ff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571750528:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619934.035032:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619934.035042:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090281211/0x1cd7fe5368736e06 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1de04 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619934.035082:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64ac700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589272960:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619934.035109:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81646c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619934.037707:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c74a9c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589272832:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619934.037734:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 386252 00010000:00010000:3.0:1788619934.037763:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6400/0x1cd7fe5368736dff lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ddfd expref: 17 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.037785:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619934.037788:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619934.037795:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6400/0x1cd7fe5368736dff lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ddfd expref: 17 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.037812:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78ccca6400, rc 0) 00000100:00100000:3.0:1788619934.037873:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69ff800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571750400:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3183us (3405us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619934.037882:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1622 00000100:00100000:2.0:1788619934.037999:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64ac700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589272960:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619934.038021:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000090281211/0x1cd7fe5368736e06 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de04 expref: 8 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.038038:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090281211/0x1cd7fe5368736e06 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de04 expref: 8 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.038048:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed0800, rc 0) 00000100:00100000:2.0:1788619934.038074:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69ff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571750528:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3052us (3658us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619934.038081:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1328 00000100:00100000:1.0:1788619934.041838:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619934.042037:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571750784 00000100:00100000:2.0:1788619934.042046:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571750784 00000100:00100000:2.0:1788619934.042146:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1623 00000100:00100000:2.0:1788619934.042156:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+18:14908:x1875503571750784:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:2.0:1788619934.042206:0:11397:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619934.042229:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6e00/0x1cd7fe5368736ddc lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dde1 expref: 18 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.042248:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619934.042252:0:11397:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788619934.043062:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368736e0d lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.043078:0:11397:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7400/0x1cd7fe5368736dea lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dde8 expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.043087:0:11397:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368736e0d lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.043117:0:11397:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7400/0x1cd7fe5368736dea lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dde8 expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.043129:0:11397:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7400/0x1cd7fe5368736dea lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dde8 expref: 15 pid: 5845 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619934.043142:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df8b3100 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589273088:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619934.043248:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619934.048753:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619934.048847:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df8b3100 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589273088:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619934.048869:0:11397:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368736e0d lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619934.048918:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571750912 00000100:00100000:3.0:1788619934.048926:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571750912 00010000:00010000:3.0:1788619934.048964:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7400/0x1cd7fe5368736dea lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dde8 expref: 16 pid: 5845 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619934.049005:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 541 00000100:00100000:3.0:1788619934.049015:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fdf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571750912:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619934.049029:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619934.049036:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7400/0x1cd7fe5368736dea lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dde8 expref: 16 pid: 5845 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.049045:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7400/0x1cd7fe5368736dea lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dde8 expref: 16 pid: 5845 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.049062:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7400/0x1cd7fe5368736dea lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dde8 expref: 16 pid: 5845 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.049075:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619934.049078:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368736e0d lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.049086:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368736e0d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.049091:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368736e0d lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.049118:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619934.049160:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fdf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571750912:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 148us (416us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619934.049167:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 541 00010000:00010000:2.0:1788619934.049221:0:11397:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368736e0d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.049229:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368736e0d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788619934.049360:0:11397:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967862 00000004:00010000:2.0:1788619934.049401:0:11397:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 386252 -> 386252 00000004:00010000:2.0:1788619934.049404:0:11397:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619933 -> 1788619933 00000004:00010000:2.0:1788619934.049407:0:11397:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619933 -> 1788619932 00000004:00010000:2.0:1788619934.049410:0:11397:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619933 -> 1788619933 00000004:00010000:2.0:1788619934.049412:0:11397:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 774 -> 774 00000004:00010000:2.0:1788619934.049419:0:11397:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000a4d849ba reply state 0000000031e67e01 transno 4294967863 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368736e0d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619934.049428:0:11397:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1788619934.049510:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+20:14908:x1875503571750784:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 7355us (7680us total) trans 4294967863 rc 0/0 00000100:00100000:2.0:1788619934.049518:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1623 00000020:00080000:2.0:1788619934.049521:0:11397:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:1.0:1788619934.055299:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368736e0d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.055357:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368736e0d lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.055367:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368736e0d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.055373:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c683d200 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:1.0:1788619934.055377:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368736e0d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.055385:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368736e0d lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.055403:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619934.055406:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788619934.055411:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368736e0d lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.055418:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe5368736e0d lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619934.063906:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619934.064254:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571751040 00000100:00100000:1.0:1788619934.064264:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571751040 00002000:00010000:1.0:1788619934.064307:0:5859:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:1.0:1788619934.064312:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69fc380 with x1875503571751040 00000100:00100000:1.0:1788619934.064369:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1008 00000100:00100000:1.0:1788619934.064378:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+19:2354:x1875503571751040:12345-192.168.201.5@tcp:10:fsx.0 00000004:00010000:1.0:1788619934.064685:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 386252 -> 364890 00000004:00010000:1.0:1788619934.064689:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619933 -> 1788619933 00000004:00010000:1.0:1788619934.064692:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619932 -> 1788619932 00000004:00010000:1.0:1788619934.064694:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619933 -> 1788619933 00000004:00010000:1.0:1788619934.064696:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 774 -> 774 00000100:00100000:1.0:1788619934.064768:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+20:2354:x1875503571751040:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 379us (859us total) trans 4294967864 rc 0/0 00000100:00100000:1.0:1788619934.064776:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1008 00002000:00010000:1.0:1788619934.064780:0:5859:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:1.0:1788619934.064782:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69fc380 with x1875503571751040 00000100:00100000:0.0:1788619934.073100:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619934.073192:0:13347:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571751168 00000100:00100000:1.0:1788619934.073201:0:13347:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571751168 00002000:00010000:1.0:1788619934.073241:0:13347:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:1.0:1788619934.073244:0:13347:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c69fd880 with x1875503571751168 ext(0->18446744073709551615) 00002000:00010000:1.0:1788619934.073256:0:13347:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c69fd880. 00000100:00100000:1.0:1788619934.073296:0:13347:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 815 00000100:00100000:1.0:1788619934.073307:0:13347:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+8:2354:x1875503571751168:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:1.0:1788619934.073643:0:13347:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1261898 -> 0 00002000:00010000:1.0:1788619934.073647:0:13347:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619933 -> 1788619933 00002000:00010000:1.0:1788619934.073650:0:13347:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619933 -> 1788619933 00002000:00010000:1.0:1788619934.073652:0:13347:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619933 -> 1788619933 00002000:00010000:1.0:1788619934.073655:0:13347:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 391 -> 391 00000100:00100000:1.0:1788619934.073707:0:13347:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571751168:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 401us (614us total) trans 4294967530 rc 0/0 00000100:00100000:1.0:1788619934.073714:0:13347:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 815 00002000:00010000:1.0:1788619934.073718:0:13347:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:1.0:1788619934.073721:0:13347:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c69fd880 with x1875503571751168 ext(0->18446744073709551615) 00002000:00010000:1.0:1788619934.073725:0:13347:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c69fd880. 00000100:00100000:0.0:1788619934.078203:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619934.078287:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571751296 00000100:00100000:3.0:1788619934.078298:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571751296 00000100:00100000:3.0:1788619934.078383:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1624 00000100:00100000:3.0:1788619934.078391:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:14908:x1875503571751296:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619934.078424:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619934.078449:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7400/0x1cd7fe5368736e14 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1de0b expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.078554:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6400/0x1cd7fe5368736e1b lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.078576:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6400/0x1cd7fe5368736e1b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.078589:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6400/0x1cd7fe5368736e1b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788619934.078619:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 364890 00000004:00200000:3.0:1788619934.078625:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788619934.078645:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6400/0x1cd7fe5368736e1b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.078675:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7400/0x1cd7fe5368736e14 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de0b expref: 17 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.078700:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6400/0x1cd7fe5368736e1b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de0b expref: 16 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.078709:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619934.078712:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788619934.078715:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78ccca6400, rc 0) 00000100:00100000:2.0:1788619934.081540:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:14908:x1875503571751296:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3134us (3332us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619934.081554:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1624 00000100:00100000:0.0:1788619934.085250:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619934.085295:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619934.085416:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571751424 00000100:00100000:3.0:1788619934.085427:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571751424 00000100:00100000:3.0:1788619934.085473:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1625 00000100:00100000:3.0:1788619934.085481:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fe680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571751424:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619934.085508:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619934.085532:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7000/0x1cd7fe5368736e22 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1de12 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.085571:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1e00/0x1cd7fe5368736d65 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd86 expref: 19 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619934.085581:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571751552 00000100:00100000:2.0:1788619934.085586:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571751552 00000100:00100000:3.0:1788619934.085611:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c74a9c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589273216:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619934.085615:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1329 00000100:00100000:2.0:1788619934.085622:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69ffb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571751552:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619934.085629:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619934.085640:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090281211/0x1cd7fe5368736e29 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1de19 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619934.085675:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64ac700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589273344:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619934.085686:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebb40 going to sleep for 11 seconds 00000100:00100000:2.0:1788619934.085699:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164480 going to sleep for 11 seconds 00000100:00100000:2.0:1788619934.089482:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64ac700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589273344:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619934.089537:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000090281211/0x1cd7fe5368736e29 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de19 expref: 8 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.089563:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090281211/0x1cd7fe5368736e29 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de19 expref: 8 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.089576:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed0800, rc 0) 00000004:00010000:3.0:1788619934.090910:0:9417:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619932 -> 1788619933 00000100:00100000:3.0:1788619934.090935:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c74a9c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589273216:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619934.090958:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 364890 00010000:00010000:3.0:1788619934.090974:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7000/0x1cd7fe5368736e22 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de12 expref: 17 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.090995:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619934.090998:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619934.091003:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7000/0x1cd7fe5368736e22 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de12 expref: 17 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.091016:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78ccca7000, rc 0) 00000100:00100000:3.0:1788619934.091073:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fe680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571751424:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5592us (5776us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619934.091083:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1625 00000100:00100000:0.0:1788619934.095684:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619934.095774:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619934.095887:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571751808 00000100:00100000:3.0:1788619934.095898:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571751808 00000100:00100000:3.0:1788619934.097746:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1330 00000100:00100000:3.0:1788619934.097779:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f77480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571751808:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619934.097800:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619934.097820:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7278240/0x1cd7fe5368736e30 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1de27 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619934.097882:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c74a9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589273472:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619934.097983:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdebb40 going to sleep for 11 seconds 00000100:00100000:1.0:1788619934.098242:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571751680 00000100:00100000:1.0:1788619934.098250:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571751680 00000100:00100000:1.0:1788619934.098304:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1626 00000100:00100000:1.0:1788619934.098405:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f74000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2353:x1875503571751680:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619934.098417:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619934.098433:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368736e37 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1de20 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.098461:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1e00/0x1cd7fe5368736d65 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd86 expref: 19 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619934.098498:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df8b3100 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589273600:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619934.098525:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619934.100943:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df8b3100 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589273600:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619934.100969:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 364890 00010000:00010000:1.0:1788619934.100987:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368736e37 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de20 expref: 17 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.101014:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619934.101017:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619934.101023:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368736e37 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de20 expref: 17 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.101039:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683ca00, rc 0) 00000100:00100000:1.0:1788619934.101092:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f74000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2353:x1875503571751680:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2780us (5421us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619934.101102:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1626 00000100:00100000:2.0:1788619934.105359:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69ffb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2355:x1875503571751552:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 19726us (20108us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619934.105371:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1329 00000100:00100000:0.0:1788619934.105548:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c74a9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589273472:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619934.105585:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d7278240/0x1cd7fe5368736e30 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de27 expref: 8 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.105609:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7278240/0x1cd7fe5368736e30 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de27 expref: 8 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.105624:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78ccca7400, rc 0) 00000100:00100000:0.0:1788619934.105721:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f77480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2353:x1875503571751808:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7953us (9946us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619934.105728:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1330 00000100:00100000:1.0:1788619934.110828:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619934.110947:0:13346:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571751936 00000100:00100000:2.0:1788619934.110956:0:13346:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571751936 00002000:00100000:2.0:1788619934.111005:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df8b3100 x1875503571751936/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:30/0 lens 488/0 e 0 to 0 dl 1788619945 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619934.111016:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df8b3100 with x1875503571751936 00000100:00100000:2.0:1788619934.111054:0:13346:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1009 00000100:00100000:2.0:1788619934.111064:0:13346:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+20:2353:x1875503571751936:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619934.147685:0:13346:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+20:2353:x1875503571751936:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 36611us (36856us total) trans 0 rc 368640/368640 00000100:00100000:2.0:1788619934.147699:0:13346:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1009 00002000:00100000:2.0:1788619934.147713:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78df8b3100 x1875503571751936/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:30/0 lens 488/440 e 0 to 0 dl 1788619945 ref 1 fl Complete:/600/0 rc 368640/368640 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619934.147724:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78df8b3100 with x1875503571751936 00000100:00100000:3.0:1788619934.150815:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619934.150857:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619934.150886:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571752192 00000100:00100000:2.0:1788619934.150897:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571752192 00000100:00100000:2.0:1788619934.151015:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1331 00000100:00100000:2.0:1788619934.151026:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74a8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571752192:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619934.151043:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619934.151061:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090281211/0x1cd7fe5368736e3e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1de35 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619934.151073:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571752064 00000100:00100000:0.0:1788619934.151077:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571752064 00000100:00100000:0.0:1788619934.151116:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1627 00000100:00100000:2.0:1788619934.151120:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64ac700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589273728:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619934.151123:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74aa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571752064:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619934.151132:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619934.151145:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64800/0x1cd7fe5368736e45 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1de2e expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619934.151196:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81640c0 going to sleep for 11 seconds 00010000:00010000:0.0:1788619934.151295:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1e00/0x1cd7fe5368736d65 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd86 expref: 19 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619934.151321:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f75180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589273856:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619934.151503:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5eafc00 going to sleep for 11 seconds 00002000:00010000:2.0:1788619934.153758:0:6698:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619933 -> 1788619934 00000100:00100000:2.0:1788619934.153789:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64ac700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589273728:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619934.153824:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000090281211/0x1cd7fe5368736e3e lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de35 expref: 8 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.153850:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090281211/0x1cd7fe5368736e3e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de35 expref: 8 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.153864:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed0800, rc 0) 00000100:00100000:2.0:1788619934.153922:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74a8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571752192:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2897us (3117us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619934.153931:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1331 00000004:00010000:0.0:1788619934.156746:0:5845:0:(mdt_lvb.c:189:mdt_dom_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime: 1788619933 -> 1788619934 00000100:00100000:0.0:1788619934.156779:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f75180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589273856:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619934.156802:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 364890 00010000:00010000:0.0:1788619934.156821:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64800/0x1cd7fe5368736e45 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de2e expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.156841:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619934.156845:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619934.156850:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64800/0x1cd7fe5368736e45 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de2e expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.156869:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e6f64800, rc 0) 00000100:00100000:0.0:1788619934.156930:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74aa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2354:x1875503571752064:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5806us (6071us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619934.156939:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1627 00000100:00100000:1.0:1788619934.161538:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619934.161598:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619934.161675:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571752448 00000100:00100000:2.0:1788619934.161686:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571752448 00000100:00100000:2.0:1788619934.161741:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1332 00000100:00100000:2.0:1788619934.161751:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571752448:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619934.161778:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619934.161796:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000090281211/0x1cd7fe5368736e4c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1de43 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619934.161851:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64ac700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589273984:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619934.161947:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81640c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619934.162012:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571752320 00000100:00100000:3.0:1788619934.162017:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571752320 00000100:00100000:3.0:1788619934.162060:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1628 00000100:00100000:3.0:1788619934.162068:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571752320:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619934.162080:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619934.162093:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7000/0x1cd7fe5368736e53 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1de3c expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.162123:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1e00/0x1cd7fe5368736d65 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd86 expref: 19 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619934.162149:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c74ab800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589274112:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619934.162177:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cfdeb000 going to sleep for 11 seconds 00000100:00100000:1.0:1788619934.165234:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c74ab800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589274112:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619934.165291:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 364890 00010000:00010000:1.0:1788619934.165827:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7000/0x1cd7fe5368736e53 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de3c expref: 17 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.165853:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619934.165856:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619934.165863:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7000/0x1cd7fe5368736e53 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de3c expref: 17 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.165880:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78ccca7000, rc 0) 00000100:00100000:1.0:1788619934.166006:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571752320:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3937us (4406us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619934.166020:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1628 00000100:00100000:2.0:1788619934.167144:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64ac700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589273984:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619934.167187:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000090281211/0x1cd7fe5368736e4c lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de43 expref: 8 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.167210:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000090281211/0x1cd7fe5368736e4c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de43 expref: 8 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.167221:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed0800, rc 0) 00000100:00100000:2.0:1788619934.167261:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571752448:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5511us (5732us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619934.167269:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1332 00000100:00100000:1.0:1788619934.173522:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619934.173573:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619934.173653:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571752576 00000100:00100000:2.0:1788619934.173664:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571752576 00000100:00100000:2.0:1788619934.173724:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1629 00000100:00100000:2.0:1788619934.173734:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2352:x1875503571752576:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619934.173765:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619934.173786:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe5368736e5a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1de4a expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.173824:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1e00/0x1cd7fe5368736d65 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd86 expref: 19 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619934.173871:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64ac700 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589274240:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619934.174073:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81640c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619934.174182:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571752704 00000100:00100000:0.0:1788619934.174188:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571752704 00000100:00100000:0.0:1788619934.174228:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1333 00000100:00100000:0.0:1788619934.174237:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2352:x1875503571752704:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619934.174245:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619934.174257:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c69aba9f/0x1cd7fe5368736e61 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1de51 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619934.174293:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f75180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589274368:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619934.174362:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5eafc00 going to sleep for 11 seconds 00000100:00100000:0.0:1788619934.177411:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f75180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589274368:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619934.177453:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c69aba9f/0x1cd7fe5368736e61 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de51 expref: 8 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.177479:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c69aba9f/0x1cd7fe5368736e61 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de51 expref: 8 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.177503:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e6f65a00, rc 0) 00000100:00100000:0.0:1788619934.177563:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2352:x1875503571752704:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3327us (3989us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619934.177574:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1333 00000100:00100000:2.0:1788619934.181263:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64ac700 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589274240:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619934.181289:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 364890 00010000:00010000:2.0:1788619934.181317:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe5368736e5a lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de4a expref: 17 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.181374:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619934.181377:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619934.181384:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed0400/0x1cd7fe5368736e5a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de4a expref: 17 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.181406:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c5ed0400, rc 0) 00000100:00100000:2.0:1788619934.181460:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2352:x1875503571752576:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7727us (7950us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619934.181469:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1629 00000100:00100000:1.0:1788619934.186143:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619934.186245:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571752832 00000100:00100000:0.0:1788619934.186255:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571752832 00000100:00100000:0.0:1788619934.186543:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1630 00000100:00100000:0.0:1788619934.186554:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:14908:x1875503571752832:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619934.186573:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619934.186592:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65a00/0x1cd7fe5368736e68 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1de58 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.186622:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1e00/0x1cd7fe5368736d65 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd86 expref: 19 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.186631:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65a00/0x1cd7fe5368736e68 lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de58 expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.186665:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1e00/0x1cd7fe5368736d65 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd86 expref: 19 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.186676:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1e00/0x1cd7fe5368736d65 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd86 expref: 19 pid: 5844 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619934.186689:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f75180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589274496:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619934.186805:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5eafc00 going to sleep for 11 seconds 00000100:00100000:1.0:1788619934.191610:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619934.191675:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571752960 00000100:00100000:2.0:1788619934.191684:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571752960 00010000:00010000:2.0:1788619934.191727:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1e00/0x1cd7fe5368736d65 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd86 expref: 20 pid: 5844 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619934.191780:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 542 00000100:00100000:2.0:1788619934.191790:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+20:2354:x1875503571752960:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619934.191805:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619934.191830:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1e00/0x1cd7fe5368736d65 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd86 expref: 20 pid: 5844 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.191843:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1e00/0x1cd7fe5368736d65 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd86 expref: 20 pid: 5844 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.191865:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619934.191871:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65a00/0x1cd7fe5368736e68 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de58 expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.191882:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65a00/0x1cd7fe5368736e68 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de58 expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.191891:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65a00/0x1cd7fe5368736e68 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de58 expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.191944:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65a00/0x1cd7fe5368736e68 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de58 expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619934.191959:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64ac700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589274624:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619934.192040:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81640c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619934.194537:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64ac700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589274624:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619934.194566:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619934.194739:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+20:2354:x1875503571752960:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 2948us (3137us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619934.194758:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 542 00010000:00010000:0.0:1788619934.195039:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5ed1e00/0x1cd7fe5368736d65 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dd86 expref: 19 pid: 5844 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619934.195064:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f75180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589274496:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619934.195088:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65a00/0x1cd7fe5368736e68 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de58 expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.195104:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619934.195107:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619934.195111:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e6f65a00, rc 0) 00000100:00100000:0.0:1788619934.195142:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+17:14908:x1875503571752832:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 8592us (9009us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619934.195149:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1630 00000100:00100000:2.0:1788619934.204285:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619934.204625:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571753088 00000100:00100000:1.0:1788619934.204636:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571753088 00002000:00100000:1.0:1788619934.204685:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c64ac700 x1875503571753088/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:30/0 lens 488/0 e 0 to 0 dl 1788619945 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619934.204696:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64ac700 with x1875503571753088 00000100:00100000:1.0:1788619934.204771:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1010 00000100:00100000:1.0:1788619934.204781:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64ac700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2355:x1875503571753088:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619934.219193:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64ac700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2355:x1875503571753088:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 14405us (14911us total) trans 0 rc 368640/368640 00000100:00100000:1.0:1788619934.219203:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1010 00002000:00100000:1.0:1788619934.219213:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c64ac700 x1875503571753088/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:30/0 lens 488/440 e 0 to 0 dl 1788619945 ref 1 fl Complete:/600/0 rc 368640/368640 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619934.219220:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64ac700 with x1875503571753088 00000100:00100000:1.0:1788619934.222984:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619934.223076:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571753216 00000100:00100000:0.0:1788619934.223085:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571753216 00000100:00100000:0.0:1788619934.223133:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1334 00000100:00100000:0.0:1788619934.223140:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571753216:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619934.223156:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619934.223174:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000aa0d867/0x1cd7fe5368736e6f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1de5f expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619934.223236:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f75180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589274752:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619934.223360:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5eafc00 going to sleep for 11 seconds 00000100:00100000:0.0:1788619934.225930:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f75180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589274752:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619934.225968:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000aa0d867/0x1cd7fe5368736e6f lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de5f expref: 8 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.225999:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000aa0d867/0x1cd7fe5368736e6f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de5f expref: 8 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.226012:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e6f64e00, rc 0) 00000100:00100000:0.0:1788619934.226072:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571753216:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2931us (3098us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619934.226080:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1334 00000100:00100000:1.0:1788619934.243422:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619934.243536:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571753344 00000100:00100000:2.0:1788619934.243545:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571753344 00000100:00100000:2.0:1788619934.243595:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1335 00000100:00100000:2.0:1788619934.243605:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2352:x1875503571753344:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619934.243623:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619934.243641:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fb5126d0/0x1cd7fe5368736e76 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1de66 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619934.243698:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64adf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589274880:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619934.243791:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81640c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619934.245784:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64adf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589274880:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619934.245822:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000fb5126d0/0x1cd7fe5368736e76 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de66 expref: 8 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.245848:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fb5126d0/0x1cd7fe5368736e76 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de66 expref: 8 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.245862:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6a3f000, rc 0) 00000100:00100000:2.0:1788619934.245918:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2352:x1875503571753344:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2314us (2505us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619934.245931:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1335 00000100:00100000:1.0:1788619934.250205:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619934.250291:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571753472 00000100:00100000:2.0:1788619934.250300:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571753472 00000100:00100000:2.0:1788619934.250384:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1336 00000100:00100000:2.0:1788619934.250404:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571753472:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619934.250422:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619934.250439:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000050e05b5d/0x1cd7fe5368736e7d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1de6d expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619934.250508:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64adf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589275008:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619934.250593:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81640c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619934.253836:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64adf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589275008:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619934.253873:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000050e05b5d/0x1cd7fe5368736e7d lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de6d expref: 8 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.253898:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000050e05b5d/0x1cd7fe5368736e7d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de6d expref: 8 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.253912:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6a3fa00, rc 0) 00000100:00100000:2.0:1788619934.253978:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571753472:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3584us (3783us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619934.253988:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1336 00000100:00100000:1.0:1788619934.258644:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619934.258725:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571753600 00000100:00100000:0.0:1788619934.258735:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571753600 00000100:00100000:0.0:1788619934.258805:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1337 00000100:00100000:0.0:1788619934.258815:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2353:x1875503571753600:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619934.258832:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619934.258850:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000aa0d867/0x1cd7fe5368736e84 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1de74 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619934.258928:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f75180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589275136:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619934.259023:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5eafc00 going to sleep for 11 seconds 00000100:00100000:0.0:1788619934.262085:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f75180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589275136:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619934.262124:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000aa0d867/0x1cd7fe5368736e84 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de74 expref: 8 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.262148:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000aa0d867/0x1cd7fe5368736e84 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de74 expref: 8 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.262165:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e6f64e00, rc 0) 00000100:00100000:0.0:1788619934.262227:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2353:x1875503571753600:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3413us (3592us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619934.262235:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1337 00000100:00100000:1.0:1788619934.266936:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619934.267008:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571753728 00000100:00100000:3.0:1788619934.267018:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571753728 00000100:00100000:3.0:1788619934.267080:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1631 00000100:00100000:3.0:1788619934.267090:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+17:14908:x1875503571753728:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619934.267111:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619934.267114:0:9417:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1788619934.267159:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65a00/0x1cd7fe5368736e68 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de58 expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.267179:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619934.267184:0:9417:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788619934.267200:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7e00/0x1cd7fe5368736e8b lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1de7b expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.267225:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7e00/0x1cd7fe5368736e8b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de7b expref: 17 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.267238:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7e00/0x1cd7fe5368736e8b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de7b expref: 17 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.267249:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619934.267252:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619934.267255:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78ccca7e00, rc 0) 00000100:00100000:3.0:1788619934.267321:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+17:14908:x1875503571753728:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 232us (394us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619934.267365:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1631 00000100:00100000:1.0:1788619934.270280:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619934.270411:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571753856 00000100:00100000:3.0:1788619934.270420:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571753856 00000100:00100000:3.0:1788619934.270499:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1011 00000100:00100000:3.0:1788619934.270510:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2352:x1875503571753856:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:3.0:1788619934.270787:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 364890 -> 364890 00000004:00010000:3.0:1788619934.270791:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619933 -> 1788619933 00000004:00010000:3.0:1788619934.270794:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619934 -> 1788619934 00000004:00010000:3.0:1788619934.270796:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619933 -> 1788619933 00000004:00010000:3.0:1788619934.270799:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 774 -> 774 00000100:00100000:3.0:1788619934.270870:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2352:x1875503571753856:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 361us (599us total) trans 4294967865 rc 0/0 00000100:00100000:3.0:1788619934.270877:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1011 00000100:00100000:1.0:1788619934.273931:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619934.274027:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571753984 00000100:00100000:0.0:1788619934.274036:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571753984 00000100:00100000:0.0:1788619934.274096:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1338 00000100:00100000:0.0:1788619934.274107:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2353:x1875503571753984:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619934.274124:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619934.274141:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000aa0d867/0x1cd7fe5368736e92 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1de82 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619934.274200:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f75180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589275264:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619934.274283:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5eafc00 going to sleep for 11 seconds 00000100:00100000:0.0:1788619934.277009:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f75180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589275264:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619934.277049:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000aa0d867/0x1cd7fe5368736e92 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de82 expref: 8 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.277074:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000aa0d867/0x1cd7fe5368736e92 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de82 expref: 8 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.277086:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e6f64e00, rc 0) 00000100:00100000:0.0:1788619934.277145:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2353:x1875503571753984:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3039us (3221us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619934.277155:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1338 00000100:00100000:1.0:1788619934.280120:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619934.280227:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571754112 00000100:00100000:2.0:1788619934.280236:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571754112 00000100:00100000:2.0:1788619934.280300:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1339 00000100:00100000:2.0:1788619934.280322:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2352:x1875503571754112:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619934.280376:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619934.280395:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000050e05b5d/0x1cd7fe5368736e99 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1de89 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619934.280461:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64adf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589275392:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619934.280551:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81640c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619934.283521:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64adf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589275392:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619934.283563:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000050e05b5d/0x1cd7fe5368736e99 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de89 expref: 8 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.283589:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000050e05b5d/0x1cd7fe5368736e99 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de89 expref: 8 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.283602:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6a3fa00, rc 0) 00000100:00100000:2.0:1788619934.283662:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2352:x1875503571754112:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3342us (3553us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619934.283670:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1339 00000100:00100000:1.0:1788619934.286698:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619934.286770:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571754368 00000100:00100000:2.0:1788619934.286779:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571754368 00000100:00100000:2.0:1788619934.286832:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1632 00000100:00100000:2.0:1788619934.286841:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+19:14908:x1875503571754368:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:2.0:1788619934.287093:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3f000/0x1cd7fe5368736ea0 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.287114:0:11397:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6400/0x1cd7fe5368736e1b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de0b expref: 17 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.287122:0:11397:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3f000/0x1cd7fe5368736ea0 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.287152:0:11397:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6400/0x1cd7fe5368736e1b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de0b expref: 17 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.287165:0:11397:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6400/0x1cd7fe5368736e1b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de0b expref: 17 pid: 5844 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619934.287179:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64adf80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589275520:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619934.287251:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81640c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619934.290264:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619934.290363:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571754496 00000100:00100000:0.0:1788619934.290373:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571754496 00010000:00010000:0.0:1788619934.290418:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6400/0x1cd7fe5368736e1b lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de0b expref: 18 pid: 5844 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619934.290459:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 543 00000100:00100000:0.0:1788619934.290469:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c80e9850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2354:x1875503571754496:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788619934.290497:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619934.290505:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6400/0x1cd7fe5368736e1b lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de0b expref: 18 pid: 5844 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.290517:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6400/0x1cd7fe5368736e1b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de0b expref: 18 pid: 5844 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.290539:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619934.290545:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3f000/0x1cd7fe5368736ea0 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.290554:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3f000/0x1cd7fe5368736ea0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.290561:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3f000/0x1cd7fe5368736ea0 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.290579:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1788619934.290623:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c80e9850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2354:x1875503571754496:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 155us (368us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619934.290633:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 543 00010000:00010000:3.0:1788619934.291577:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6400/0x1cd7fe5368736e1b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de0b expref: 17 pid: 5844 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619934.291597:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64adf80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589275520:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619934.291617:0:11397:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3f000/0x1cd7fe5368736ea0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.291631:0:11397:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3f000/0x1cd7fe5368736ea0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.291637:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3f000/0x1cd7fe5368736ea0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1788619934.291819:0:11397:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967866 00000004:00010000:3.0:1788619934.291869:0:11397:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 364890 -> 364890 00000004:00010000:3.0:1788619934.291872:0:11397:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619933 -> 1788619934 00000004:00010000:3.0:1788619934.291874:0:11397:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619934 -> 1788619934 00000004:00010000:3.0:1788619934.291876:0:11397:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619933 -> 1788619934 00000004:00010000:3.0:1788619934.291878:0:11397:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 774 -> 774 00000004:00010000:3.0:1788619934.291886:0:11397:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000c0c03197 reply state 00000000e33a98ba transno 4294967867 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3f000/0x1cd7fe5368736ea0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619934.291895:0:11397:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788619934.291997:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+22:14908:x1875503571754368:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 5156us (5307us total) trans 4294967867 rc 0/0 00000100:00100000:3.0:1788619934.292005:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1632 00000020:00080000:3.0:1788619934.292010:0:11397:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1788619934.294474:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3f000/0x1cd7fe5368736ea0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.294503:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3f000/0x1cd7fe5368736ea0 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.294512:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3f000/0x1cd7fe5368736ea0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.294517:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c6a3f000 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788619934.294522:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3f000/0x1cd7fe5368736ea0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.294530:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3f000/0x1cd7fe5368736ea0 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.294546:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619934.294548:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788619934.294553:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3f000/0x1cd7fe5368736ea0 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.294559:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3f000/0x1cd7fe5368736ea0 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619934.296599:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619934.297008:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571754624 00000100:00100000:0.0:1788619934.297018:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571754624 00000100:00100000:0.0:1788619934.297073:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1633 00000100:00100000:0.0:1788619934.297082:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+21:14908:x1875503571754624:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619934.297103:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619934.297123:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64e00/0x1cd7fe5368736ea7 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1de90 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.297152:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7e00/0x1cd7fe5368736e8b lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de7b expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.297162:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64e00/0x1cd7fe5368736ea7 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de90 expref: 22 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.297193:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7e00/0x1cd7fe5368736e8b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de7b expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.297206:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7e00/0x1cd7fe5368736e8b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de7b expref: 16 pid: 9417 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619934.297219:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f75180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589275648:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619934.297289:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5eaf240 going to sleep for 11 seconds 00000100:00100000:1.0:1788619934.306730:0:3317:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fe063b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3317:1875503589276032:0@lo:400:kworker.0 00000100:00100000:0.0:1788619934.306847:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3316:1875503589275904:0@lo:400:kworker.0 00000100:00100000:0.0:1788619934.306929:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788619934.306961:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fe060a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589276416:0@lo:400:kworker.0 00000100:00100000:0.0:1788619934.306987:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788619934.307028:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589275904 00000100:00100000:0.0:1788619934.307036:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589275904 00000100:00100000:0.0:1788619934.307084:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 195 00000100:00100000:0.0:1788619934.307093:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f75c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3316:x1875503589275904:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788619934.307140:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f75c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3316:x1875503589275904:12345-0@lo:400:kworker.0 Request processed in 48us (211us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619934.307151:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 195 00000100:00100000:0.0:1788619934.307192:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3316:1875503589275904:0@lo:400:kworker.0 00000100:00100000:0.0:1788619934.307216:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589276416 00000100:00100000:0.0:1788619934.307220:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589276416 00000100:00100000:0.0:1788619934.307265:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 132 00000100:00100000:0.0:1788619934.307273:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f77100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503589276416:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788619934.307309:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f77100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503589276416:12345-0@lo:400:kworker.0 Request processed in 38us (321us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619934.307314:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 132 00000100:00080000:0.0:1788619934.307353:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1788619934.307373:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe060a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589276416:0@lo:400:kworker.0 00000100:00100000:3.0:1788619934.307559:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fe062d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589276160:0@lo:400:kworker.0 00000100:00100000:3.0:1788619934.307591:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619934.307621:0:3318:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:3.0:1788619934.307626:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:3318:1875503589275776:0@lo:400:kworker.0 00000100:00100000:3.0:1788619934.307651:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619934.307676:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fe060e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3318:1875503589276288:0@lo:400:kworker.0 00000100:00100000:3.0:1788619934.307697:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619934.307724:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589275776 00000100:00100000:3.0:1788619934.307728:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589275776 00000100:00100000:3.0:1788619934.307767:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 296 00000100:00100000:3.0:1788619934.307774:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5cae050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+19:3318:x1875503589275776:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788619934.307810:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5cae050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+19:3318:x1875503589275776:12345-0@lo:400:kworker.0 Request processed in 37us (158us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619934.307817:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 296 00000100:00100000:3.0:1788619934.307856:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:3318:1875503589275776:0@lo:400:kworker.0 00000100:00100000:3.0:1788619934.307874:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589276288 00000100:00100000:3.0:1788619934.307877:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589276288 00000100:00100000:3.0:1788619934.307920:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 196 00000100:00100000:3.0:1788619934.307927:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74aad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3318:x1875503589276288:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788619934.307959:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74aad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3318:x1875503589276288:12345-0@lo:400:kworker.0 Request processed in 33us (261us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619934.307964:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 196 00000100:00080000:3.0:1788619934.307977:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788619934.307991:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe060e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3318:1875503589276288:0@lo:400:kworker.0 00000100:00100000:1.0:1788619934.308357:0:3317:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788619934.308472:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589276032 00000100:00100000:2.0:1788619934.308477:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589276032 00000100:00100000:2.0:1788619934.308535:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 197 00000100:00100000:2.0:1788619934.308543:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe063480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3317:x1875503589276032:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788619934.308586:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe063480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3317:x1875503589276032:12345-0@lo:400:kworker.0 Request processed in 44us (229us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619934.308592:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 197 00000100:00100000:2.0:1788619934.308625:0:3317:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe063b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3317:1875503589276032:0@lo:400:kworker.0 00000100:00100000:0.0:1788619934.308769:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589276160 00000100:00100000:0.0:1788619934.308774:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589276160 00000100:00100000:0.0:1788619934.308812:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 133 00000100:00100000:0.0:1788619934.308820:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74a9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589276160:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788619934.308859:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74a9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589276160:12345-0@lo:400:kworker.0 Request processed in 40us (1267us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619934.308865:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 133 00000100:00080000:3.0:1788619934.309230:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1788619934.309234:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788619934.309274:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe062d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589276160:0@lo:400:kworker.0 00000100:00100000:2.0:1788619934.314428:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619934.314531:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571754752 00000100:00100000:1.0:1788619934.314541:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571754752 00010000:00010000:1.0:1788619934.314580:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7e00/0x1cd7fe5368736e8b lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de7b expref: 17 pid: 9417 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619934.314622:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 544 00000100:00100000:1.0:1788619934.314632:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64ac380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+17:2355:x1875503571754752:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619934.314646:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619934.314674:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7e00/0x1cd7fe5368736e8b lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de7b expref: 17 pid: 9417 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.314687:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7e00/0x1cd7fe5368736e8b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de7b expref: 17 pid: 9417 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.314709:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619934.314716:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64e00/0x1cd7fe5368736ea7 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de90 expref: 22 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.314728:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64e00/0x1cd7fe5368736ea7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de90 expref: 22 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.314737:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64e00/0x1cd7fe5368736ea7 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de90 expref: 22 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.314790:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64e00/0x1cd7fe5368736ea7 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de90 expref: 22 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619934.314803:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fe060700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589276544:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619934.314875:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc9c0 going to sleep for 11 seconds 00010000:00010000:0.0:1788619934.316559:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca7e00/0x1cd7fe5368736e8b lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de7b expref: 17 pid: 9417 timeout: 526 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619934.316586:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f75180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589275648:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619934.316619:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64e00/0x1cd7fe5368736ea7 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de90 expref: 22 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.316641:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619934.316644:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619934.316648:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e6f64e00, rc 0) 00000100:00100000:0.0:1788619934.316713:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+22:14908:x1875503571754624:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 19631us (20122us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619934.316722:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1633 00000100:00100000:1.0:1788619934.321804:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe060700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589276544:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619934.321828:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619934.321893:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64ac380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571754752:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 7260us (7472us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619934.322836:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 544 00000100:00100000:1.0:1788619934.325619:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619934.325709:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571754880 00000100:00100000:3.0:1788619934.325720:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571754880 00002000:00010000:3.0:1788619934.325776:0:5857:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:3.0:1788619934.325782:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fe060700 with x1875503571754880 00000100:00100000:3.0:1788619934.325819:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1012 00000100:00100000:3.0:1788619934.325829:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe060700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+22:2354:x1875503571754880:12345-192.168.201.5@tcp:10:fsx.0 00000004:00010000:3.0:1788619934.326159:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 364890 -> 295642 00000004:00010000:3.0:1788619934.326163:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619934 -> 1788619934 00000004:00010000:3.0:1788619934.326166:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619934 -> 1788619934 00000004:00010000:3.0:1788619934.326169:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619934 -> 1788619934 00000004:00010000:3.0:1788619934.326171:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 774 -> 774 00000100:00100000:3.0:1788619934.326247:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe060700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+23:2354:x1875503571754880:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 419us (637us total) trans 4294967868 rc 0/0 00000100:00100000:3.0:1788619934.326553:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1012 00002000:00010000:3.0:1788619934.326558:0:5857:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:3.0:1788619934.326560:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fe060700 with x1875503571754880 00000100:00100000:2.0:1788619934.331429:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619934.331811:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571755008 00000100:00100000:3.0:1788619934.331824:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571755008 00002000:00010000:3.0:1788619934.331867:0:6701:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:3.0:1788619934.331872:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64aed80 with x1875503571755008 ext(0->18446744073709551615) 00002000:00010000:3.0:1788619934.331884:0:6701:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64aed80. 00000100:00100000:3.0:1788619934.331967:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 816 00000100:00100000:3.0:1788619934.331978:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64aed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2355:x1875503571755008:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:3.0:1788619934.332203:0:6701:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:3.0:1788619934.332207:0:6701:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619933 -> 1788619934 00002000:00010000:3.0:1788619934.332210:0:6701:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619934 -> 1788619933 00002000:00010000:3.0:1788619934.332212:0:6701:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619933 -> 1788619934 00002000:00010000:3.0:1788619934.332213:0:6701:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 391 -> 391 00000100:00100000:3.0:1788619934.332278:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64aed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2355:x1875503571755008:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 300us (857us total) trans 4294967531 rc 0/0 00000100:00100000:3.0:1788619934.332285:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 816 00002000:00010000:3.0:1788619934.332289:0:6701:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:3.0:1788619934.332292:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64aed80 with x1875503571755008 ext(0->18446744073709551615) 00002000:00010000:3.0:1788619934.332296:0:6701:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64aed80. 00000100:00100000:2.0:1788619934.343912:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619934.345841:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571755136 00000100:00100000:3.0:1788619934.345855:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571755136 00000100:00100000:3.0:1788619934.345930:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1634 00000100:00100000:3.0:1788619934.345942:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64aea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+23:14908:x1875503571755136:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619934.345966:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619934.345985:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6400/0x1cd7fe5368736eae lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1de97 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.346072:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6200/0x1cd7fe5368736eb5 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.346087:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6200/0x1cd7fe5368736eb5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.346099:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6200/0x1cd7fe5368736eb5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788619934.346126:0:11397:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 295642 00000004:00200000:3.0:1788619934.346131:0:11397:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788619934.346153:0:11397:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6200/0x1cd7fe5368736eb5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.346184:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6400/0x1cd7fe5368736eae lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de97 expref: 25 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.346204:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6200/0x1cd7fe5368736eb5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de97 expref: 24 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.346219:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619934.346222:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1788619934.346225:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78ccca6200, rc 0) 00000100:00100000:3.0:1788619934.346295:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64aea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+24:14908:x1875503571755136:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 355us (2391us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619934.346302:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1634 00000100:00100000:1.0:1788619934.357245:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619934.357360:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571755392 00000100:00100000:2.0:1788619934.357371:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571755392 00000100:00100000:2.0:1788619934.357476:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1635 00000100:00100000:2.0:1788619934.357496:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe062a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:14908:x1875503571755392:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:2.0:1788619934.357773:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3f800/0x1cd7fe5368736ebc lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.357791:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6200/0x1cd7fe5368736eb5 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de97 expref: 23 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.357800:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3f800/0x1cd7fe5368736ebc lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.357834:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6200/0x1cd7fe5368736eb5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de97 expref: 23 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.357847:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6200/0x1cd7fe5368736eb5 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de97 expref: 23 pid: 11397 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619934.357860:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64afb80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589276672:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619934.357942:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164840 going to sleep for 11 seconds 00000100:00100000:2.0:1788619934.360776:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619934.360899:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571755520 00000100:00100000:3.0:1788619934.360919:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571755520 00010000:00010000:3.0:1788619934.360956:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6200/0x1cd7fe5368736eb5 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de97 expref: 24 pid: 11397 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619934.360993:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 545 00000100:00100000:3.0:1788619934.361003:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64ace00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+24:2355:x1875503571755520:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619934.361017:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619934.361023:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6200/0x1cd7fe5368736eb5 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de97 expref: 24 pid: 11397 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.361037:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6200/0x1cd7fe5368736eb5 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de97 expref: 24 pid: 11397 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.361058:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619934.361063:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3f800/0x1cd7fe5368736ebc lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.361074:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3f800/0x1cd7fe5368736ebc lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.361087:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3f800/0x1cd7fe5368736ebc lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.361105:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619934.361159:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64ace00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+24:2355:x1875503571755520:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 158us (393us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619934.361167:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 545 00010000:00010000:0.0:1788619934.361268:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78ccca6200/0x1cd7fe5368736eb5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de97 expref: 23 pid: 11397 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619934.361286:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64afb80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589276672:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619934.361308:0:5846:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3f800/0x1cd7fe5368736ebc lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.361317:0:5846:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3f800/0x1cd7fe5368736ebc lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.361364:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3f800/0x1cd7fe5368736ebc lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788619934.361496:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967869 00000004:00010000:0.0:1788619934.361541:0:5846:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 295642 -> 295642 00000004:00010000:0.0:1788619934.361545:0:5846:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619934 -> 1788619934 00000004:00010000:0.0:1788619934.361548:0:5846:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619934 -> 1788619934 00000004:00010000:0.0:1788619934.361550:0:5846:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619934 -> 1788619934 00000004:00010000:0.0:1788619934.361553:0:5846:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 774 -> 774 00000004:00010000:0.0:1788619934.361560:0:5846:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000002cf6d2c3 reply state 00000000c821834b transno 4294967870 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3f800/0x1cd7fe5368736ebc lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619934.361569:0:5846:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788619934.361634:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe062a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+19:14908:x1875503571755392:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 4141us (4401us total) trans 4294967870 rc 0/0 00000100:00100000:0.0:1788619934.361641:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1635 00000020:00080000:0.0:1788619934.361644:0:5846:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:1.0:1788619934.364017:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3f800/0x1cd7fe5368736ebc lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.364033:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3f800/0x1cd7fe5368736ebc lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.364040:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3f800/0x1cd7fe5368736ebc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.364046:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c6a3f800 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:1.0:1788619934.364051:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3f800/0x1cd7fe5368736ebc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.364058:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3f800/0x1cd7fe5368736ebc lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.364078:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619934.364081:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788619934.364085:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3f800/0x1cd7fe5368736ebc lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.364091:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3f800/0x1cd7fe5368736ebc lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619934.364646:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619934.364717:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571755648 00000100:00100000:0.0:1788619934.364726:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571755648 00000100:00100000:0.0:1788619934.364785:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1636 00000100:00100000:0.0:1788619934.364796:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64ad180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+18:14908:x1875503571755648:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619934.364819:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619934.364836:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65000/0x1cd7fe5368736ec3 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1de9e expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.364870:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64e00/0x1cd7fe5368736ea7 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de90 expref: 22 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.364880:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65000/0x1cd7fe5368736ec3 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de9e expref: 19 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.364924:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64e00/0x1cd7fe5368736ea7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de90 expref: 22 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.364937:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64e00/0x1cd7fe5368736ea7 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de90 expref: 22 pid: 5845 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619934.364949:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f75180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589276800:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619934.365029:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5eaf240 going to sleep for 11 seconds 00000100:00100000:3.0:1788619934.368119:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619934.368191:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571755776 00000100:00100000:2.0:1788619934.368199:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571755776 00010000:00010000:2.0:1788619934.368237:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64e00/0x1cd7fe5368736ea7 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de90 expref: 23 pid: 5845 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619934.368275:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 546 00000100:00100000:2.0:1788619934.368284:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74ab800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+23:2354:x1875503571755776:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619934.368297:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619934.368349:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64e00/0x1cd7fe5368736ea7 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de90 expref: 23 pid: 5845 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.368362:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64e00/0x1cd7fe5368736ea7 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de90 expref: 23 pid: 5845 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.368384:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619934.368389:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65000/0x1cd7fe5368736ec3 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de9e expref: 19 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.368401:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65000/0x1cd7fe5368736ec3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de9e expref: 19 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.368410:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65000/0x1cd7fe5368736ec3 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de9e expref: 19 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.368444:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65000/0x1cd7fe5368736ec3 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de9e expref: 19 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619934.368458:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64ae300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589276928:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619934.368550:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164000 going to sleep for 11 seconds 00010000:00010000:0.0:1788619934.369062:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64e00/0x1cd7fe5368736ea7 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de90 expref: 23 pid: 5845 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619934.369079:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f75180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589276800:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619934.369101:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65000/0x1cd7fe5368736ec3 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de9e expref: 19 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.369112:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619934.369115:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619934.369118:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e6f65000, rc 0) 00000100:00100000:0.0:1788619934.369159:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64ad180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+19:14908:x1875503571755648:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4365us (4517us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619934.369167:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1636 00000100:00100000:2.0:1788619934.372528:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64ae300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589276928:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619934.372556:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619934.372612:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74ab800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:1b0df97a-4f84-4680-9108-88353da6ca92+22:2354:x1875503571755776:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 4328us (4500us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619934.372623:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 546 00000100:00100000:1.0:1788619934.375088:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619934.375167:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571755904 00000100:00100000:0.0:1788619934.375177:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571755904 00000100:00100000:0.0:1788619934.375249:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1340 00000100:00100000:0.0:1788619934.375259:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe062300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:14908:x1875503571755904:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619934.375277:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619934.375296:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ab469d71/0x1cd7fe5368736eca lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dea5 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.375368:0:6696:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000989ca2b5/0x1cd7fe5368736dce lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ddda expref: 9 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.375377:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=1s v=(11 11) 00010000:00010000:0.0:1788619934.375382:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(12 11) 00010000:00010000:0.0:1788619934.375388:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab469d71/0x1cd7fe5368736eca lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dea5 expref: 8 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.375423:0:6696:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000989ca2b5/0x1cd7fe5368736dce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ddda expref: 9 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.375436:0:6696:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000989ca2b5/0x1cd7fe5368736dce lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ddda expref: 9 pid: 6697 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619934.375450:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f75180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589277056:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619934.375538:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5eaf240 going to sleep for 11 seconds 00000100:00100000:3.0:1788619934.378620:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619934.378733:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571756032 00000100:00100000:1.0:1788619934.378742:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571756032 00010000:00010000:1.0:1788619934.378791:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000989ca2b5/0x1cd7fe5368736dce lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ddda expref: 10 pid: 6697 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619934.378835:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 547 00000100:00100000:1.0:1788619934.378846:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74a9c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571756032:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619934.378862:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619934.378883:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000989ca2b5/0x1cd7fe5368736dce lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ddda expref: 10 pid: 6697 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.378897:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000989ca2b5/0x1cd7fe5368736dce lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ddda expref: 10 pid: 6697 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.378922:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(12 11) 00010000:00010000:1.0:1788619934.378927:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(12 11) 00010000:00010000:1.0:1788619934.378936:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000ab469d71/0x1cd7fe5368736eca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dea5 expref: 8 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.378953:0:5829:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ab469d71/0x1cd7fe5368736eca lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dea5 expref: 8 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.378989:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000ab469d71/0x1cd7fe5368736eca lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dea5 expref: 8 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619934.379005:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fe063100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1875503589277184:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619934.379086:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c54cc9c0 going to sleep for 11 seconds 00010000:00010000:0.0:1788619934.379716:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000989ca2b5/0x1cd7fe5368736dce lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ddda expref: 10 pid: 6697 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619934.379735:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f75180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589277056:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619934.379768:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ab469d71/0x1cd7fe5368736eca lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dea5 expref: 8 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.379780:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e6f64400, rc 0) 00000100:00100000:0.0:1788619934.379819:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe062300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+8:14908:x1875503571755904:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4562us (4740us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619934.379826:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1340 00000100:00100000:3.0:1788619934.383208:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619934.383237:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619934.383363:0:13347:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571756288 00000100:00100000:2.0:1788619934.383372:0:13347:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571756288 00002000:00010000:2.0:1788619934.383414:0:13347:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:2.0:1788619934.383419:0:13347:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c74abb80 with x1875503571756288 ext(0->18446744073709551615) 00002000:00010000:2.0:1788619934.383431:0:13347:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c74abb80. 00000100:00100000:2.0:1788619934.383454:0:13347:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 817 00000100:00100000:2.0:1788619934.383463:0:13347:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74abb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571756288:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:2.0:1788619934.383704:0:13347:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:2.0:1788619934.383707:0:13347:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619934 -> 1788619934 00002000:00010000:2.0:1788619934.383710:0:13347:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619933 -> 1788619933 00002000:00010000:2.0:1788619934.383712:0:13347:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619934 -> 1788619934 00002000:00010000:2.0:1788619934.383715:0:13347:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 391 -> 391 00000100:00100000:2.0:1788619934.383776:0:13347:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74abb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2354:x1875503571756288:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 313us (575us total) trans 4294967532 rc 0/0 00000100:00100000:2.0:1788619934.383783:0:13347:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 817 00002000:00010000:2.0:1788619934.383787:0:13347:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:2.0:1788619934.383790:0:13347:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c74abb80 with x1875503571756288 ext(0->18446744073709551615) 00002000:00010000:2.0:1788619934.383795:0:13347:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c74abb80. 00000100:00100000:0.0:1788619934.384268:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571756160 00000100:00100000:0.0:1788619934.384275:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571756160 00002000:00010000:0.0:1788619934.384307:0:5858:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:0.0:1788619934.384312:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c74a9500 with x1875503571756160 00000100:00100000:0.0:1788619934.384352:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1013 00000100:00100000:0.0:1788619934.384358:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74a9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2354:x1875503571756160:12345-192.168.201.5@tcp:10:fsx.0 00000004:00010000:0.0:1788619934.384473:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 295642 -> 757380 00000004:00010000:0.0:1788619934.384477:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619934 -> 1788619934 00000004:00010000:0.0:1788619934.384490:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619934 -> 1788619934 00000004:00010000:0.0:1788619934.384493:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619934 -> 1788619934 00000004:00010000:0.0:1788619934.384495:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 774 -> 774 00000100:00100000:0.0:1788619934.384531:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74a9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2354:x1875503571756160:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 172us (1290us total) trans 4294967871 rc 0/0 00000100:00100000:0.0:1788619934.384538:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1013 00002000:00010000:0.0:1788619934.384541:0:5858:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:0.0:1788619934.384543:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c74a9500 with x1875503571756160 00000100:00100000:1.0:1788619934.384916:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe063100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1875503589277184:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619934.384932:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619934.384961:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74a9c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571756032:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 6118us (6355us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619934.384968:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 547 00000100:00100000:3.0:1788619934.389850:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619934.389938:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571756416 00000100:00100000:0.0:1788619934.389946:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571756416 00002000:00100000:0.0:1788619934.389986:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c74aa680 x1875503571756416/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:30/0 lens 488/0 e 0 to 0 dl 1788619945 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619934.389994:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c74aa680 with x1875503571756416 00000100:00100000:0.0:1788619934.390026:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1014 00000100:00100000:0.0:1788619934.390032:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74aa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2353:x1875503571756416:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619934.395446:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74aa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2353:x1875503571756416:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 5403us (5594us total) trans 0 rc 757760/757760 00000100:00100000:1.0:1788619934.395458:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1014 00002000:00100000:1.0:1788619934.395469:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c74aa680 x1875503571756416/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:30/0 lens 488/440 e 0 to 0 dl 1788619945 ref 1 fl Complete:/600/0 rc 757760/757760 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619934.395476:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c74aa680 with x1875503571756416 00000100:00100000:3.0:1788619934.399051:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619934.399560:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571756544 00000100:00100000:0.0:1788619934.399570:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571756544 00002000:00100000:0.0:1788619934.399621:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c501e300 x1875503571756544/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:30/0 lens 488/0 e 0 to 0 dl 1788619945 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619934.399632:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c501e300 with x1875503571756544 00000100:00100000:0.0:1788619934.399673:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1015 00000100:00100000:0.0:1788619934.399681:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+20:2354:x1875503571756544:12345-192.168.201.5@tcp:4:fsx.0 00000004:00010000:1.0:1788619934.402690:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 757380 -> 757380 00000004:00010000:1.0:1788619934.402700:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619934 -> 1788619934 00000004:00010000:1.0:1788619934.402703:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619934 -> 1788619934 00000004:00010000:1.0:1788619934.402706:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619934 -> 1788619934 00000004:00010000:1.0:1788619934.402709:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 774 -> 774 00000100:00100000:1.0:1788619934.403142:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2354:x1875503571756544:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 3455us (4095us total) trans 4294967872 rc 0/0 00000100:00100000:1.0:1788619934.403154:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1015 00002000:00100000:1.0:1788619934.403167:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c501e300 x1875503571756544/t4294967872(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:30/0 lens 488/448 e 0 to 0 dl 1788619945 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619934.403177:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c501e300 with x1875503571756544 00000100:00100000:3.0:1788619934.404956:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619934.405049:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571756672 00000100:00100000:2.0:1788619934.405060:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571756672 00000100:00100000:2.0:1788619934.405119:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1637 00000100:00100000:2.0:1788619934.405128:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64d0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2352:x1875503571756672:12345-192.168.201.5@tcp:16:fsx.0 00000001:00080000:0.0:1788619934.419247:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967871 is committed 00000001:00080000:0.0:1788619934.419273:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967872 is committed 00000001:00080000:0.0:1788619934.419283:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967864 is committed 00000001:00080000:0.0:1788619934.419294:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967866 is committed 00000001:00080000:0.0:1788619934.419322:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967868 is committed 00000100:00100000:3.0:1788619934.419421:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64d0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571756672:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 14287us (14466us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619934.419434:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1637 00000100:00100000:3.0:1788619934.421042:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619934.421108:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571756800 00000100:00100000:0.0:1788619934.421123:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571756800 00000100:00100000:0.0:1788619934.421190:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1638 00000100:00100000:0.0:1788619934.421200:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69d2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571756800:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619934.421226:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619934.421248:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64a00/0x1cd7fe5368736ed1 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1deac expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.421317:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64e00/0x1cd7fe5368736ed8 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.421365:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64e00/0x1cd7fe5368736ed8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.421378:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64e00/0x1cd7fe5368736ed8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619934.421403:0:5846:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 757380 00000004:00200000:0.0:1788619934.421407:0:5846:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619934.421424:0:5846:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64e00/0x1cd7fe5368736ed8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.421443:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64a00/0x1cd7fe5368736ed1 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1deac expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.421457:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64e00/0x1cd7fe5368736ed8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1deac expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.421464:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619934.421466:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619934.421469:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e6f64e00, rc 0) 00000100:00100000:0.0:1788619934.421536:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69d2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571756800:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 337us (499us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619934.421543:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1638 00000100:00100000:0.0:1788619934.423573:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619934.423631:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571757056 00000100:00100000:3.0:1788619934.423638:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571757056 00000100:00100000:3.0:1788619934.423678:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1639 00000100:00100000:3.0:1788619934.423685:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f75180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+8:14908:x1875503571757056:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:3.0:1788619934.423940:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07200/0x1cd7fe5368736edf lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.423954:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64e00/0x1cd7fe5368736ed8 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1deac expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.423963:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07200/0x1cd7fe5368736edf lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.423988:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64e00/0x1cd7fe5368736ed8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1deac expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.423998:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64e00/0x1cd7fe5368736ed8 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1deac expref: 13 pid: 5846 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619934.424007:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69d3100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589277312:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619934.424153:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5d2d780 going to sleep for 11 seconds 00000100:00100000:0.0:1788619934.425433:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69d3100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589277312:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619934.425461:0:5845:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07200/0x1cd7fe5368736edf lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619934.430827:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619934.431477:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571757184 00000100:00100000:1.0:1788619934.431499:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571757184 00010000:00010000:1.0:1788619934.431540:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64e00/0x1cd7fe5368736ed8 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1deac expref: 14 pid: 5846 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619934.431611:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 548 00000100:00100000:1.0:1788619934.431619:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f75880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571757184:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619934.431632:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619934.431637:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64e00/0x1cd7fe5368736ed8 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1deac expref: 14 pid: 5846 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.431647:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64e00/0x1cd7fe5368736ed8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1deac expref: 14 pid: 5846 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.431664:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64e00/0x1cd7fe5368736ed8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1deac expref: 14 pid: 5846 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.431675:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619934.431679:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07200/0x1cd7fe5368736edf lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.431688:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07200/0x1cd7fe5368736edf lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.431693:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07200/0x1cd7fe5368736edf lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.431717:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619934.431768:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f75880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571757184:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 151us (953us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619934.431773:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 548 00010000:00010000:0.0:1788619934.431912:0:5845:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07200/0x1cd7fe5368736edf lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.431921:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07200/0x1cd7fe5368736edf lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788619934.432048:0:5845:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967873 00000004:00010000:0.0:1788619934.432088:0:5845:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 757380 -> 757380 00000004:00010000:0.0:1788619934.432090:0:5845:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619934 -> 1788619934 00000004:00010000:0.0:1788619934.432092:0:5845:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619934 -> 1788619934 00000004:00010000:0.0:1788619934.432094:0:5845:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619934 -> 1788619934 00000004:00010000:0.0:1788619934.432097:0:5845:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 774 -> 774 00000004:00010000:0.0:1788619934.432102:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000004d5f76f9 reply state 000000007d4fb671 transno 4294967874 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07200/0x1cd7fe5368736edf lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619934.432108:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788619934.432174:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f75180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571757056:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 8489us (8607us total) trans 4294967874 rc 0/0 00000100:00100000:0.0:1788619934.432179:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1639 00000020:00080000:0.0:1788619934.432182:0:5845:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:2.0:1788619934.433805:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07200/0x1cd7fe5368736edf lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.433822:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07200/0x1cd7fe5368736edf lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.433828:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07200/0x1cd7fe5368736edf lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.433834:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78dfa07200 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:2.0:1788619934.433839:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07200/0x1cd7fe5368736edf lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.433847:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07200/0x1cd7fe5368736edf lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.433861:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619934.433864:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788619934.433868:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07200/0x1cd7fe5368736edf lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.433873:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07200/0x1cd7fe5368736edf lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619934.434760:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619934.434811:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571757312 00000100:00100000:2.0:1788619934.434818:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571757312 00000100:00100000:2.0:1788619934.434856:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1640 00000100:00100000:2.0:1788619934.434862:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69d2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+10:14908:x1875503571757312:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619934.434881:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619934.434898:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e000/0x1cd7fe5368736ee6 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1deb3 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.434932:0:11397:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65000/0x1cd7fe5368736ec3 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de9e expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.434942:0:11397:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e000/0x1cd7fe5368736ee6 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1deb3 expref: 11 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.434975:0:11397:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65000/0x1cd7fe5368736ec3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de9e expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.434989:0:11397:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65000/0x1cd7fe5368736ec3 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de9e expref: 12 pid: 5845 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619934.435003:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64ae300 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589277440:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619934.435077:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81649c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619934.436967:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64ae300 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589277440:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619934.437002:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e000/0x1cd7fe5368736ee6 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1deb3 expref: 11 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.437027:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619934.437031:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619934.437036:0:11397:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e000/0x1cd7fe5368736ee6 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1deb3 expref: 11 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.437046:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6a3e000, rc 0) 00000100:00100000:2.0:1788619934.437100:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69d2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571757312:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2237us (2344us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619934.437109:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1640 00000100:00100000:0.0:1788619934.438642:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619934.438737:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571757440 00000100:00100000:2.0:1788619934.438747:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571757440 00010000:00010000:2.0:1788619934.438785:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65000/0x1cd7fe5368736ec3 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de9e expref: 13 pid: 5845 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619934.438881:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 549 00000100:00100000:2.0:1788619934.438892:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f74e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571757440:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619934.439008:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619934.439035:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65000/0x1cd7fe5368736ec3 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de9e expref: 13 pid: 5845 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.439049:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65000/0x1cd7fe5368736ec3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de9e expref: 13 pid: 5845 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.439070:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65000/0x1cd7fe5368736ec3 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1de9e expref: 13 pid: 5845 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.439083:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619934.439087:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e000/0x1cd7fe5368736ee6 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1deb3 expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.439094:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e000/0x1cd7fe5368736ee6 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1deb3 expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.439101:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e000/0x1cd7fe5368736ee6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1deb3 expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.439129:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e000/0x1cd7fe5368736ee6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1deb3 expref: 10 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619934.439146:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64ae300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589277568:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619934.439227:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81649c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619934.443878:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619934.443997:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64ae300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589277568:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619934.444021:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619934.444082:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f74e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571757440:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 5192us (5446us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619934.444094:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 549 00000100:00100000:0.0:1788619934.447567:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571757568 00000100:00100000:0.0:1788619934.447582:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571757568 00000100:00100000:0.0:1788619934.447655:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1341 00000100:00100000:0.0:1788619934.447667:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69d0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+9:14908:x1875503571757568:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619934.447686:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619934.447705:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c60d9008/0x1cd7fe5368736eed lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1deba expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.447739:0:7654:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000ab469d71/0x1cd7fe5368736eca lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dea5 expref: 8 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.447749:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(12 11) 00010000:00010000:0.0:1788619934.447754:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(13 11) 00010000:00010000:0.0:1788619934.447760:0:7654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c60d9008/0x1cd7fe5368736eed lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1deba expref: 10 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.447798:0:7654:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000ab469d71/0x1cd7fe5368736eca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dea5 expref: 8 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.447813:0:7654:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000ab469d71/0x1cd7fe5368736eca lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dea5 expref: 8 pid: 6696 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619934.447829:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f75500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589277696:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619934.447918:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5eaff00 going to sleep for 11 seconds 00000100:00100000:0.0:1788619934.452103:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f75500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589277696:192.168.201.5@tcp:104: 00010000:00010000:0.0:1788619934.452139:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c60d9008/0x1cd7fe5368736eed lrc: 3/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1deba expref: 10 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.452162:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e6f64800, rc 0) 00000100:00100000:0.0:1788619934.452225:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69d0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+10:14908:x1875503571757568:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4560us (8356us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619934.452235:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1341 00000100:00100000:3.0:1788619934.459652:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619934.459820:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571757696 00000100:00100000:2.0:1788619934.459832:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571757696 00010000:00010000:2.0:1788619934.459880:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000ab469d71/0x1cd7fe5368736eca lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dea5 expref: 9 pid: 6696 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619934.459930:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 550 00000100:00100000:2.0:1788619934.459940:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c80ebc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+9:2352:x1875503571757696:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619934.459956:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619934.459980:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000ab469d71/0x1cd7fe5368736eca lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dea5 expref: 9 pid: 6696 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.459994:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000ab469d71/0x1cd7fe5368736eca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dea5 expref: 9 pid: 6696 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.460018:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ab469d71/0x1cd7fe5368736eca lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dea5 expref: 9 pid: 6696 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.460041:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(13 11) 00010000:00010000:2.0:1788619934.460046:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(13 11) 00010000:00010000:2.0:1788619934.460056:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000c60d9008/0x1cd7fe5368736eed lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1deba expref: 9 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.460066:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c60d9008/0x1cd7fe5368736eed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1deba expref: 9 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.460102:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000c60d9008/0x1cd7fe5368736eed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1deba expref: 9 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619934.460118:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64ae300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589277824:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619934.460197:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81649c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619934.464414:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64ae300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589277824:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619934.464444:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619934.464513:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c80ebc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2352:x1875503571757696:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 4572us (4869us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619934.464523:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 550 00000100:00100000:3.0:1788619934.464996:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619934.465551:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571757824 00000100:00100000:0.0:1788619934.465560:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571757824 00002000:00010000:0.0:1788619934.465599:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1980470->18446744073709551615) 00002000:00100000:0.0:1788619934.465604:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c69d1f80 with x1875503571757824 ext(1980470->18446744073709551615) 00002000:00010000:0.0:1788619934.465616:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c69d1f80. 00000100:00100000:0.0:1788619934.465644:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 818 00000100:00100000:0.0:1788619934.465654:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69d1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+10:2354:x1875503571757824:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:0.0:1788619934.465883:0:6703:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 0 -> 1980470 00002000:00010000:0.0:1788619934.465887:0:6703:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619934 -> 1788619934 00002000:00010000:0.0:1788619934.465890:0:6703:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619933 -> 1788619933 00002000:00010000:0.0:1788619934.465893:0:6703:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619934 -> 1788619934 00002000:00010000:0.0:1788619934.465895:0:6703:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 391 -> 391 00000100:00100000:0.0:1788619934.466822:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69d1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2354:x1875503571757824:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 1164us (1822us total) trans 4294967533 rc 0/0 00000100:00100000:0.0:1788619934.466833:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 818 00002000:00010000:0.0:1788619934.466837:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1980470->18446744073709551615) 00002000:00100000:0.0:1788619934.466841:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c69d1f80 with x1875503571757824 ext(1980470->18446744073709551615) 00002000:00010000:0.0:1788619934.466847:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c69d1f80. 00000100:00100000:3.0:1788619934.477514:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619934.477807:0:13347:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571757952 00000100:00100000:2.0:1788619934.477819:0:13347:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571757952 00002000:00100000:2.0:1788619934.477870:0:13347:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1982463) req@ffff8e77c69d0700 x1875503571757952/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:30/0 lens 488/0 e 0 to 0 dl 1788619945 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619934.477881:0:13347:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c69d0700 with x1875503571757952 ext(1048576->1982463) 00002000:00010000:2.0:1788619934.477887:0:13347:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c69d0700 00000100:00100000:2.0:1788619934.477937:0:13347:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 819 00000100:00100000:2.0:1788619934.477959:0:13347:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69d0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571757952:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619934.480444:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619934.480617:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571758080 00000100:00100000:0.0:1788619934.480626:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571758080 00002000:00100000:0.0:1788619934.480672:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fe063100 x1875503571758080/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:30/0 lens 488/0 e 0 to 0 dl 1788619945 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619934.480683:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fe063100 with x1875503571758080 00000100:00100000:0.0:1788619934.480720:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1016 00000100:00100000:0.0:1788619934.480730:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe063100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571758080:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619934.491148:0:13347:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69d0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571757952:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 13192us (13634us total) trans 0 rc 933888/933888 00000100:00100000:2.0:1788619934.491162:0:13347:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 819 00002000:00100000:2.0:1788619934.491173:0:13347:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1982463) req@ffff8e77c69d0700 x1875503571757952/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:30/0 lens 488/440 e 0 to 0 dl 1788619945 ref 1 fl Complete:/600/0 rc 933888/933888 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619934.491185:0:13347:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c69d0700 with x1875503571757952 ext(1048576->1982463) 00002000:00010000:2.0:1788619934.491193:0:13347:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c69d0700 00000100:00100000:1.0:1788619934.494525:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe063100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2355:x1875503571758080:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 13786us (14078us total) trans 0 rc 389120/389120 00000100:00100000:1.0:1788619934.494539:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1016 00002000:00100000:1.0:1788619934.494551:0:5859:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_002: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fe063100 x1875503571758080/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:30/0 lens 488/440 e 0 to 0 dl 1788619945 ref 1 fl Complete:/600/0 rc 389120/389120 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619934.494561:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fe063100 with x1875503571758080 00010000:00010000:2.0:1788619934.499648:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788619934.501360:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619934.501430:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571758208 00000100:00100000:0.0:1788619934.501441:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571758208 00002000:00100000:0.0:1788619934.501503:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1925120->1980469) req@ffff8e77c64ac380 x1875503571758208/t0(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:30/0 lens 488/0 e 0 to 0 dl 1788619945 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619934.501513:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64ac380 with x1875503571758208 ext(1925120->1980469) 00002000:00010000:0.0:1788619934.501520:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64ac380 00000100:00100000:0.0:1788619934.501553:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 820 00000100:00100000:0.0:1788619934.501563:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64ac380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:2352:x1875503571758208:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:0.0:1788619934.505055:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64ac380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+13:2352:x1875503571758208:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 3483us (3720us total) trans 4294967534 rc 0/0 00000100:00100000:0.0:1788619934.505072:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 820 00002000:00100000:0.0:1788619934.505084:0:6703:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (1925120->1980469) req@ffff8e77c64ac380 x1875503571758208/t4294967534(0) o4->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:30/0 lens 488/448 e 0 to 0 dl 1788619945 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619934.505094:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64ac380 with x1875503571758208 ext(1925120->1980469) 00002000:00010000:0.0:1788619934.505100:0:6703:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64ac380 00000100:00100000:1.0:1788619934.510532:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619934.510646:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571758336 00000100:00100000:2.0:1788619934.510656:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571758336 00000100:00100000:2.0:1788619934.510720:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1342 00000100:00100000:2.0:1788619934.510730:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe061c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+13:2353:x1875503571758336:12345-192.168.201.5@tcp:16:fsx.0 00000100:00100000:3.0:1788619934.533194:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619934.533383:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571758464 00000100:00100000:1.0:1788619934.533394:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571758464 00000100:00100000:1.0:1788619934.533449:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 297 00000100:00100000:1.0:1788619934.533459:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5cadc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:597450a2-e90c-408c-b401-a0ce48eed204+13:2353:x1875503571758464:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:1.0:1788619934.533535:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5cadc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:597450a2-e90c-408c-b401-a0ce48eed204+13:2353:x1875503571758464:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 77us (348us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619934.533543:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 297 00000001:00080000:2.0:1788619934.540208:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967534 is committed 00000001:00080000:2.0:1788619934.540233:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967532 is committed 00000100:00100000:0.0:1788619934.543609:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe061c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571758336:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 32866us (33071us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619934.543624:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1342 00000100:00100000:3.0:1788619934.546932:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619934.547104:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571758592 00000100:00100000:0.0:1788619934.547114:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571758592 00000100:00100000:0.0:1788619934.547169:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1641 00000100:00100000:0.0:1788619934.547179:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69d1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+11:14908:x1875503571758592:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619934.547198:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619934.547218:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777ca00/0x1cd7fe5368736ef4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dec1 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.547304:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c600/0x1cd7fe5368736efb lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.547319:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c600/0x1cd7fe5368736efb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.547646:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c600/0x1cd7fe5368736efb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619934.547678:0:5846:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1980470 00000004:00200000:0.0:1788619934.547682:0:5846:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619934.547703:0:5846:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c600/0x1cd7fe5368736efb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.547732:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777ca00/0x1cd7fe5368736ef4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dec1 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.547751:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c600/0x1cd7fe5368736efb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dec1 expref: 12 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.547759:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619934.547763:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619934.547767:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c777c600, rc 0) 00000100:00100000:0.0:1788619934.548291:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69d1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+12:14908:x1875503571758592:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 1110us (1366us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619934.548301:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1641 00000100:00100000:1.0:1788619934.553773:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619934.554124:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571758720 00000100:00100000:0.0:1788619934.554135:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571758720 00000100:00100000:0.0:1788619934.554197:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1642 00000100:00100000:0.0:1788619934.554207:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe061880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+12:14908:x1875503571758720:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619934.554227:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619934.554245:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777ca00/0x1cd7fe5368736f02 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dec8 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.554369:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d800/0x1cd7fe5368736f09 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.554384:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d800/0x1cd7fe5368736f09 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.554395:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d800/0x1cd7fe5368736f09 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619934.554422:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1980470 00000004:00200000:0.0:1788619934.554426:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619934.554444:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d800/0x1cd7fe5368736f09 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.554472:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777ca00/0x1cd7fe5368736f02 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dec8 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.554502:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d800/0x1cd7fe5368736f09 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dec8 expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.554512:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619934.554515:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619934.554519:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c777d800, rc 0) 00000100:00100000:0.0:1788619934.554586:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe061880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571758720:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 380us (823us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619934.554594:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1642 00000100:00100000:3.0:1788619934.560516:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619934.560609:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571758848 00000100:00100000:2.0:1788619934.560619:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571758848 00000100:00100000:2.0:1788619934.560681:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1643 00000100:00100000:2.0:1788619934.560692:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69d2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571758848:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619934.560712:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619934.560734:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3fe00/0x1cd7fe5368736f10 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1decf expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.560786:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e000/0x1cd7fe5368736ee6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1deb3 expref: 11 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619934.560828:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64ace00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589277952:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619934.560914:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164300 going to sleep for 11 seconds 00000100:00100000:2.0:1788619934.563517:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619934.563643:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571758976 00000100:00100000:3.0:1788619934.563651:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571758976 00000100:00100000:3.0:1788619934.563711:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1343 00000100:00100000:3.0:1788619934.563721:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64ae300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571758976:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619934.563737:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619934.563762:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ce26d4f7/0x1cd7fe5368736f17 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1ded6 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619934.563825:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69d0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589278080:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619934.563909:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7600 going to sleep for 11 seconds 00000100:00100000:0.0:1788619934.564201:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64ace00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589277952:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619934.564221:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 757380 00010000:00010000:0.0:1788619934.564235:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3fe00/0x1cd7fe5368736f10 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1decf expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.564253:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619934.564255:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619934.564261:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3fe00/0x1cd7fe5368736f10 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1decf expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.564279:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6a3fe00, rc 0) 00000100:00100000:0.0:1788619934.564310:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69d2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571758848:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3620us (3804us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619934.564317:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1643 00002000:00010000:2.0:1788619934.567613:0:6698:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619933 -> 1788619934 00000100:00100000:2.0:1788619934.567649:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69d0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589278080:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619934.567685:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ce26d4f7/0x1cd7fe5368736f17 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ded6 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.567711:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ce26d4f7/0x1cd7fe5368736f17 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1ded6 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.567724:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa07800, rc 0) 00000100:00100000:2.0:1788619934.567802:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64ae300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571758976:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4082us (4292us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619934.567810:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1343 00000100:00100000:2.0:1788619934.572137:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619934.572178:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619934.572258:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571759104 00000100:00100000:0.0:1788619934.572268:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571759104 00000100:00100000:0.0:1788619934.572362:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1644 00000100:00100000:0.0:1788619934.572372:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64af480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571759104:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619934.572391:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619934.572411:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777ca00/0x1cd7fe5368736f1e lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dedd expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.572445:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e000/0x1cd7fe5368736ee6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1deb3 expref: 11 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619934.572495:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f75500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589278208:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619934.572566:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5eaf300 going to sleep for 11 seconds 00000100:00100000:3.0:1788619934.573257:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571759232 00000100:00100000:3.0:1788619934.573264:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571759232 00000100:00100000:3.0:1788619934.573310:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1344 00000100:00100000:3.0:1788619934.573318:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64af100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571759232:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619934.573347:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619934.573360:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff771b43/0x1cd7fe5368736f25 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dee4 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619934.573402:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69d0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589278336:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619934.573555:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e70c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619934.577187:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f75500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589278208:192.168.201.5@tcp:106: 00002000:00010000:0.0:1788619934.577210:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 757380 00010000:00010000:0.0:1788619934.577226:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777ca00/0x1cd7fe5368736f1e lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dedd expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.577246:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619934.577249:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619934.577254:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777ca00/0x1cd7fe5368736f1e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dedd expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.577271:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c777ca00, rc 0) 00000100:00100000:0.0:1788619934.577355:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64af480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571759104:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4983us (5225us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619934.577364:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1644 00000100:00100000:3.0:1788619934.581096:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69d0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589278336:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619934.581134:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ff771b43/0x1cd7fe5368736f25 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dee4 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.581158:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff771b43/0x1cd7fe5368736f25 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dee4 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.581170:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa06400, rc 0) 00000100:00100000:3.0:1788619934.581228:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64af100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571759232:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7911us (9049us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619934.581236:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1344 00000100:00100000:2.0:1788619934.587075:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619934.587629:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619934.587851:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571759360 00000100:00100000:3.0:1788619934.587863:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571759360 00000100:00100000:3.0:1788619934.588147:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1645 00000100:00100000:3.0:1788619934.588159:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64ace00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571759360:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619934.588182:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619934.588202:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06600/0x1cd7fe5368736f2c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1deeb expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.588242:0:9417:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e000/0x1cd7fe5368736ee6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1deb3 expref: 11 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619934.588285:0:9417:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69d0380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589278464:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619934.588403:0:9417:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e70c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619934.590508:0:9417:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69d0380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:9417:1875503589278464:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619934.590532:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 757380 00010000:00010000:3.0:1788619934.590549:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06600/0x1cd7fe5368736f2c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1deeb expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.590570:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619934.590573:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619934.590579:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06600/0x1cd7fe5368736f2c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1deeb expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.590593:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa06600, rc 0) 00000100:00100000:3.0:1788619934.590640:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64ace00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2353:x1875503571759360:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2484us (3760us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619934.590648:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1645 00000100:00100000:1.0:1788619934.590733:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571759488 00000100:00100000:1.0:1788619934.590740:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571759488 00000100:00100000:1.0:1788619934.590793:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1345 00000100:00100000:1.0:1788619934.590802:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64afb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571759488:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619934.590814:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619934.590830:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad29727f/0x1cd7fe5368736f33 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1def2 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619934.590880:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fe062680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589278592:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619934.590924:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78e6e803c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619934.593969:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe062680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589278592:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619934.594232:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ad29727f/0x1cd7fe5368736f33 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1def2 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.594261:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad29727f/0x1cd7fe5368736f33 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1def2 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.594276:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c000, rc 0) 00000100:00100000:0.0:1788619934.594374:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64afb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571759488:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3571us (6745us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619934.594384:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1345 00000100:00100000:3.0:1788619934.597578:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619934.597760:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571759616 00000100:00100000:0.0:1788619934.597770:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571759616 00000100:00100000:0.0:1788619934.597821:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1646 00000100:00100000:0.0:1788619934.597830:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69d0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571759616:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619934.597854:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619934.597875:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777ca00/0x1cd7fe5368736f3a lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1def9 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.597913:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e000/0x1cd7fe5368736ee6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1deb3 expref: 11 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619934.597954:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f75500 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589278720:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619934.598047:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5eaf300 going to sleep for 11 seconds 00000100:00100000:3.0:1788619934.601208:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619934.601526:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f75500 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589278720:192.168.201.5@tcp:106: 00002000:00010000:2.0:1788619934.601551:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 757380 00010000:00010000:2.0:1788619934.601571:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777ca00/0x1cd7fe5368736f3a lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1def9 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.601592:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619934.601595:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788619934.601601:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777ca00/0x1cd7fe5368736f3a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1def9 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.601616:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c777ca00, rc 0) 00000100:00100000:2.0:1788619934.601680:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69d0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571759616:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3851us (4114us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619934.601690:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1646 00000100:00100000:0.0:1788619934.601785:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571759744 00000100:00100000:0.0:1788619934.601793:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571759744 00000100:00100000:0.0:1788619934.601853:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1346 00000100:00100000:0.0:1788619934.601863:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69d3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571759744:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619934.601878:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619934.601896:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008c4426fe/0x1cd7fe5368736f41 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1df00 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619934.601953:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f77800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589278848:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619934.602005:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5eafd80 going to sleep for 11 seconds 00000100:00100000:0.0:1788619934.605502:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f77800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589278848:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619934.605544:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008c4426fe/0x1cd7fe5368736f41 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df00 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.605572:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008c4426fe/0x1cd7fe5368736f41 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df00 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.605587:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c777d600, rc 0) 00000100:00100000:0.0:1788619934.605652:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69d3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571759744:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3790us (4456us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619934.605662:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1346 00000100:00100000:3.0:1788619934.613972:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619934.614200:0:13346:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571759872 00000100:00100000:2.0:1788619934.614210:0:13346:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571759872 00000100:00100000:2.0:1788619934.614283:0:13346:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1017 00000100:00100000:2.0:1788619934.614294:0:13346:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd939f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+12:2353:x1875503571759872:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:2.0:1788619934.614589:0:13346:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 757380 -> 757380 00000004:00010000:2.0:1788619934.614593:0:13346:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619934 -> 1788619934 00000004:00010000:2.0:1788619934.614595:0:13346:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619934 -> 1788619934 00000004:00010000:2.0:1788619934.614597:0:13346:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619934 -> 1788619934 00000004:00010000:2.0:1788619934.614599:0:13346:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 774 -> 774 00000100:00100000:2.0:1788619934.614666:0:13346:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd939f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:1b0df97a-4f84-4680-9108-88353da6ca92+13:2353:x1875503571759872:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 373us (704us total) trans 4294967875 rc 0/0 00000100:00100000:2.0:1788619934.614674:0:13346:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1017 00000100:00100000:3.0:1788619934.622863:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619934.623104:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571760128 00000100:00100000:0.0:1788619934.623113:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571760128 00000100:00100000:0.0:1788619934.623176:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1647 00000100:00100000:0.0:1788619934.623186:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd939500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+13:14908:x1875503571760128:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:0.0:1788619934.623242:0:5846:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788619934.623269:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c600/0x1cd7fe5368736efb lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dec1 expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.623288:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619934.623293:0:5846:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788619934.623569:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c000/0x1cd7fe5368736f48 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.623586:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d800/0x1cd7fe5368736f09 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dec8 expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.623595:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c000/0x1cd7fe5368736f48 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.623626:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d800/0x1cd7fe5368736f09 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dec8 expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.623641:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d800/0x1cd7fe5368736f09 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dec8 expref: 12 pid: 5845 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619934.623655:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f77800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589278976:192.168.201.5@tcp:104: 00000100:00100000:0.0:1788619934.623737:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5eaf900 going to sleep for 11 seconds 00000100:00100000:2.0:1788619934.626387:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f77800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589278976:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619934.626416:0:5846:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c000/0x1cd7fe5368736f48 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619934.628854:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619934.628966:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571760256 00000100:00100000:1.0:1788619934.628976:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571760256 00010000:00010000:1.0:1788619934.629026:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d800/0x1cd7fe5368736f09 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dec8 expref: 13 pid: 5845 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619934.629071:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 551 00000100:00100000:1.0:1788619934.629081:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd938700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2352:x1875503571760256:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619934.629096:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619934.629103:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d800/0x1cd7fe5368736f09 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dec8 expref: 13 pid: 5845 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.629116:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d800/0x1cd7fe5368736f09 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dec8 expref: 13 pid: 5845 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.629137:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d800/0x1cd7fe5368736f09 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dec8 expref: 13 pid: 5845 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.629156:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619934.629162:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c000/0x1cd7fe5368736f48 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.629172:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c000/0x1cd7fe5368736f48 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.629179:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c000/0x1cd7fe5368736f48 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.629212:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619934.629266:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd938700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2352:x1875503571760256:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 187us (422us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619934.629274:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 551 00010000:00010000:0.0:1788619934.629457:0:5846:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c000/0x1cd7fe5368736f48 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.629468:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c000/0x1cd7fe5368736f48 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1788619934.629592:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967876 00000004:00010000:0.0:1788619934.629641:0:5846:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 757380 -> 757380 00000004:00010000:0.0:1788619934.629644:0:5846:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619934 -> 1788619934 00000004:00010000:0.0:1788619934.629647:0:5846:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619934 -> 1788619934 00000004:00010000:0.0:1788619934.629649:0:5846:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619934 -> 1788619934 00000004:00010000:0.0:1788619934.629652:0:5846:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 774 -> 774 00000004:00010000:0.0:1788619934.629659:0:5846:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000b786ac3d reply state 00000000cd45efdd transno 4294967877 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c000/0x1cd7fe5368736f48 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619934.629669:0:5846:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788619934.629740:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd939500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+15:14908:x1875503571760128:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 6555us (6886us total) trans 4294967877 rc 0/0 00000100:00100000:0.0:1788619934.629747:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1647 00000020:00080000:0.0:1788619934.629751:0:5846:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1788619934.633859:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00010000:00010000:1.0:1788619934.633931:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c000/0x1cd7fe5368736f48 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.633952:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c000/0x1cd7fe5368736f48 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.633960:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c000/0x1cd7fe5368736f48 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.633965:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c777c000 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:1.0:1788619934.633970:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c000/0x1cd7fe5368736f48 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.633979:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c000/0x1cd7fe5368736f48 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.633998:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619934.634010:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788619934.634014:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c000/0x1cd7fe5368736f48 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.634021:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c000/0x1cd7fe5368736f48 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619934.634512:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571760384 00000100:00100000:3.0:1788619934.634523:0:6702:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571760384 00002000:00010000:3.0:1788619934.634561:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1520395->18446744073709551615) 00002000:00100000:3.0:1788619934.634565:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5f74a80 with x1875503571760384 ext(1520395->18446744073709551615) 00002000:00010000:3.0:1788619934.634576:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5f74a80. 00000100:00100000:3.0:1788619934.634611:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 821 00000100:00100000:3.0:1788619934.634620:0:6702:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f74a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571760384:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:0.0:1788619934.635998:0:6702:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1980470 -> 1520395 00002000:00010000:0.0:1788619934.636134:0:6702:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619934 -> 1788619934 00002000:00010000:0.0:1788619934.636137:0:6702:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619934 -> 1788619933 00002000:00010000:0.0:1788619934.636139:0:6702:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619934 -> 1788619934 00002000:00010000:0.0:1788619934.636142:0:6702:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 391 -> 263 00000100:00100000:0.0:1788619934.636228:0:6702:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f74a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+7:2355:x1875503571760384:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 1604us (2374us total) trans 4294967535 rc 0/0 00000100:00100000:0.0:1788619934.636239:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 821 00002000:00010000:0.0:1788619934.636244:0:6702:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (1520395->18446744073709551615) 00002000:00100000:0.0:1788619934.636248:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5f74a80 with x1875503571760384 ext(1520395->18446744073709551615) 00002000:00010000:0.0:1788619934.636254:0:6702:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5f74a80. 00000100:00100000:3.0:1788619934.640894:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619934.641261:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571760512 00000100:00100000:0.0:1788619934.641272:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571760512 00000100:00100000:0.0:1788619934.641373:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1648 00000100:00100000:0.0:1788619934.641385:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd938380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+14:14908:x1875503571760512:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619934.641409:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619934.641429:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c600/0x1cd7fe5368736f4f lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1df07 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.641533:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c800/0x1cd7fe5368736f56 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.641549:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c800/0x1cd7fe5368736f56 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.641561:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c800/0x1cd7fe5368736f56 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788619934.641592:0:9417:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1520395 00000004:00200000:0.0:1788619934.641596:0:9417:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788619934.641615:0:9417:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c800/0x1cd7fe5368736f56 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.641641:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c600/0x1cd7fe5368736f4f lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df07 expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.641660:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c800/0x1cd7fe5368736f56 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df07 expref: 15 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.641676:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619934.641679:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619934.641683:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c777c800, rc 0) 00000100:00100000:0.0:1788619934.641762:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd938380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1b0df97a-4f84-4680-9108-88353da6ca92+15:14908:x1875503571760512:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 379us (877us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619934.641769:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1648 00000100:00100000:3.0:1788619934.646234:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619934.646456:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571760640 00000100:00100000:2.0:1788619934.646467:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571760640 00000100:00100000:2.0:1788619934.646530:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1649 00000100:00100000:2.0:1788619934.646539:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd939c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+12:14908:x1875503571760640:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619934.646561:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619934.646582:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e600/0x1cd7fe5368736f5d lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1df0e expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.646669:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3ea00/0x1cd7fe5368736f64 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.646685:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3ea00/0x1cd7fe5368736f64 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.646697:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3ea00/0x1cd7fe5368736f64 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788619934.646724:0:11397:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 1520395 00000004:00200000:2.0:1788619934.646728:0:11397:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788619934.646756:0:11397:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3ea00/0x1cd7fe5368736f64 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.646783:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e600/0x1cd7fe5368736f5d lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df0e expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.646800:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3ea00/0x1cd7fe5368736f64 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df0e expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.646808:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619934.646812:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788619934.646816:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6a3ea00, rc 0) 00000100:00100000:2.0:1788619934.646876:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd939c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571760640:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 337us (651us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619934.646883:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1649 00000100:00100000:2.0:1788619934.652730:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619934.652776:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619934.652871:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571760768 00000100:00100000:0.0:1788619934.652882:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571760768 00000100:00100000:0.0:1788619934.652945:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1650 00000100:00100000:0.0:1788619934.652955:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64adf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571760768:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619934.652976:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619934.652996:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c600/0x1cd7fe5368736f6b lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1df15 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.653045:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e000/0x1cd7fe5368736ee6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1deb3 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619934.653087:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f75f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589279104:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619934.653167:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5eafa80 going to sleep for 11 seconds 00000100:00100000:3.0:1788619934.655118:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571760896 00000100:00100000:3.0:1788619934.655130:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571760896 00000100:00100000:3.0:1788619934.655189:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1347 00000100:00100000:3.0:1788619934.655199:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64ad500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571760896:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619934.655215:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619934.655232:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000034a58768/0x1cd7fe5368736f72 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1df1c expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619934.655285:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fd939880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589279232:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619934.655410:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7780 going to sleep for 11 seconds 00000100:00100000:3.0:1788619934.656436:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f75f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589279104:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619934.656457:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 757380 00010000:00010000:3.0:1788619934.656484:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c600/0x1cd7fe5368736f6b lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df15 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.656505:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619934.656508:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619934.656514:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c600/0x1cd7fe5368736f6b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df15 expref: 14 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.656528:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c777c600, rc 0) 00000100:00100000:3.0:1788619934.656581:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64adf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571760768:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3627us (3862us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619934.656589:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1650 00002000:00010000:3.0:1788619934.658863:0:6698:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime: 1788619933 -> 1788619934 00000100:00100000:3.0:1788619934.658894:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fd939880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589279232:192.168.201.5@tcp:106: 00010000:00010000:3.0:1788619934.658936:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000034a58768/0x1cd7fe5368736f72 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df1c expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.658969:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000034a58768/0x1cd7fe5368736f72 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df1c expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.658982:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa06600, rc 0) 00000100:00100000:3.0:1788619934.659046:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64ad500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571760896:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3848us (6268us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619934.659056:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1347 00000100:00100000:3.0:1788619934.662254:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619934.662294:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619934.662441:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571761024 00000100:00100000:1.0:1788619934.662452:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571761024 00000100:00100000:1.0:1788619934.662521:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1651 00000100:00100000:1.0:1788619934.662530:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd939880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571761024:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619934.662549:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619934.662569:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368736f79 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1df23 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.662605:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e000/0x1cd7fe5368736ee6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1deb3 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619934.662646:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fe063800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589279360:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619934.662723:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78e6e80c00 going to sleep for 11 seconds 00000100:00100000:1.0:1788619934.665469:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe063800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589279360:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619934.665506:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 757380 00010000:00010000:1.0:1788619934.665525:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368736f79 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df23 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.665546:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619934.665550:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619934.665555:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368736f79 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df23 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.665571:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683de00, rc 0) 00000100:00100000:1.0:1788619934.665628:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd939880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571761024:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3098us (3383us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619934.665636:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1651 00000100:00100000:2.0:1788619934.667148:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571761152 00000100:00100000:2.0:1788619934.667159:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571761152 00000100:00100000:2.0:1788619934.667262:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1348 00000100:00100000:2.0:1788619934.667273:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd93b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571761152:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619934.667290:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619934.667315:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000021f58f35/0x1cd7fe5368736f80 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1df2a expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619934.667402:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64ad880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589279488:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619934.667485:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164300 going to sleep for 11 seconds 00000100:00100000:2.0:1788619934.669831:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64ad880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589279488:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619934.669869:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000021f58f35/0x1cd7fe5368736f80 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df2a expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.669895:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000021f58f35/0x1cd7fe5368736f80 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df2a expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.669914:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6a3ec00, rc 0) 00000100:00100000:2.0:1788619934.669971:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd93b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571761152:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2699us (7675us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619934.669981:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1348 00000100:00100000:1.0:1788619934.676193:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619934.676289:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571761280 00000100:00100000:0.0:1788619934.676542:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571761280 00000100:00100000:1.0:1788619934.676695:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1652 00000100:00100000:1.0:1788619934.676705:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe063800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571761280:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619934.676727:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619934.676760:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368736f87 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1df31 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.676794:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e000/0x1cd7fe5368736ee6 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1deb3 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.676805:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368736f87 lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df31 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.676839:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e000/0x1cd7fe5368736ee6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1deb3 expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.676854:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e000/0x1cd7fe5368736ee6 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1deb3 expref: 14 pid: 11397 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619934.676876:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fe060380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589279616:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619934.676965:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78e6e80c00 going to sleep for 11 seconds 00000100:00100000:1.0:1788619934.681186:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe060380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589279616:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619934.681227:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368736f87 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df31 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.681249:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619934.681252:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619934.681257:0:5844:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368736f87 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df31 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.681269:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683dc00, rc 0) 00000100:00100000:1.0:1788619934.681358:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe063800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571761280:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4653us (5172us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619934.681367:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1652 00000100:00100000:3.0:1788619934.682391:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619934.682539:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571761408 00000100:00100000:2.0:1788619934.682548:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571761408 00010000:00010000:2.0:1788619934.682583:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e000/0x1cd7fe5368736ee6 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1deb3 expref: 15 pid: 11397 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619934.682613:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 552 00000100:00100000:2.0:1788619934.682621:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd939180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+15:2354:x1875503571761408:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619934.682636:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619934.682668:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e000/0x1cd7fe5368736ee6 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1deb3 expref: 15 pid: 11397 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.682681:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e000/0x1cd7fe5368736ee6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1deb3 expref: 15 pid: 11397 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.682706:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e000/0x1cd7fe5368736ee6 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1deb3 expref: 15 pid: 11397 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.682724:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619934.682730:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368736f87 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df31 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.682753:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368736f87 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df31 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.682765:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368736f87 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df31 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.682805:0:13330:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368736f87 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df31 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619934.682818:0:13330:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64ad880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589279744:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619934.682887:0:13330:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164300 going to sleep for 11 seconds 00000100:00100000:3.0:1788619934.685848:0:13330:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64ad880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-MDT0000_UUID:13330:1875503589279744:192.168.201.5@tcp:105: 00010000:00010000:3.0:1788619934.685875:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619934.685936:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd939180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:1b0df97a-4f84-4680-9108-88353da6ca92+14:2354:x1875503571761408:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3313us (3547us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619934.685947:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 552 00000100:00100000:3.0:1788619934.691832:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619934.691976:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571761536 00000100:00100000:1.0:1788619934.691987:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571761536 00002000:00100000:1.0:1788619934.692047:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fd938e00 x1875503571761536/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:30/0 lens 488/0 e 0 to 0 dl 1788619945 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619934.692058:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fd938e00 with x1875503571761536 00000100:00100000:1.0:1788619934.692100:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1018 00000100:00100000:1.0:1788619934.692110:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd938e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571761536:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619934.700943:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd938e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571761536:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 8821us (9107us total) trans 0 rc 757760/757760 00000100:00100000:1.0:1788619934.700963:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1018 00002000:00100000:1.0:1788619934.700976:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78fd938e00 x1875503571761536/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:30/0 lens 488/440 e 0 to 0 dl 1788619945 ref 1 fl Complete:/600/0 rc 757760/757760 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619934.700987:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78fd938e00 with x1875503571761536 00000100:00100000:0.0:1788619934.704410:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619934.704505:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571761664 00000100:00100000:1.0:1788619934.704515:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571761664 00000100:00100000:1.0:1788619934.704572:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1349 00000100:00100000:1.0:1788619934.704582:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f76300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571761664:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619934.704602:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619934.704620:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003b84bbc9/0x1cd7fe5368736f8e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1df38 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619934.704682:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fe060380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589279872:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619934.704779:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78e6e80c00 going to sleep for 11 seconds 00000100:00100000:1.0:1788619934.707788:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe060380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589279872:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619934.707827:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003b84bbc9/0x1cd7fe5368736f8e lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df38 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.707853:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b84bbc9/0x1cd7fe5368736f8e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df38 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.707865:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683de00, rc 0) 00000100:00100000:1.0:1788619934.708190:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f76300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571761664:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3607us (3789us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619934.708203:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1349 00000100:00100000:0.0:1788619934.711296:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619934.711627:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571761792 00000100:00100000:2.0:1788619934.711638:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571761792 00000100:00100000:2.0:1788619934.711730:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1350 00000100:00100000:2.0:1788619934.711741:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f76a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571761792:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619934.711772:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619934.711792:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007753fc62/0x1cd7fe5368736f95 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1df3f expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619934.711855:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64adc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589280000:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619934.712121:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619934.714269:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64adc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589280000:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619934.714320:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007753fc62/0x1cd7fe5368736f95 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df3f expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.714376:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007753fc62/0x1cd7fe5368736f95 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df3f expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.714391:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6a3e200, rc 0) 00000100:00100000:2.0:1788619934.714447:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f76a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571761792:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2707us (3161us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619934.714455:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1350 00000100:00100000:3.0:1788619934.717178:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619934.717518:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571761920 00000100:00100000:0.0:1788619934.717527:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571761920 00000100:00100000:0.0:1788619934.717602:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1653 00000100:00100000:0.0:1788619934.717613:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd93b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+14:2353:x1875503571761920:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619934.717634:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619934.717656:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c200/0x1cd7fe5368736f9c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1df46 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.717691:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c200/0x1cd7fe5368736f9c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df46 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1788619934.717704:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 757380 00010000:00010000:0.0:1788619934.717711:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c200/0x1cd7fe5368736f9c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df46 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.717725:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619934.717728:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1788619934.717732:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c777c200, rc 0) 00000100:00100000:0.0:1788619934.717814:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd93b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+15:2353:x1875503571761920:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 203us (642us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619934.717821:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1653 00000100:00100000:1.0:1788619934.721195:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619934.721291:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571762048 00000100:00100000:3.0:1788619934.722885:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571762048 00000100:00100000:3.0:1788619934.723025:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1351 00000100:00100000:3.0:1788619934.723037:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe060380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571762048:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619934.723057:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619934.723074:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000034a58768/0x1cd7fe5368736fa3 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1df4d expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619934.723137:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fd938a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589280128:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619934.723228:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7780 going to sleep for 11 seconds 00000100:00100000:2.0:1788619934.725739:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fd938a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589280128:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619934.725790:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000034a58768/0x1cd7fe5368736fa3 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df4d expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.725816:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000034a58768/0x1cd7fe5368736fa3 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df4d expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.725829:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa06600, rc 0) 00000100:00100000:2.0:1788619934.725890:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe060380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571762048:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2854us (4704us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619934.725899:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1351 00000100:00100000:0.0:1788619934.734648:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619934.734812:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571762176 00000100:00100000:1.0:1788619934.734822:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571762176 00000100:00100000:1.0:1788619934.734966:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1352 00000100:00100000:1.0:1788619934.734978:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f76d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571762176:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619934.734995:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619934.735084:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003b84bbc9/0x1cd7fe5368736faa lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1df54 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619934.735145:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fe060000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589280256:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619934.735652:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78e6e80c00 going to sleep for 11 seconds 00000100:00100000:0.0:1788619934.742601:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe060000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589280256:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619934.742648:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003b84bbc9/0x1cd7fe5368736faa lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df54 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.742674:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b84bbc9/0x1cd7fe5368736faa lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df54 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.742694:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683de00, rc 0) 00000100:00100000:0.0:1788619934.742867:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f76d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571762176:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7890us (8227us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619934.742877:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1352 00000100:00100000:3.0:1788619934.747715:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619934.747833:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571762304 00000100:00100000:1.0:1788619934.747844:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571762304 00000100:00100000:1.0:1788619934.747915:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1353 00000100:00100000:1.0:1788619934.747926:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd93aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571762304:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619934.747946:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619934.747965:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000095a50c3e/0x1cd7fe5368736fb1 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1df5b expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619934.748032:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fe061f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589280384:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619934.748112:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78e6e80c00 going to sleep for 11 seconds 00000100:00100000:1.0:1788619934.750844:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe061f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589280384:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619934.750887:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000095a50c3e/0x1cd7fe5368736fb1 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df5b expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.750929:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000095a50c3e/0x1cd7fe5368736fb1 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df5b expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.750943:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d600, rc 0) 00000100:00100000:1.0:1788619934.751590:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd93aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571762304:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3663us (3883us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619934.751599:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1353 00000100:00100000:3.0:1788619934.754957:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619934.755154:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571762432 00000100:00100000:2.0:1788619934.755163:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571762432 00000100:00100000:2.0:1788619934.755223:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1354 00000100:00100000:2.0:1788619934.755232:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd93a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:14908:x1875503571762432:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619934.755250:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619934.755269:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007753fc62/0x1cd7fe5368736fb8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1df62 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.755300:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000c60d9008/0x1cd7fe5368736eed lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1deba expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.755309:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(13 11) 00010000:00010000:2.0:1788619934.755314:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(14 11) 00010000:00010000:2.0:1788619934.755319:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007753fc62/0x1cd7fe5368736fb8 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1224704->1228799] (req 1224704->1228799) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df62 expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.755391:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000c60d9008/0x1cd7fe5368736eed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1deba expref: 6 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.755405:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000c60d9008/0x1cd7fe5368736eed lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1deba expref: 6 pid: 7654 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619934.755419:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64adc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589280512:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619934.755498:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619934.759559:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619934.759636:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571762560 00000100:00100000:1.0:1788619934.759645:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571762560 00010000:00010000:1.0:1788619934.759685:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000c60d9008/0x1cd7fe5368736eed lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1deba expref: 7 pid: 7654 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619934.759735:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 553 00000100:00100000:1.0:1788619934.759745:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f74380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571762560:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619934.759768:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619934.759794:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000c60d9008/0x1cd7fe5368736eed lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1deba expref: 7 pid: 7654 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.759809:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000c60d9008/0x1cd7fe5368736eed lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1deba expref: 7 pid: 7654 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.759837:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(14 11) 00010000:00010000:1.0:1788619934.759843:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(14 11) 00010000:00010000:1.0:1788619934.759852:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000007753fc62/0x1cd7fe5368736fb8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1224704->1228799) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df62 expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.759864:0:5829:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007753fc62/0x1cd7fe5368736fb8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1224704->1228799) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df62 expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.759906:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000007753fc62/0x1cd7fe5368736fb8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1224704->1228799) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df62 expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619934.759923:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fe061f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1875503589280640:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619934.760021:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78e6e80c00 going to sleep for 11 seconds 00010000:00010000:2.0:1788619934.760598:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c60d9008/0x1cd7fe5368736eed lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1deba expref: 7 pid: 7654 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619934.760615:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64adc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589280512:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619934.760641:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007753fc62/0x1cd7fe5368736fb8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1224704->1228799) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df62 expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.760653:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6a3e200, rc 0) 00000100:00100000:2.0:1788619934.760697:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd93a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:14908:x1875503571762432:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5466us (5748us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619934.760705:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1354 00000100:00100000:0.0:1788619934.767434:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619934.767485:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619934.767698:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe061f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1875503589280640:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619934.767725:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619934.767798:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f74380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571762560:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 8054us (8247us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619934.767810:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 553 00000100:00100000:3.0:1788619934.767914:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571762688 00000100:00100000:3.0:1788619934.767924:0:6701:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571762688 00002000:00100000:3.0:1788619934.767971:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1523711) req@ffff8e77c5f77b80 x1875503571762688/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:30/0 lens 488/0 e 0 to 0 dl 1788619945 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619934.767981:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5f77b80 with x1875503571762688 ext(1048576->1523711) 00002000:00010000:3.0:1788619934.767986:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5f77b80 00000100:00100000:3.0:1788619934.768029:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 822 00000100:00100000:3.0:1788619934.768038:0:6701:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f77b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571762688:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619934.771772:0:13346:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571762816 00000100:00100000:2.0:1788619934.771789:0:13346:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571762816 00002000:00100000:2.0:1788619934.771848:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5f75f80 x1875503571762816/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:30/0 lens 488/0 e 0 to 0 dl 1788619945 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619934.771858:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5f75f80 with x1875503571762816 00000100:00100000:2.0:1788619934.771902:0:13346:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1019 00000100:00100000:2.0:1788619934.771911:0:13346:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f75f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571762816:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:3.0:1788619934.778641:0:6701:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f77b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571762688:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 10591us (11204us total) trans 0 rc 475136/475136 00000100:00100000:3.0:1788619934.778656:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 822 00002000:00100000:3.0:1788619934.778671:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (1048576->1523711) req@ffff8e77c5f77b80 x1875503571762688/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:30/0 lens 488/440 e 0 to 0 dl 1788619945 ref 1 fl Complete:/600/0 rc 475136/475136 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619934.778682:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5f77b80 with x1875503571762688 ext(1048576->1523711) 00002000:00010000:3.0:1788619934.778689:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5f77b80 00000100:00100000:2.0:1788619934.779181:0:13346:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f75f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571762816:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 7270us (11694us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619934.779190:0:13346:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1019 00002000:00100000:2.0:1788619934.779197:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c5f75f80 x1875503571762816/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:30/0 lens 488/440 e 0 to 0 dl 1788619945 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619934.779206:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5f75f80 with x1875503571762816 00000100:00100000:0.0:1788619934.784855:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619934.784990:0:13347:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571762944 00000100:00100000:2.0:1788619934.785068:0:13347:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571762944 00002000:00100000:2.0:1788619934.785120:0:13347:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x4:0x0] (1224704->1257471) req@ffff8e77c5f77800 x1875503571762944/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:30/0 lens 488/0 e 0 to 0 dl 1788619945 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619934.785130:0:13347:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5f77800 with x1875503571762944 ext(1224704->1257471) 00002000:00010000:2.0:1788619934.785136:0:13347:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5f77800 00000100:00100000:2.0:1788619934.785172:0:13347:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 823 00000100:00100000:2.0:1788619934.785182:0:13347:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f77800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571762944:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:2.0:1788619934.789221:0:13347:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f77800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2353:x1875503571762944:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 4030us (4367us total) trans 4294967536 rc 0/0 00000100:00100000:2.0:1788619934.789234:0:13347:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 823 00002000:00100000:2.0:1788619934.789246:0:13347:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x4:0x0] (1224704->1257471) req@ffff8e77c5f77800 x1875503571762944/t4294967536(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:30/0 lens 488/448 e 0 to 0 dl 1788619945 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619934.789263:0:13347:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c5f77800 with x1875503571762944 ext(1224704->1257471) 00002000:00010000:2.0:1788619934.789270:0:13347:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c5f77800 00000100:00100000:0.0:1788619934.793737:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619934.793968:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571763072 00000100:00100000:2.0:1788619934.793979:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571763072 00000100:00100000:2.0:1788619934.794044:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1355 00000100:00100000:2.0:1788619934.794054:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900986c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2352:x1875503571763072:12345-192.168.201.5@tcp:16:fsx.0 00000100:00100000:1.0:1788619934.818498:0:3317:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fe061f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3317:1875503589280768:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1788619934.818574:0:3317:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619934.818652:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589280768 00000100:00100000:1.0:1788619934.818661:0:6699:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589280768 00000100:00100000:1.0:1788619934.818719:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 94 00000100:00100000:1.0:1788619934.818727:0:6699:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe061180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3317:x1875503589280768:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1788619934.818776:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3856662528 avail 3854565376 00000100:00100000:1.0:1788619934.818823:0:6699:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe061180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3317:x1875503589280768:12345-0@lo:13:osp-pre-1-0.0 Request processed in 96us (249us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619934.818830:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 94 00000100:00100000:1.0:1788619934.819082:0:3317:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe061f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3317:1875503589280768:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1788619934.830398:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900986c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2352:x1875503571763072:12345-192.168.201.5@tcp:16:fsx.0 Request processed in 36335us (36662us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619934.830410:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1355 00000001:00080000:3.0:1788619934.830543:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967536 is committed 00000001:00080000:3.0:1788619934.830560:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967535 is committed 00000100:00100000:3.0:1788619934.834065:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619934.834131:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571763200 00000100:00100000:0.0:1788619934.834140:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571763200 00000100:00100000:0.0:1788619934.834191:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1356 00000100:00100000:0.0:1788619934.834198:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd93ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571763200:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619934.834214:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619934.834237:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000039b3144a/0x1cd7fe5368736fbf lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1df69 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619934.834301:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f75500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589280896:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619934.834422:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5eafd80 going to sleep for 11 seconds 00000100:00100000:2.0:1788619934.838937:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f75500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589280896:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619934.838980:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000039b3144a/0x1cd7fe5368736fbf lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df69 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.839004:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000039b3144a/0x1cd7fe5368736fbf lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df69 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.839017:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c777d400, rc 0) 00000100:00100000:2.0:1788619934.839079:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd93ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571763200:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4880us (5024us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619934.839088:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1356 00000100:00100000:3.0:1788619934.845132:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619934.845209:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571763328 00000100:00100000:1.0:1788619934.845220:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571763328 00000100:00100000:1.0:1788619934.845282:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1357 00000100:00100000:1.0:1788619934.845293:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd93b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571763328:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619934.845320:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619934.845371:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a72c5200/0x1cd7fe5368736fc6 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1df70 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619934.845436:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fe061f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589281024:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619934.845522:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78e6e80e40 going to sleep for 11 seconds 00000100:00100000:1.0:1788619934.848220:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe061f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589281024:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619934.848264:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a72c5200/0x1cd7fe5368736fc6 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df70 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.848291:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a72c5200/0x1cd7fe5368736fc6 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df70 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.848303:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c400, rc 0) 00000100:00100000:1.0:1788619934.848401:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd93b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571763328:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3108us (3278us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619934.848410:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1357 00000100:00100000:0.0:1788619934.853928:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619934.854128:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571763456 00000100:00100000:2.0:1788619934.854138:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571763456 00000100:00100000:2.0:1788619934.854199:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1358 00000100:00100000:2.0:1788619934.854210:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f75880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571763456:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619934.854229:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619934.854246:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000095c7ad35/0x1cd7fe5368736fcd lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1df77 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619934.854364:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64ae300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589281152:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619934.854440:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81646c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619934.857528:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64ae300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589281152:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619934.857569:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000095c7ad35/0x1cd7fe5368736fcd lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df77 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.857595:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000095c7ad35/0x1cd7fe5368736fcd lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df77 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.857608:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6a3fc00, rc 0) 00000100:00100000:2.0:1788619934.857681:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f75880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571763456:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3473us (3763us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619934.857689:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1358 00000100:00100000:3.0:1788619934.861272:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619934.861426:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571763584 00000100:00100000:2.0:1788619934.861439:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571763584 00000100:00100000:2.0:1788619934.861521:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1359 00000100:00100000:2.0:1788619934.861532:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd93bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571763584:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619934.861551:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619934.861571:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b9db1e03/0x1cd7fe5368736fd4 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1df7e expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619934.861652:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64ae300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589281280:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619934.861729:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81646c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619934.864810:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64ae300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589281280:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619934.864852:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b9db1e03/0x1cd7fe5368736fd4 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df7e expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.865047:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b9db1e03/0x1cd7fe5368736fd4 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df7e expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.865063:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6a3e800, rc 0) 00000100:00100000:2.0:1788619934.865130:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd93bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2352:x1875503571763584:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3600us (3869us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619934.865139:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1359 00000100:00100000:0.0:1788619934.872233:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619934.872441:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571763712 00000100:00100000:1.0:1788619934.872452:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571763712 00000100:00100000:1.0:1788619934.872513:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1654 00000100:00100000:1.0:1788619934.872524:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f74e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+15:14908:x1875503571763712:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619934.872544:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619934.872547:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:1.0:1788619934.872592:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c200/0x1cd7fe5368736f9c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df46 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.872611:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619934.872617:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788619934.872642:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368736fdb lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1df85 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.872668:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368736f87 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df31 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.872678:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368736fdb lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df85 expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.872711:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368736f87 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df31 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.872724:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368736f87 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df31 expref: 13 pid: 5844 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619934.872749:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fe061f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589281408:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619934.872839:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78e6e80e40 going to sleep for 11 seconds 00000100:00100000:1.0:1788619934.878079:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe061f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589281408:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619934.878123:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368736fdb lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df85 expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.878148:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619934.878151:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619934.878156:0:5845:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368736fdb lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df85 expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.878167:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c400, rc 0) 00000100:00100000:1.0:1788619934.878226:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f74e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+15:14908:x1875503571763712:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 5704us (6005us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619934.878236:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1654 00000100:00100000:2.0:1788619934.885296:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619934.885616:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571763840 00000100:00100000:3.0:1788619934.885637:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571763840 00010000:00010000:2.0:1788619934.885991:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368736f87 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df31 expref: 14 pid: 5844 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619934.886071:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 554 00000100:00100000:2.0:1788619934.886082:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64ae300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571763840:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619934.886099:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619934.886128:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368736f87 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df31 expref: 14 pid: 5844 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.886148:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368736f87 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df31 expref: 14 pid: 5844 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.886171:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683dc00/0x1cd7fe5368736f87 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df31 expref: 14 pid: 5844 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.886188:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619934.886195:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368736fdb lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df85 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.886207:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368736fdb lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df85 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.886217:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368736fdb lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df85 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.886269:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368736fdb lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df85 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619934.886283:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64af100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589281536:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619934.886412:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81646c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619934.890245:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619934.890383:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64af100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:5830:1875503589281536:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619934.890406:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619934.890457:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64ae300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571763840:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 4376us (5173us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619934.890484:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 554 00000100:00100000:1.0:1788619934.890595:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571763968 00000100:00100000:1.0:1788619934.890603:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571763968 00000100:00100000:1.0:1788619934.890665:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1020 00000100:00100000:1.0:1788619934.890675:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f75500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+15:2354:x1875503571763968:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:1.0:1788619934.890886:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 757380 -> 757380 00000004:00010000:1.0:1788619934.890890:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619934 -> 1788619934 00000004:00010000:1.0:1788619934.890893:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619934 -> 1788619934 00000004:00010000:1.0:1788619934.890895:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619934 -> 1788619934 00000004:00010000:1.0:1788619934.890898:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 774 -> 774 00000100:00100000:1.0:1788619934.890934:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f75500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+16:2354:x1875503571763968:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 261us (699us total) trans 4294967878 rc 0/0 00000100:00100000:1.0:1788619934.890942:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1020 00000100:00100000:0.0:1788619934.895228:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619934.897380:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571764096 00000100:00100000:2.0:1788619934.897398:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571764096 00000100:00100000:2.0:1788619934.897539:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1360 00000100:00100000:2.0:1788619934.897551:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f77480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571764096:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619934.897571:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619934.897590:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b9db1e03/0x1cd7fe5368736fe2 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1df8c expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619934.897653:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64af100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589281664:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619934.897759:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81646c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619934.900112:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64af100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589281664:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619934.900152:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b9db1e03/0x1cd7fe5368736fe2 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df8c expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.900179:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b9db1e03/0x1cd7fe5368736fe2 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df8c expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.900192:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6a3e800, rc 0) 00000100:00100000:2.0:1788619934.900247:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f77480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:2355:x1875503571764096:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2698us (5027us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619934.900271:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1360 00000100:00100000:3.0:1788619934.903855:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619934.903974:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571764224 00000100:00100000:1.0:1788619934.903985:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571764224 00000100:00100000:1.0:1788619934.904078:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1361 00000100:00100000:1.0:1788619934.904089:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd93b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571764224:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619934.904109:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619934.904128:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe5368736fe9 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1df93 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619934.904190:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fe061f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589281792:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619934.904288:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78e6e80e40 going to sleep for 11 seconds 00000100:00100000:1.0:1788619934.913964:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe061f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589281792:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619934.914005:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe5368736fe9 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df93 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.914032:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e488899f/0x1cd7fe5368736fe9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df93 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.914046:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c600, rc 0) 00000100:00100000:1.0:1788619934.914144:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd93b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571764224:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 10056us (10299us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619934.914155:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1361 00000100:00100000:0.0:1788619934.918422:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619934.918485:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571764352 00000100:00100000:1.0:1788619934.918496:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571764352 00000100:00100000:1.0:1788619934.918559:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1655 00000100:00100000:1.0:1788619934.918569:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f77100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571764352:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619934.918590:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619934.918611:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368736ff0 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1df9a expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.918650:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368736fdb lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df85 expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.918662:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368736ff0 lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df9a expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.918697:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368736fdb lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df85 expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.918710:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368736fdb lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df85 expref: 15 pid: 5845 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619934.918725:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fe061f80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589281920:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619934.918819:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78e6e80e40 going to sleep for 11 seconds 00000100:00100000:3.0:1788619934.923188:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619934.923297:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571764480 00000100:00100000:2.0:1788619934.923307:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571764480 00010000:00010000:2.0:1788619934.923383:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368736fdb lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df85 expref: 16 pid: 5845 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619934.923427:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 555 00000100:00100000:2.0:1788619934.923438:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd93aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+16:2354:x1875503571764480:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619934.923453:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619934.923491:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368736fdb lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df85 expref: 16 pid: 5845 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.923506:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368736fdb lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df85 expref: 16 pid: 5845 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.923531:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619934.923538:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368736ff0 lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df9a expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.923551:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368736ff0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df9a expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.923560:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368736ff0 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df9a expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.923598:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368736ff0 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df9a expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619934.923610:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64af100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589282048:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619934.923690:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81646c0 going to sleep for 11 seconds 00010000:00010000:1.0:1788619934.925513:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c400/0x1cd7fe5368736fdb lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df85 expref: 16 pid: 5845 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619934.925539:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe061f80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589281920:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619934.925571:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368736ff0 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df9a expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.925587:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619934.925591:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619934.925595:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d400, rc 0) 00000100:00100000:1.0:1788619934.925709:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f77100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571764352:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 7140us (7296us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619934.925720:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1655 00000100:00100000:0.0:1788619934.932175:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619934.932266:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571764608 00000100:00100000:1.0:1788619934.932277:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571764608 00000100:00100000:1.0:1788619934.932382:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1021 00000100:00100000:1.0:1788619934.932393:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f76300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571764608:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:1.0:1788619934.932641:0:5858:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 757380 -> 757380 00000004:00010000:1.0:1788619934.932645:0:5858:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619934 -> 1788619934 00000004:00010000:1.0:1788619934.932648:0:5858:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619934 -> 1788619934 00000004:00010000:1.0:1788619934.932650:0:5858:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619934 -> 1788619934 00000004:00010000:1.0:1788619934.932652:0:5858:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 774 -> 774 00000100:00100000:1.0:1788619934.932725:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f76300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571764608:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 333us (560us total) trans 4294967879 rc 0/0 00000100:00100000:1.0:1788619934.932733:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1021 00000100:00100000:2.0:1788619934.933696:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64af100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589282048:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619934.933720:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619934.934087:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd93aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+15:2354:x1875503571764480:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 10649us (10910us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619934.934098:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 555 00000100:00100000:0.0:1788619934.939381:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619934.939429:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619934.939669:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571764736 00000100:00100000:3.0:1788619934.939680:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571764736 00000100:00100000:3.0:1788619934.939749:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1656 00000100:00100000:3.0:1788619934.939761:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f76d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+15:2354:x1875503571764736:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619934.939783:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619934.939803:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe5368736ff7 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dfa1 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.939848:0:5846:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368736ff0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df9a expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619934.939891:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fd93a680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589282176:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619934.939992:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e70c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619934.940201:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571764864 00000100:00100000:2.0:1788619934.940207:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571764864 00000100:00100000:2.0:1788619934.940246:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1362 00000100:00100000:2.0:1788619934.940255:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f76a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571764864:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619934.940264:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619934.940275:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b9db1e03/0x1cd7fe5368736ffe lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dfa8 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619934.940322:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64af100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589282304:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619934.940387:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81646c0 going to sleep for 11 seconds 00000100:00100000:2.0:1788619934.944692:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64af100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589282304:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619934.944734:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b9db1e03/0x1cd7fe5368736ffe lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dfa8 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.944774:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b9db1e03/0x1cd7fe5368736ffe lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dfa8 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.944788:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6a3e800, rc 0) 00000100:00100000:2.0:1788619934.944849:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f76a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+5:2354:x1875503571764864:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4594us (5476us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619934.944862:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1362 00000100:00100000:3.0:1788619934.948929:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fd93a680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589282176:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619934.948955:0:5846:0:(mdt_io.c:1550:mdt_lvb2reply()) size 757380 00010000:00010000:3.0:1788619934.948976:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe5368736ff7 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dfa1 expref: 16 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.948999:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619934.949002:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619934.949008:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe5368736ff7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dfa1 expref: 16 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.949024:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa06400, rc 0) 00000100:00100000:3.0:1788619934.949086:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f76d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+15:2354:x1875503571764736:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 9327us (9655us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619934.949094:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1656 00000100:00100000:1.0:1788619934.952192:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619934.952238:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619934.952365:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571765120 00000100:00100000:2.0:1788619934.952374:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571765120 00000100:00100000:2.0:1788619934.952425:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1363 00000100:00100000:2.0:1788619934.952436:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe062300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571765120:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619934.952455:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619934.952483:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b9db1e03/0x1cd7fe5368737005 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dfb6 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619934.952542:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64af100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589282432:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619934.952619:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c81646c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619934.952782:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571764992 00000100:00100000:0.0:1788619934.952787:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571764992 00000100:00100000:0.0:1788619934.952828:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1657 00000100:00100000:0.0:1788619934.952836:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe061f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+15:2353:x1875503571764992:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:0.0:1788619934.952846:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788619934.952858:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d800/0x1cd7fe536873700c lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dfaf expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.952896:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368736ff0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df9a expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619934.952923:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f4a80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589282560:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619934.952956:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5eafa80 going to sleep for 11 seconds 00000100:00100000:1.0:1788619934.955369:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f4a80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589282560:192.168.201.5@tcp:106: 00002000:00010000:1.0:1788619934.955398:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 757380 00010000:00010000:1.0:1788619934.955416:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d800/0x1cd7fe536873700c lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dfaf expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.955436:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619934.955440:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619934.955446:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d800/0x1cd7fe536873700c lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dfaf expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.955460:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c777d800, rc 0) 00000100:00100000:1.0:1788619934.955529:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe061f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+15:2353:x1875503571764992:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2693us (3345us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619934.955538:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1657 00000100:00100000:2.0:1788619934.956127:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64af100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589282432:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619934.956150:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b9db1e03/0x1cd7fe5368737005 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dfb6 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.956165:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b9db1e03/0x1cd7fe5368737005 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dfb6 expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619934.956175:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6a3e800, rc 0) 00000100:00100000:2.0:1788619934.956214:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe062300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1b0df97a-4f84-4680-9108-88353da6ca92+5:2353:x1875503571765120:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3779us (3976us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619934.956222:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1363 00000100:00100000:0.0:1788619934.959660:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619934.959736:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571765376 00000100:00100000:1.0:1788619934.959753:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571765376 00000100:00100000:1.0:1788619934.959811:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1658 00000100:00100000:1.0:1788619934.959821:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+15:14908:x1875503571765376:12345-192.168.201.5@tcp:36:fsx.0 00010000:00010000:1.0:1788619934.959874:0:5844:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619934.959900:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c800/0x1cd7fe5368736f56 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df07 expref: 15 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.959926:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619934.959931:0:5844:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788619934.960128:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368737013 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.960144:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3ea00/0x1cd7fe5368736f64 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df0e expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.960153:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368737013 lrc: 4/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.960186:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3ea00/0x1cd7fe5368736f64 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df0e expref: 14 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.960200:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3ea00/0x1cd7fe5368736f64 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df0e expref: 14 pid: 11397 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619934.960214:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fe061c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589282688:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788619934.960293:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78defaa0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1788619934.962810:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe061c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589282688:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619934.962840:0:5844:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368737013 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619934.964524:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619934.964865:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571765504 00000100:00100000:3.0:1788619934.964874:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571765504 00010000:00010000:3.0:1788619934.965006:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3ea00/0x1cd7fe5368736f64 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df0e expref: 15 pid: 11397 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619934.965084:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 556 00000100:00100000:3.0:1788619934.965095:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f7480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571765504:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619934.965109:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619934.965116:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3ea00/0x1cd7fe5368736f64 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df0e expref: 15 pid: 11397 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.965129:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3ea00/0x1cd7fe5368736f64 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df0e expref: 15 pid: 11397 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.965150:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3ea00/0x1cd7fe5368736f64 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df0e expref: 15 pid: 11397 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.965165:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619934.965170:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368737013 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.965180:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368737013 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.965187:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368737013 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.965217:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619934.965351:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f7480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571765504:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 234us (808us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619934.965359:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 556 00010000:00010000:1.0:1788619934.965418:0:5844:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368737013 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.965427:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368737013 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1788619934.965546:0:5844:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967880 00000004:00010000:1.0:1788619934.965595:0:5844:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 757380 -> 757380 00000004:00010000:1.0:1788619934.965598:0:5844:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619934 -> 1788619934 00000004:00010000:1.0:1788619934.965601:0:5844:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619934 -> 1788619934 00000004:00010000:1.0:1788619934.965604:0:5844:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619934 -> 1788619934 00000004:00010000:1.0:1788619934.965606:0:5844:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 774 -> 774 00000004:00010000:1.0:1788619934.965613:0:5844:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000002160229c reply state 00000000f44345b4 transno 4294967881 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368737013 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619934.965620:0:5844:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788619934.965683:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+17:14908:x1875503571765376:12345-192.168.201.5@tcp:36:fsx.0 Request processed in 5864us (6032us total) trans 4294967881 rc 0/0 00000100:00100000:1.0:1788619934.965691:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1658 00000020:00080000:1.0:1788619934.965695:0:5844:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1788619934.968647:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619934.969128:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571765632 00000100:00100000:3.0:1788619934.969138:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571765632 00000100:00100000:3.0:1788619934.969196:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1659 00000100:00100000:3.0:1788619934.969207:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64af100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+17:14908:x1875503571765632:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619934.969232:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619934.969252:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe536873701a lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dfbd expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.969282:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368736ff0 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df9a expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.969293:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe536873701a lrc: 4/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dfbd expref: 18 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.969362:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368736ff0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df9a expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.969376:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368736ff0 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df9a expref: 13 pid: 5844 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619934.969389:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fd93a680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589282816:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619934.969461:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e70c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788619934.971364:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619934.971388:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571765760 00000100:00100000:3.0:1788619934.971393:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571765760 00010000:00010000:3.0:1788619934.971419:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368736ff0 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df9a expref: 14 pid: 5844 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619934.971446:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 557 00000100:00100000:3.0:1788619934.971454:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c80e8450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571765760:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619934.971467:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619934.971502:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368736ff0 lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df9a expref: 14 pid: 5844 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.971514:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368736ff0 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df9a expref: 14 pid: 5844 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.971534:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619934.971540:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe536873701a lrc: 3/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dfbd expref: 18 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.971551:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe536873701a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dfbd expref: 18 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.971561:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe536873701a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dfbd expref: 18 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.971592:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe536873701a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dfbd expref: 18 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619934.971603:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fd938a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589282944:192.168.201.5@tcp:105: 00000100:00100000:3.0:1788619934.971650:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7d80 going to sleep for 11 seconds 00010000:00010000:1.0:1788619934.971746:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe5368736ff0 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df9a expref: 14 pid: 5844 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619934.971761:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fd93a680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589282816:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788619934.971784:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe536873701a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dfbd expref: 18 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.971794:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619934.971796:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619934.971799:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa06400, rc 0) 00000100:00100000:1.0:1788619934.971823:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64af100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+18:14908:x1875503571765632:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2618us (3185us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619934.971830:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1659 00000100:00100000:1.0:1788619934.974496:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619934.974592:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fd938a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-MDT0000_UUID:5829:1875503589282944:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619934.974613:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619934.974663:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c80e8450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2355:x1875503571765760:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 3208us (3304us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619934.974674:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 557 00000100:00100000:3.0:1788619934.974820:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571765888 00000100:00100000:3.0:1788619934.974827:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571765888 00000100:00100000:3.0:1788619934.974874:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1364 00000100:00100000:3.0:1788619934.974884:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe061c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+5:14908:x1875503571765888:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619934.974897:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619934.974927:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000081ae252b/0x1cd7fe5368737021 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dfc4 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.974958:0:6696:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000007753fc62/0x1cd7fe5368736fb8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1224704->1228799) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df62 expref: 5 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.974967:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x4:0x0].0x0 since=0s v=(14 11) 00010000:00010000:3.0:1788619934.974972:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(15 11) 00010000:00010000:3.0:1788619934.974977:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000081ae252b/0x1cd7fe5368737021 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dfc4 expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.975008:0:6696:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000007753fc62/0x1cd7fe5368736fb8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1224704->1228799) gid 0 flags: 0x50000000020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df62 expref: 5 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.975020:0:6696:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000007753fc62/0x1cd7fe5368736fb8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1224704->1228799) gid 0 flags: 0x70000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df62 expref: 5 pid: 6697 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619934.975032:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fd938700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589283072:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619934.975074:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7d80 going to sleep for 11 seconds 00000100:00100000:2.0:1788619934.978364:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619934.978447:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571766016 00000100:00100000:1.0:1788619934.978455:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571766016 00010000:00010000:1.0:1788619934.978505:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000007753fc62/0x1cd7fe5368736fb8 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1224704->1228799) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df62 expref: 6 pid: 6697 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619934.978548:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 558 00000100:00100000:1.0:1788619934.978558:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64ae300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2354:x1875503571766016:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788619934.978571:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788619934.978594:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000007753fc62/0x1cd7fe5368736fb8 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1224704->1228799) gid 0 flags: 0x60000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df62 expref: 6 pid: 6697 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.978607:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000007753fc62/0x1cd7fe5368736fb8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1224704->1228799) gid 0 flags: 0x50000400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df62 expref: 6 pid: 6697 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.978629:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(15 11) 00010000:00010000:1.0:1788619934.978633:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(15 11) 00010000:00010000:1.0:1788619934.978643:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000081ae252b/0x1cd7fe5368737021 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dfc4 expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.978653:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000081ae252b/0x1cd7fe5368737021 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dfc4 expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.978689:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000081ae252b/0x1cd7fe5368737021 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dfc4 expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619934.978704:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fe060380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589283200:192.168.201.5@tcp:105: 00000100:00100000:1.0:1788619934.978783:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78e6e80c00 going to sleep for 11 seconds 00010000:00010000:3.0:1788619934.979122:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007753fc62/0x1cd7fe5368736fb8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1224704->1228799) gid 0 flags: 0x44801400020020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1df62 expref: 6 pid: 6697 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619934.979138:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fd938700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589283072:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788619934.979160:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000081ae252b/0x1cd7fe5368737021 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dfc4 expref: 6 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619934.979176:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa07c00, rc 0) 00000100:00100000:3.0:1788619934.979211:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe061c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:14908:x1875503571765888:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4328us (4724us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619934.979218:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1364 00010000:00010000:0.0:1788619934.980819:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368737013 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.980834:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368737013 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.980842:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368737013 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.980849:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c683c200 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788619934.980854:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368737013 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.980867:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368737013 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.980883:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619934.980886:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788619934.980890:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368737013 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619934.980896:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368737013 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619934.983619:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619934.983667:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe060380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1875503589283200:192.168.201.5@tcp:105: 00000100:00100000:2.0:1788619934.983685:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00010000:00010000:1.0:1788619934.983689:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788619934.983744:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64ae300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571766016:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 5187us (5420us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619934.983755:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 558 00000100:00100000:3.0:1788619934.983767:0:6703:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571766272 00000100:00100000:3.0:1788619934.983775:0:6703:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571766272 00002000:00010000:3.0:1788619934.983811:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:3.0:1788619934.983816:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64ad500 with x1875503571766272 ext(0->18446744073709551615) 00002000:00010000:3.0:1788619934.983826:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64ad500. 00000100:00100000:3.0:1788619934.983862:0:6703:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 824 00000100:00100000:3.0:1788619934.983871:0:6703:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64ad500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+6:2354:x1875503571766272:12345-192.168.201.5@tcp:10:fsx.0 00002000:00010000:3.0:1788619934.985495:0:6703:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 1520395 -> 0 00002000:00010000:3.0:1788619934.985502:0:6703:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1788619934 -> 1788619934 00002000:00010000:3.0:1788619934.985505:0:6703:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb atime from disk: 1788619934 -> 1788619934 00002000:00010000:3.0:1788619934.985508:0:6703:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1788619934 -> 1788619934 00002000:00010000:3.0:1788619934.985510:0:6703:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb blocks from disk: 263 -> 263 00000100:00100000:3.0:1788619934.985605:0:6703:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64ad500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571766272:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 1731us (1914us total) trans 4294967537 rc 0/0 00000100:00100000:3.0:1788619934.985614:0:6703:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 824 00002000:00010000:3.0:1788619934.985618:0:6703:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/4 (0->18446744073709551615) 00002000:00100000:3.0:1788619934.985621:0:6703:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff8e77c64ad500 with x1875503571766272 ext(0->18446744073709551615) 00002000:00010000:3.0:1788619934.985627:0:6703:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff8e77c64ad500. 00000100:00100000:0.0:1788619934.985708:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571766144 00000100:00100000:0.0:1788619934.985715:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571766144 00002000:00010000:1.0:1788619934.985969:0:5859:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:1.0:1788619934.985974:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64afb80 with x1875503571766144 00000100:00100000:1.0:1788619934.986001:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1022 00000100:00100000:1.0:1788619934.986010:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64afb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+17:2354:x1875503571766144:12345-192.168.201.5@tcp:10:fsx.0 00000004:00010000:1.0:1788619934.986394:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 757380 -> 49870 00000004:00010000:1.0:1788619934.986399:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619934 -> 1788619934 00000004:00010000:1.0:1788619934.986401:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619934 -> 1788619934 00000004:00010000:1.0:1788619934.986404:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619934 -> 1788619934 00000004:00010000:1.0:1788619934.986406:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 774 -> 774 00000100:00100000:1.0:1788619934.986456:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64afb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+18:2354:x1875503571766144:12345-192.168.201.5@tcp:10:fsx.0 Request processed in 447us (2847us total) trans 4294967882 rc 0/0 00000100:00100000:1.0:1788619934.986463:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1022 00002000:00010000:1.0:1788619934.986466:0:5859:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0xe:0x0] 00002000:00100000:1.0:1788619934.986478:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64afb80 with x1875503571766144 00000100:00100000:2.0:1788619934.989211:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619934.989374:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571766400 00000100:00100000:1.0:1788619934.989383:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571766400 00000100:00100000:1.0:1788619934.989441:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1660 00000100:00100000:1.0:1788619934.989451:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64adc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+18:14908:x1875503571766400:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619934.989482:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619934.989503:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368737028 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dfcb expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.989571:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe536873702f lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.989583:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe536873702f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.989591:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe536873702f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788619934.989611:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 49870 00000004:00200000:1.0:1788619934.989614:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788619934.989626:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe536873702f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.989644:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368737028 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dfcb expref: 20 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.989660:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe536873702f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dfcb expref: 19 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619934.989665:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619934.989667:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788619934.989670:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683cc00, rc 0) 00000100:00100000:1.0:1788619934.989715:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64adc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+19:14908:x1875503571766400:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 266us (513us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619934.989721:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1660 00000100:00100000:2.0:1788619934.993317:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619934.993507:0:5858:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571766528 00000100:00100000:1.0:1788619934.993518:0:5858:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571766528 00002000:00100000:1.0:1788619934.993568:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c64ad880 x1875503571766528/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:30/0 lens 488/0 e 0 to 0 dl 1788619945 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619934.993580:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64ad880 with x1875503571766528 00000100:00100000:1.0:1788619934.993614:0:5858:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1023 00000100:00100000:1.0:1788619934.993624:0:5858:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64ad880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+19:2352:x1875503571766528:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:1.0:1788619935.003290:0:5858:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64ad880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_001:1b0df97a-4f84-4680-9108-88353da6ca92+19:2352:x1875503571766528:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 9658us (9973us total) trans 0 rc 53248/53248 00000100:00100000:1.0:1788619935.003309:0:5858:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1023 00002000:00100000:1.0:1788619935.003322:0:5858:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_001: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e77c64ad880 x1875503571766528/t0(0) o3->1b0df97a-4f84-4680-9108-88353da6ca92@192.168.201.5@tcp:30/0 lens 488/440 e 0 to 0 dl 1788619945 ref 1 fl Complete:/600/0 rc 53248/53248 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1788619935.003360:0:5858:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c64ad880 with x1875503571766528 00000100:00100000:2.0:1788619935.009222:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619935.009422:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571766656 00000100:00100000:1.0:1788619935.009432:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571766656 00000100:00100000:1.0:1788619935.009508:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1024 00000100:00100000:1.0:1788619935.009519:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f74380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+19:2353:x1875503571766656:12345-192.168.201.5@tcp:22:fsx.0 00000004:00010000:1.0:1788619935.009759:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb size from disk: 49870 -> 49870 00000004:00010000:1.0:1788619935.009763:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb mtime from disk: 1788619934 -> 1788619934 00000004:00010000:1.0:1788619935.009765:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb atime from disk: 1788619934 -> 1788619934 00000004:00010000:1.0:1788619935.009768:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb ctime from disk: 1788619934 -> 1788619934 00000004:00010000:1.0:1788619935.009770:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xe:0x0] updating lvb blocks from disk: 774 -> 774 00000100:00100000:1.0:1788619935.009837:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f74380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:1b0df97a-4f84-4680-9108-88353da6ca92+20:2353:x1875503571766656:12345-192.168.201.5@tcp:22:fsx.0 Request processed in 320us (625us total) trans 4294967883 rc 0/0 00000100:00100000:1.0:1788619935.009844:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1024 00000100:00100000:3.0:1788619935.011947:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619935.012173:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571766784 00000100:00100000:1.0:1788619935.012182:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571766784 00000100:00100000:1.0:1788619935.012247:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1661 00000100:00100000:1.0:1788619935.012257:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd938700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:14908:x1875503571766784:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619935.012279:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619935.012310:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368737036 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dfd2 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619935.012440:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe536873703d lrc: 3/1,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619935.012456:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe536873703d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619935.012480:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe536873703d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788619935.012508:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xe:0x0]: returning size 49870 00000004:00200000:1.0:1788619935.012513:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788619935.012530:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe536873703d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619935.012557:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368737036 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dfd2 expref: 15 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619935.012575:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d400/0x1cd7fe536873703d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dfd2 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619935.012584:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619935.012587:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788619935.012591:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d400, rc 0) 00000100:00100000:1.0:1788619935.012652:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd938700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571766784:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 396us (714us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619935.012660:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1661 00000100:00100000:2.0:1788619935.015159:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619935.015241:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571766912 00000100:00100000:3.0:1788619935.015250:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571766912 00000100:00100000:3.0:1788619935.015305:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1662 00000100:00100000:3.0:1788619935.015315:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c71f1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2352:x1875503571766912:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619935.015363:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619935.015382:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07400/0x1cd7fe5368737044 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dfd9 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619935.015416:0:11397:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe536873701a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dfbd expref: 19 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619935.015453:0:11397:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fd939180 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589283328:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619935.015536:0:11397:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7d80 going to sleep for 11 seconds 00000100:00100000:2.0:1788619935.017370:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619935.017496:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571767040 00000100:00100000:1.0:1788619935.017504:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571767040 00000100:00100000:1.0:1788619935.017553:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1365 00000100:00100000:1.0:1788619935.017562:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64a8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571767040:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619935.017578:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619935.017594:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe536873704b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dfe0 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619935.017643:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fe060380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589283456:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619935.017714:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78e6e80c00 going to sleep for 11 seconds 00000100:00100000:3.0:1788619935.019401:0:11397:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fd939180 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:11397:1875503589283328:192.168.201.5@tcp:106: 00002000:00010000:3.0:1788619935.019425:0:11397:0:(mdt_io.c:1550:mdt_lvb2reply()) size 49870 00010000:00010000:3.0:1788619935.019442:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07400/0x1cd7fe5368737044 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dfd9 expref: 15 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619935.019461:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:3.0:1788619935.019464:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788619935.019480:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07400/0x1cd7fe5368737044 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dfd9 expref: 15 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619935.019496:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa07400, rc 0) 00000100:00100000:3.0:1788619935.019546:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c71f1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2352:x1875503571766912:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 4232us (4394us total) trans 0 rc 301/301 00000100:00100000:3.0:1788619935.019554:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1662 00000100:00100000:1.0:1788619935.020959:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe060380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589283456:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619935.020990:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe536873704b lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dfe0 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619935.021011:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c91d7b84/0x1cd7fe536873704b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dfe0 expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619935.021021:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683ca00, rc 0) 00000100:00100000:1.0:1788619935.021067:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64a8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571767040:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3507us (3704us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619935.021074:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1365 00000100:00100000:2.0:1788619935.024138:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619935.024221:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619935.024287:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571767168 00000100:00100000:1.0:1788619935.024306:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571767168 00000100:00100000:1.0:1788619935.024391:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1663 00000100:00100000:1.0:1788619935.024402:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7433100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2352:x1875503571767168:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619935.024421:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619935.024440:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368737052 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dfe7 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619935.024490:0:5845:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe536873701a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dfbd expref: 19 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619935.024529:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fe060380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589283584:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619935.024601:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78e6e80c00 going to sleep for 11 seconds 00000100:00100000:0.0:1788619935.026615:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571767296 00000100:00100000:1.0:1788619935.026618:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe060380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589283584:192.168.201.5@tcp:106: 00000100:00100000:0.0:1788619935.026623:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571767296 00002000:00010000:1.0:1788619935.026638:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 49870 00010000:00010000:1.0:1788619935.026655:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368737052 lrc: 1/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dfe7 expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619935.026671:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1366 00010000:00010000:1.0:1788619935.026679:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00000100:00100000:0.0:1788619935.026679:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64a9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571767296:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619935.026681:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619935.026687:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368737052 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dfe7 expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619935.026693:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619935.026700:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683ce00, rc 0) 00010000:00010000:0.0:1788619935.026707:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000759096b0/0x1cd7fe5368737059 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dfee expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788619935.026753:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e67f6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589283712:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619935.026872:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7433100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2352:x1875503571767168:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2471us (2650us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619935.026879:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1663 00000100:00100000:0.0:1788619935.027034:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5eaf900 going to sleep for 11 seconds 00000100:00100000:2.0:1788619935.029570:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e67f6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589283712:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619935.029600:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000759096b0/0x1cd7fe5368737059 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dfee expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619935.029618:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000759096b0/0x1cd7fe5368737059 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dfee expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619935.029626:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c777c000, rc 0) 00000100:00100000:2.0:1788619935.029671:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64a9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571767296:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2993us (5542us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619935.029683:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1366 00000100:00100000:2.0:1788619935.031758:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619935.031891:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571767424 00000100:00100000:3.0:1788619935.031901:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571767424 00000100:00100000:3.0:1788619935.032006:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1664 00000100:00100000:3.0:1788619935.032017:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c7431c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:14908:x1875503571767424:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619935.032037:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619935.032058:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07400/0x1cd7fe5368737060 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dff5 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619935.032088:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe536873701a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dfbd expref: 19 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619935.032099:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07400/0x1cd7fe5368737060 lrc: 4/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dff5 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619935.032126:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe536873701a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dfbd expref: 19 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619935.032135:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe536873701a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dfbd expref: 19 pid: 5846 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619935.032145:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fd939180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589283840:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619935.032209:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e7d80 going to sleep for 11 seconds 00000100:00100000:3.0:1788619935.034668:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619935.034710:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571767552 00000100:00100000:2.0:1788619935.034718:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571767552 00010000:00010000:2.0:1788619935.034759:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe536873701a lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dfbd expref: 20 pid: 5846 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619935.034798:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 559 00000100:00100000:2.0:1788619935.034808:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd93aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+20:2354:x1875503571767552:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619935.034822:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619935.034847:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe536873701a lrc: 5/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dfbd expref: 20 pid: 5846 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619935.034860:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe536873701a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dfbd expref: 20 pid: 5846 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619935.034892:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:2.0:1788619935.034899:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 6 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07400/0x1cd7fe5368737060 lrc: 3/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dff5 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619935.034917:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07400/0x1cd7fe5368737060 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dff5 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619935.034927:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07400/0x1cd7fe5368737060 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x50080000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dff5 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619935.034973:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07400/0x1cd7fe5368737060 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dff5 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619935.034991:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cdb800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589283968:192.168.201.5@tcp:105: 00010000:00010000:1.0:1788619935.035041:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe536873701a lrc: 0/0,0 mode: --/PW res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 8 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dfbd expref: 20 pid: 5846 timeout: 527 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619935.035057:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fd939180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5846:1875503589283840:192.168.201.5@tcp:104: 00000100:00100000:2.0:1788619935.035077:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164480 going to sleep for 11 seconds 00010000:00010000:1.0:1788619935.035080:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07400/0x1cd7fe5368737060 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xe:0x0].0x0 bits 0x40/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dff5 expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619935.035090:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:1.0:1788619935.035093:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788619935.035096:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa07400, rc 0) 00000100:00100000:1.0:1788619935.035126:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c7431c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:14908:x1875503571767424:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 3112us (3378us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619935.035133:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1664 00000100:00100000:2.0:1788619935.036544:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cdb800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-MDT0000_UUID:5831:1875503589283968:192.168.201.5@tcp:105: 00010000:00010000:2.0:1788619935.036563:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619935.036609:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd93aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:1b0df97a-4f84-4680-9108-88353da6ca92+19:2354:x1875503571767552:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 1802us (1947us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619935.036618:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 559 00000100:00100000:0.0:1788619935.038162:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619935.038205:0:13346:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571767680 00000100:00100000:2.0:1788619935.038214:0:13346:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571767680 00002000:00100000:2.0:1788619935.038260:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e67f5180 x1875503571767680/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:31/0 lens 488/0 e 0 to 0 dl 1788619946 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619935.038271:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e67f5180 with x1875503571767680 00000100:00100000:2.0:1788619935.038366:0:13346:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1025 00000100:00100000:2.0:1788619935.038376:0:13346:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571767680:12345-192.168.201.5@tcp:3:fsx.0 00000100:00100000:2.0:1788619935.040843:0:13346:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571767680:12345-192.168.201.5@tcp:3:fsx.0 Request processed in 2458us (2676us total) trans 0 rc 53248/53248 00000100:00100000:2.0:1788619935.040856:0:13346:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1025 00002000:00100000:2.0:1788619935.040867:0:13346:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_003: refresh rw locks for [0x200000401:0xe:0x0] req@ffff8e78e67f5180 x1875503571767680/t0(0) o3->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:31/0 lens 488/440 e 0 to 0 dl 1788619946 ref 1 fl Complete:/600/0 rc 53248/53248 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1788619935.040877:0:13346:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e78e67f5180 with x1875503571767680 00000100:00100000:3.0:1788619935.043423:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619935.043933:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571767808 00000100:00100000:2.0:1788619935.043942:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571767808 00000100:00100000:2.0:1788619935.043994:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1367 00000100:00100000:2.0:1788619935.044004:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd938a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571767808:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619935.044022:0:6698:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619935.044039:0:6698:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006d293779/0x1cd7fe5368737067 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1dffc expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619935.044113:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cdb800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589284096:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619935.044200:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164480 going to sleep for 11 seconds 00000100:00100000:2.0:1788619935.046660:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cdb800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6698:1875503589284096:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619935.046691:0:6698:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000006d293779/0x1cd7fe5368737067 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dffc expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619935.046710:0:6698:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006d293779/0x1cd7fe5368737067 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dffc expref: 6 pid: 6698 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619935.046719:0:6698:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6a3ee00, rc 0) 00000100:00100000:2.0:1788619935.046772:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd938a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571767808:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2768us (3356us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619935.046781:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1367 00000100:00100000:0.0:1788619935.049676:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619935.049767:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571767936 00000100:00100000:3.0:1788619935.049777:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571767936 00000100:00100000:3.0:1788619935.049883:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1368 00000100:00100000:3.0:1788619935.049895:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571767936:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:3.0:1788619935.049923:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619935.049940:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005221cf7b/0x1cd7fe536873706e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1e003 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619935.050007:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fd93a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589284224:192.168.201.5@tcp:106: 00000100:00100000:3.0:1788619935.050084:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e79010e70c0 going to sleep for 11 seconds 00000100:00100000:0.0:1788619935.052621:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fd93a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6696:1875503589284224:192.168.201.5@tcp:106: 00010000:00010000:0.0:1788619935.052659:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005221cf7b/0x1cd7fe536873706e lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e003 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619935.052685:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005221cf7b/0x1cd7fe536873706e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e003 expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619935.052699:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa07600, rc 0) 00000100:00100000:0.0:1788619935.052865:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571767936:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2971us (3196us total) trans 0 rc 301/301 00000100:00100000:0.0:1788619935.052881:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1368 00000100:00100000:0.0:1788619935.055672:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619935.055805:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571768064 00000100:00100000:1.0:1788619935.055813:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571768064 00000100:00100000:1.0:1788619935.055860:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1369 00000100:00100000:1.0:1788619935.055872:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571768064:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:1.0:1788619935.055890:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788619935.055908:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c4fc38e/0x1cd7fe5368737075 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1e00a expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619935.056038:0:7654:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fe060380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589284352:192.168.201.5@tcp:106: 00000100:00100000:1.0:1788619935.056157:0:7654:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78e6e80c00 going to sleep for 11 seconds 00000100:00100000:1.0:1788619935.058543:0:7654:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe060380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:7654:1875503589284352:192.168.201.5@tcp:106: 00010000:00010000:1.0:1788619935.058579:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004c4fc38e/0x1cd7fe5368737075 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e00a expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619935.058604:0:7654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c4fc38e/0x1cd7fe5368737075 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e00a expref: 6 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619935.058618:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683ce00, rc 0) 00000100:00100000:1.0:1788619935.058763:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571768064:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 2891us (3099us total) trans 0 rc 301/301 00000100:00100000:1.0:1788619935.058771:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1369 00000100:00100000:3.0:1788619935.061564:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619935.061678:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571768192 00000100:00100000:2.0:1788619935.061687:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571768192 00000100:00100000:2.0:1788619935.061777:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1370 00000100:00100000:2.0:1788619935.061789:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd93a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571768192:12345-192.168.201.5@tcp:101:fsx.0 00010000:00010000:2.0:1788619935.061806:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788619935.061823:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fb5126d0/0x1cd7fe536873707c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1e011 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619935.061895:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5cdb800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589284480:192.168.201.5@tcp:106: 00000100:00100000:2.0:1788619935.062069:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c8164480 going to sleep for 11 seconds 00000100:00100000:2.0:1788619935.070966:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5cdb800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589284480:192.168.201.5@tcp:106: 00010000:00010000:2.0:1788619935.071009:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000fb5126d0/0x1cd7fe536873707c lrc: 1/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e011 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619935.071036:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fb5126d0/0x1cd7fe536873707c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e011 expref: 6 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619935.071048:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6a3f000, rc 0) 00000100:00100000:2.0:1788619935.071154:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd93a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571768192:12345-192.168.201.5@tcp:101:fsx.0 Request processed in 9367us (9598us total) trans 0 rc 301/301 00000100:00100000:2.0:1788619935.071164:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1370 00000100:00100000:2.0:1788619935.073258:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619935.073355:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571768448 00000100:00100000:0.0:1788619935.073364:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571768448 00000100:00100000:0.0:1788619935.073415:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 38 00000100:00100000:0.0:1788619935.073423:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cdb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:14908:x1875503571768448:12345-192.168.201.5@tcp:35:fsx.0 00000100:00100000:0.0:1788619935.073941:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cdb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:14908:x1875503571768448:12345-192.168.201.5@tcp:35:fsx.0 Request processed in 517us (689us total) trans 4294967884 rc 0/0 00000100:00100000:0.0:1788619935.073950:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 38 00000100:00100000:2.0:1788619935.076188:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619935.076254:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571768704 00000100:00100000:3.0:1788619935.076262:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571768704 00000100:00100000:3.0:1788619935.076320:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 39 00000100:00100000:3.0:1788619935.076462:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cdb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:1b0df97a-4f84-4680-9108-88353da6ca92+19:14908:x1875503571768704:12345-192.168.201.5@tcp:35:fsx.0 00000100:00100000:3.0:1788619935.076731:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cdb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:1b0df97a-4f84-4680-9108-88353da6ca92+20:14908:x1875503571768704:12345-192.168.201.5@tcp:35:fsx.0 Request processed in 272us (548us total) trans 4294967885 rc 0/0 00000100:00100000:3.0:1788619935.076748:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 39 00000100:00100000:1.0:1788619935.082578:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619935.082674:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571768960 00000100:00100000:0.0:1788619935.082684:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571768960 00000100:00100000:0.0:1788619935.082753:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 40 00000100:00100000:0.0:1788619935.082764:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78f1d9c850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:14908:x1875503571768960:12345-192.168.201.5@tcp:35:fsx.0 00000100:00100000:0.0:1788619935.083079:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78f1d9c850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+17:14908:x1875503571768960:12345-192.168.201.5@tcp:35:fsx.0 Request processed in 316us (510us total) trans 4294967886 rc 0/0 00000100:00100000:0.0:1788619935.083086:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 40 00000100:00100000:2.0:1788619935.085412:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619935.085554:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571769216 00000100:00100000:0.0:1788619935.085563:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571769216 00000100:00100000:0.0:1788619935.085615:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 41 00000100:00100000:0.0:1788619935.085625:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cdb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+17:14908:x1875503571769216:12345-192.168.201.5@tcp:35:fsx.0 00000100:00100000:0.0:1788619935.085973:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cdb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+18:14908:x1875503571769216:12345-192.168.201.5@tcp:35:fsx.0 Request processed in 349us (570us total) trans 4294967887 rc 0/0 00000100:00100000:0.0:1788619935.085982:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 41 00010000:00010000:3.0:1788619935.522656:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788619935.842770:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fe060380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589284608:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1788619935.842844:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619935.843155:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589284608 00000100:00100000:1.0:1788619935.843163:0:6699:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589284608 00000100:00100000:1.0:1788619935.843283:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 95 00000100:00100000:1.0:1788619935.843294:0:6699:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cdb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589284608:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1788619935.843317:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 183959552 pending 0 free 3859808256 avail 3854565376 00000100:00100000:1.0:1788619935.843395:0:6699:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cdb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589284608:12345-0@lo:13:osp-pre-0-0.0 Request processed in 104us (551us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619935.843404:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 95 00000100:00100000:3.0:1788619935.843492:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe060380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589284608:0@lo:13:osp-pre-0-0.0 00010000:00010000:0.0:1788619936.547018:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788619937.570682:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788619938.598537:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788619939.433864:0:3319:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fe060000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:3319:1875503589284736:0@lo:400:kworker.0 00000100:00100000:1.0:1788619939.434071:0:3319:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619939.434117:0:3319:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fe060e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3319:1875503589285248:0@lo:400:kworker.0 00000100:00100000:1.0:1788619939.434187:0:3319:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619939.434210:0:3319:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1788619939.434279:0:3319:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fe063b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3319:1875503589285120:0@lo:400:kworker.0 00000100:00100000:1.0:1788619939.434316:0:3319:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619939.434360:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fe062680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3316:1875503589284864:0@lo:400:kworker.0 00000100:00100000:1.0:1788619939.434381:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619939.434408:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fe060a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589285376:0@lo:400:kworker.0 00000100:00100000:1.0:1788619939.434425:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619939.434439:0:3316:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1788619939.434444:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fe062d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3316:1875503589284992:0@lo:400:kworker.0 00000100:00100000:1.0:1788619939.434459:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619939.434485:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589285248 00000100:00100000:1.0:1788619939.434492:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589285248 00000100:00100000:1.0:1788619939.434554:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589284864 00000100:00100000:1.0:1788619939.434557:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589284864 00000100:00100000:1.0:1788619939.434580:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589284992 00000100:00100000:1.0:1788619939.434582:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589284992 00000100:00100000:1.0:1788619939.434596:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 198 00000100:00100000:1.0:1788619939.434603:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe060380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3319:x1875503589285248:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788619939.434648:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe060380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3319:x1875503589285248:12345-0@lo:400:kworker.0 Request processed in 45us (460us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619939.434658:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 198 00000100:00100000:1.0:1788619939.434672:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 199 00000100:00100000:1.0:1788619939.434677:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e02d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3316:x1875503589284864:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788619939.434700:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e02d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3316:x1875503589284864:12345-0@lo:400:kworker.0 Request processed in 23us (319us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619939.434706:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 199 00000100:00100000:1.0:1788619939.434715:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 200 00000100:00100000:1.0:1788619939.434719:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e02680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3316:x1875503589284992:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788619939.434737:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e02680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3316:x1875503589284992:12345-0@lo:400:kworker.0 Request processed in 18us (278us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619939.434742:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 200 00000100:00100000:1.0:1788619939.434759:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589285120 00000100:00100000:1.0:1788619939.434762:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589285120 00000100:00100000:1.0:1788619939.434784:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589285376 00000100:00100000:1.0:1788619939.434786:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589285376 00000100:00100000:1.0:1788619939.434803:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 134 00000100:00100000:1.0:1788619939.434808:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e00e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3319:x1875503589285120:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788619939.434832:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e00e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3319:x1875503589285120:12345-0@lo:400:kworker.0 Request processed in 24us (516us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619939.434837:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 134 00000100:00100000:1.0:1788619939.434846:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 135 00000100:00100000:1.0:1788619939.434851:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e01500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503589285376:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788619939.434869:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e01500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503589285376:12345-0@lo:400:kworker.0 Request processed in 19us (444us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619939.434874:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 135 00000100:00080000:1.0:1788619939.434890:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1788619939.434893:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1788619939.434895:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788619939.435121:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589284736 00000100:00100000:3.0:1788619939.435126:0:5836:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589284736 00000100:00100000:3.0:1788619939.435149:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 298 00000100:00100000:3.0:1788619939.435154:0:5836:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5caf850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+19:3319:x1875503589284736:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788619939.435183:0:5836:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5caf850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+19:3319:x1875503589284736:12345-0@lo:400:kworker.0 Request processed in 30us (1114us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619939.435189:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 298 00000100:00100000:1.0:1788619939.435452:0:3319:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe063b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3319:1875503589285120:0@lo:400:kworker.0 00000100:00100000:1.0:1788619939.435464:0:3319:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe060000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:3319:1875503589284736:0@lo:400:kworker.0 00000100:00100000:1.0:1788619939.435474:0:3319:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe060e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3319:1875503589285248:0@lo:400:kworker.0 00000100:00100000:1.0:1788619939.435510:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe062d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3316:1875503589284992:0@lo:400:kworker.0 00000100:00100000:1.0:1788619939.435519:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe062680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3316:1875503589284864:0@lo:400:kworker.0 00000100:00100000:1.0:1788619939.435527:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe060a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589285376:0@lo:400:kworker.0 00000001:00080000:0.0:1788619939.486801:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967887 is committed 00000001:00080000:0.0:1788619939.486821:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967877 is committed 00000001:00080000:0.0:1788619939.486832:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967883 is committed 00000001:00080000:0.0:1788619939.486852:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967885 is committed 00010000:00010000:0.0:1788619939.619664:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788619939.666094:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619939.666125:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619939.666140:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619939.666163:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619939.666179:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619939.666240:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571769600 00000100:00100000:0.0:1788619939.666249:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571769600 00000100:00100000:0.0:1788619939.666300:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571769856 00000100:00100000:0.0:1788619939.666302:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571769856 00000100:00100000:0.0:1788619939.666316:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 136 00000100:00100000:0.0:1788619939.666339:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c09500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571769600:12345-192.168.201.5@tcp:400:kworker.0 00000100:00080000:2.0:1788619939.666540:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1788619939.666592:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571769344 00000100:00100000:3.0:1788619939.666597:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571769344 00000100:00100000:1.0:1788619939.666708:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 137 00000100:00100000:1.0:1788619939.666719:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c0a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571769856:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:3.0:1788619939.666739:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 299 00000100:00100000:3.0:1788619939.666747:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5cad050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:597450a2-e90c-408c-b401-a0ce48eed204+13:2354:x1875503571769344:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:1.0:1788619939.666756:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c0a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+7:2354:x1875503571769856:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 42us (615us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619939.666764:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 137 00000100:00100000:3.0:1788619939.666770:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5cad050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:597450a2-e90c-408c-b401-a0ce48eed204+13:2354:x1875503571769344:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 26us (685us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619939.666776:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 299 00000100:00100000:1.0:1788619939.666786:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571769472 00000100:00100000:1.0:1788619939.666790:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571769472 00000100:00100000:0.0:1788619939.666818:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c09500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571769600:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 495us (690us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619939.666820:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571769728 00000100:00100000:1.0:1788619939.666822:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571769728 00000100:00100000:0.0:1788619939.666825:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 136 00000100:00100000:1.0:1788619939.666843:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 201 00000100:00100000:1.0:1788619939.666850:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c08a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571769472:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:1.0:1788619939.666881:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c08a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571769472:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 31us (716us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619939.666887:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 201 00000100:00100000:1.0:1788619939.666899:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 202 00000100:00100000:1.0:1788619939.666998:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63f8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571769728:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:1.0:1788619939.667028:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63f8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571769728:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 35us (847us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619939.667034:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 202 00000100:00080000:1.0:1788619939.667065:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1788619939.940452:0:3317:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900e02300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3317:1875503589285504:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1788619939.940521:0:3317:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619939.940595:0:6700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589285504 00000100:00100000:1.0:1788619939.940603:0:6700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589285504 00000100:00100000:1.0:1788619939.940649:0:6700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 96 00000100:00100000:1.0:1788619939.940658:0:6700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e03800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3317:x1875503589285504:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1788619939.940694:0:6700:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3856662528 avail 3854565376 00000100:00100000:1.0:1788619939.940737:0:6700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e03800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3317:x1875503589285504:12345-0@lo:13:osp-pre-1-0.0 Request processed in 80us (215us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619939.940744:0:6700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 96 00000100:00100000:1.0:1788619939.941057:0:3317:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900e02300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3317:1875503589285504:0@lo:13:osp-pre-1-0.0 00000001:00080000:3.0:1788619940.013896:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967537 is committed 00010000:00010000:1.0:1788619940.643186:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788619940.964507:0:3319:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1788619940.964533:0:3319:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900e02300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3319:1875503589285632:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1788619940.964619:0:3319:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619940.964807:0:6700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589285632 00000100:00100000:1.0:1788619940.964817:0:6700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589285632 00000100:00100000:1.0:1788619940.964886:0:6700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 97 00000100:00100000:1.0:1788619940.964896:0:6700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c63fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1875503589285632:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1788619940.964930:0:6700:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 183959552 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1788619940.964987:0:6700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c63fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1875503589285632:12345-0@lo:13:osp-pre-0-0.0 Request processed in 92us (368us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619940.964994:0:6700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 97 00000100:00100000:3.0:1788619940.965236:0:3319:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900e02300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3319:1875503589285632:0@lo:13:osp-pre-0-0.0 00010000:00010000:0.0:1788619941.680234:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:0.0:1788619942.094784:0:13546:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-dom test sanity: Run sanity with Data-on-MDT files ========================================================== 10:52:20 (1788619940) 00010000:00010000:0.0:1788619942.690694:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788619943.717232:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788619944.546598:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900e02a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589286144:0@lo:400:kworker.0 00000100:00100000:1.0:1788619944.546678:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619944.546739:0:3318:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:1.0:1788619944.546745:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900e00380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:3318:1875503589285760:0@lo:400:kworker.0 00000100:00100000:1.0:1788619944.546773:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619944.546796:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900e00a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3318:1875503589286272:0@lo:400:kworker.0 00000100:00100000:1.0:1788619944.546823:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619944.546849:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900e01f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3316:1875503589285888:0@lo:400:kworker.0 00000100:00100000:1.0:1788619944.546871:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619944.546885:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900e03100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589286400:0@lo:400:kworker.0 00000100:00100000:1.0:1788619944.546904:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619944.549714:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589285760 00000100:00100000:3.0:1788619944.549730:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589285760 00000100:00100000:3.0:1788619944.549784:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 300 00000100:00100000:3.0:1788619944.549795:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5caec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+19:3318:x1875503589285760:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788619944.549882:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5caec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+19:3318:x1875503589285760:12345-0@lo:400:kworker.0 Request processed in 90us (3107us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619944.549889:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 300 00000100:00100000:0.0:1788619944.552284:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589286272 00000100:00100000:0.0:1788619944.552295:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589286272 00000100:00100000:0.0:1788619944.552393:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589285888 00000100:00100000:0.0:1788619944.552397:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589285888 00000100:00100000:0.0:1788619944.552421:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 203 00000100:00100000:0.0:1788619944.552431:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e01c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3318:x1875503589286272:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788619944.552515:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e01c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3318:x1875503589286272:12345-0@lo:400:kworker.0 Request processed in 86us (5691us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619944.552522:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 203 00000100:00100000:0.0:1788619944.552539:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 204 00000100:00100000:0.0:1788619944.552545:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e00700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3316:x1875503589285888:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788619944.552574:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e00700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3316:x1875503589285888:12345-0@lo:400:kworker.0 Request processed in 29us (5703us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619944.552580:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 204 00000100:00100000:0.0:1788619944.552639:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900e00380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:3318:1875503589285760:0@lo:400:kworker.0 00000100:00100000:0.0:1788619944.552655:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900e00a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3318:1875503589286272:0@lo:400:kworker.0 00000100:00080000:0.0:1788619944.552688:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1788619944.552714:0:3317:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900e00000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3317:1875503589286016:0@lo:400:kworker.0 00000100:00100000:0.0:1788619944.552750:0:3317:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788619944.552788:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589286016 00000100:00100000:0.0:1788619944.552791:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589286016 00000100:00100000:0.0:1788619944.552818:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 205 00000100:00100000:0.0:1788619944.552824:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3317:x1875503589286016:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788619944.552855:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3317:x1875503589286016:12345-0@lo:400:kworker.0 Request processed in 31us (105us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619944.552862:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 205 00000100:00080000:0.0:1788619944.552874:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1788619944.552890:0:3317:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900e00000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3317:1875503589286016:0@lo:400:kworker.0 00000100:00100000:1.0:1788619944.556362:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900e01f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3316:1875503589285888:0@lo:400:kworker.0 00000100:00100000:3.0:1788619944.560480:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589286144 00000100:00100000:3.0:1788619944.560494:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589286144 00000100:00100000:3.0:1788619944.560552:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589286400 00000100:00100000:3.0:1788619944.560556:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589286400 00000100:00100000:3.0:1788619944.561231:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 138 00000100:00100000:3.0:1788619944.561245:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e01880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589286144:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788619944.561380:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e01880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589286144:12345-0@lo:400:kworker.0 Request processed in 139us (14702us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619944.561388:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 138 00000100:00100000:3.0:1788619944.561406:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 139 00000100:00100000:3.0:1788619944.561413:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e02300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503589286400:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788619944.561452:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e02300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503589286400:12345-0@lo:400:kworker.0 Request processed in 40us (14547us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619944.561457:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 139 00000100:00080000:3.0:1788619944.561477:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1788619944.561481:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1788619944.561729:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900e03100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589286400:0@lo:400:kworker.0 00000100:00100000:0.0:1788619944.561833:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900e02a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589286144:0@lo:400:kworker.0 00010000:00010000:0.0:1788619944.742525:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788619944.786831:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619944.786862:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619944.786882:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619944.786905:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619944.786931:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571770112 00000100:00100000:0.0:1788619944.786940:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571770112 00000100:00100000:0.0:1788619944.786995:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 206 00000100:00100000:0.0:1788619944.787006:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2353:x1875503571770112:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:0.0:1788619944.787074:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2353:x1875503571770112:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 69us (167us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619944.787082:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 206 00000100:00100000:3.0:1788619944.788185:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571769984 00000100:00100000:3.0:1788619944.788194:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571769984 00000100:00100000:3.0:1788619944.788281:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 301 00000100:00100000:3.0:1788619944.788290:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5cacc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:597450a2-e90c-408c-b401-a0ce48eed204+13:2353:x1875503571769984:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:3.0:1788619944.788375:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5cacc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:597450a2-e90c-408c-b401-a0ce48eed204+13:2353:x1875503571769984:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 87us (1553us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619944.788383:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 301 00000100:00100000:3.0:1788619944.788416:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571770240 00000100:00100000:3.0:1788619944.788420:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571770240 00000100:00100000:3.0:1788619944.788448:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571770496 00000100:00100000:3.0:1788619944.788450:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571770496 00000100:00100000:3.0:1788619944.788469:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 140 00000100:00100000:3.0:1788619944.788475:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571770240:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:3.0:1788619944.788501:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2353:x1875503571770240:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 26us (1637us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619944.788507:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 140 00000100:00100000:3.0:1788619944.788519:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 141 00000100:00100000:3.0:1788619944.788525:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571770496:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:3.0:1788619944.788547:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2353:x1875503571770496:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 21us (1663us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619944.788553:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 141 00000100:00080000:3.0:1788619944.788920:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1788619944.788926:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1788619944.807741:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619944.807891:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571770368 00000100:00100000:0.0:1788619944.807901:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571770368 00000100:00100000:0.0:1788619944.807961:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 207 00000100:00100000:0.0:1788619944.807970:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e67f4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571770368:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:0.0:1788619944.808037:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e67f4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:2353:x1875503571770368:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 68us (302us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619944.808043:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 207 00000100:00080000:0.0:1788619944.808062:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788619945.061477:0:3316:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1788619945.061500:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fd939180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589286528:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1788619945.061570:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619945.061632:0:6700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589286528 00000100:00100000:3.0:1788619945.061641:0:6700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589286528 00000100:00100000:3.0:1788619945.061685:0:6700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 98 00000100:00100000:3.0:1788619945.061693:0:6700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503589286528:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1788619945.061726:0:6700:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3856662528 avail 3854565376 00000100:00100000:3.0:1788619945.061768:0:6700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503589286528:12345-0@lo:13:osp-pre-1-0.0 Request processed in 75us (198us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619945.061775:0:6700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 98 00000100:00100000:3.0:1788619945.061823:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fd939180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589286528:0@lo:13:osp-pre-1-0.0 00010000:00010000:0.0:1788619945.762615:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788619946.086148:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900e03100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589286656:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1788619946.086226:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619946.086385:0:6700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589286656 00000100:00100000:3.0:1788619946.086396:0:6700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589286656 00000100:00100000:3.0:1788619946.086478:0:6700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 99 00000100:00100000:3.0:1788619946.086489:0:6700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64d1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589286656:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1788619946.086527:0:6700:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 183959552 pending 0 free 3859808256 avail 3857711104 00000100:00080000:2.0:1788619946.086557:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1788619946.089702:0:6700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64d1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589286656:12345-0@lo:13:osp-pre-0-0.0 Request processed in 3205us (3465us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619946.089712:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900e03100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589286656:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1788619946.089720:0:6700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 99 00000100:00100000:2.0:1788619946.313984:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619946.314032:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619946.314066:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571770624 00000100:00100000:0.0:1788619946.314075:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571770624 00002000:00100000:0.0:1788619946.314135:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0x10:0x0] req@ffff8e77c69d0a80 x1875503571770624/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:42/0 lens 488/0 e 0 to 0 dl 1788619957 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788619946.314146:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69d0a80 with x1875503571770624 00000100:00080000:2.0:1788619946.314162:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1788619946.314181:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1026 00000100:00100000:0.0:1788619946.314189:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69d0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571770624:12345-192.168.201.5@tcp:4:fsx.0 00000100:00100000:0.0:1788619946.314363:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571770752 00000100:00100000:0.0:1788619946.314367:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571770752 00000100:00100000:0.0:1788619946.314389:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1665 00000100:00100000:0.0:1788619946.314394:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69d3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571770752:12345-192.168.201.5@tcp:16:kworker.0 00000100:00100000:0.0:1788619946.314448:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69d3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571770752:12345-192.168.201.5@tcp:16:kworker.0 Request processed in 54us (417us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619946.314454:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1665 00000004:00010000:3.0:1788619946.317153:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x10:0x0] updating lvb size from disk: 57344 -> 83777 00000004:00010000:3.0:1788619946.317161:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x10:0x0] updating lvb mtime from disk: 1788619922 -> 1788619934 00000004:00010000:3.0:1788619946.317163:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x10:0x0] updating lvb atime from disk: 1788619890 -> 1788619890 00000004:00010000:3.0:1788619946.317165:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x10:0x0] updating lvb ctime from disk: 1788619922 -> 1788619934 00000004:00010000:3.0:1788619946.317166:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x10:0x0] updating lvb blocks from disk: 2 -> 130 00000100:00100000:3.0:1788619946.317228:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69d0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:2355:x1875503571770624:12345-192.168.201.5@tcp:4:fsx.0 Request processed in 3034us (3248us total) trans 4294967888 rc 0/0 00000100:00100000:3.0:1788619946.317236:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1026 00002000:00100000:3.0:1788619946.317245:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0x10:0x0] req@ffff8e77c69d0a80 x1875503571770624/t4294967888(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:42/0 lens 488/448 e 0 to 0 dl 1788619957 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1788619946.317252:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c69d0a80 with x1875503571770624 00010000:00010000:1.0:1788619946.787894:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788619947.827450:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788619948.834517:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788619949.670320:0:3319:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900e01f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:3319:1875503589286784:0@lo:400:kworker.0 00000100:00100000:1.0:1788619949.670411:0:3319:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619949.670456:0:3319:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900e00a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3319:1875503589287296:0@lo:400:kworker.0 00000100:00100000:3.0:1788619949.670626:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900e03100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3316:1875503589286912:0@lo:400:kworker.0 00000100:00100000:1.0:1788619949.670688:0:3319:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619949.670724:0:3319:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1788619949.670731:0:3319:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900e00000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3319:1875503589287168:0@lo:400:kworker.0 00000100:00100000:1.0:1788619949.670757:0:3319:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619949.670781:0:3317:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900e02a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3317:1875503589287040:0@lo:400:kworker.0 00000100:00100000:1.0:1788619949.670803:0:3317:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619949.671595:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589287296 00000100:00100000:3.0:1788619949.671605:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589287296 00000100:00100000:3.0:1788619949.671664:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589287040 00000100:00100000:3.0:1788619949.671668:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589287040 00000100:00100000:3.0:1788619949.671693:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 208 00000100:00100000:3.0:1788619949.671702:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe060a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3319:x1875503589287296:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788619949.671767:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe060a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3319:x1875503589287296:12345-0@lo:400:kworker.0 Request processed in 65us (1078us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619949.671775:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 208 00000100:00100000:3.0:1788619949.671793:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 209 00000100:00100000:3.0:1788619949.671800:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe062d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3317:x1875503589287040:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788619949.671833:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe062d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3317:x1875503589287040:12345-0@lo:400:kworker.0 Request processed in 33us (1028us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619949.671839:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 209 00000100:00100000:3.0:1788619949.671861:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589287168 00000100:00100000:3.0:1788619949.671866:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589287168 00000100:00100000:3.0:1788619949.671898:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 142 00000100:00100000:3.0:1788619949.671905:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe062680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3319:x1875503589287168:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788619949.671937:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe062680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3319:x1875503589287168:12345-0@lo:400:kworker.0 Request processed in 32us (1178us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619949.671943:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 142 00000100:00100000:3.0:1788619949.671992:0:3319:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900e00000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3319:1875503589287168:0@lo:400:kworker.0 00000100:00100000:3.0:1788619949.672005:0:3319:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900e00a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3319:1875503589287296:0@lo:400:kworker.0 00000100:00100000:3.0:1788619949.672052:0:3317:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900e02a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3317:1875503589287040:0@lo:400:kworker.0 00000100:00100000:3.0:1788619949.672077:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619949.672101:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900e00380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589287424:0@lo:400:kworker.0 00000100:00100000:3.0:1788619949.672127:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619949.672158:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589286912 00000100:00100000:3.0:1788619949.672161:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589286912 00000100:00100000:3.0:1788619949.672183:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 210 00000100:00100000:3.0:1788619949.672189:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3316:x1875503589286912:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788619949.672238:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3316:x1875503589286912:12345-0@lo:400:kworker.0 Request processed in 48us (159us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619949.672244:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 210 00000100:00100000:3.0:1788619949.672270:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900e03100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3316:1875503589286912:0@lo:400:kworker.0 00000100:00100000:0.0:1788619949.673134:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589286784 00000100:00100000:0.0:1788619949.673142:0:5836:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589286784 00000100:00100000:0.0:1788619949.673173:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 302 00000100:00100000:0.0:1788619949.673180:0:5836:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5caf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+19:3319:x1875503589286784:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788619949.673239:0:5836:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5caf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+19:3319:x1875503589286784:12345-0@lo:400:kworker.0 Request processed in 59us (2828us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619949.673246:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 302 00000100:00100000:0.0:1788619949.673283:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589287424 00000100:00100000:0.0:1788619949.673287:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589287424 00000100:00100000:0.0:1788619949.673316:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 143 00000100:00100000:0.0:1788619949.673341:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503589287424:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788619949.673376:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503589287424:12345-0@lo:400:kworker.0 Request processed in 53us (1248us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619949.673381:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 143 00000100:00100000:3.0:1788619949.674176:0:3319:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900e01f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:3319:1875503589286784:0@lo:400:kworker.0 00000100:00100000:3.0:1788619949.674224:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900e00380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589287424:0@lo:400:kworker.0 00000100:00080000:2.0:1788619949.674372:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1788619949.674377:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1788619949.674380:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:0.0:1788619949.744438:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967888 is committed 00010000:00010000:2.0:1788619949.859005:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788619949.905392:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619949.905448:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619949.905484:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619949.905511:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619949.905540:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619949.905572:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571771136 00000100:00100000:1.0:1788619949.905589:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571770880 00000100:00100000:1.0:1788619949.905596:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571770880 00000100:00100000:3.0:1788619949.905632:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571771136 00000100:00100000:1.0:1788619949.905645:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 303 00000100:00100000:1.0:1788619949.905656:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5cae450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:597450a2-e90c-408c-b401-a0ce48eed204+13:2355:x1875503571770880:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:1.0:1788619949.905720:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5cae450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:597450a2-e90c-408c-b401-a0ce48eed204+13:2355:x1875503571770880:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 66us (271us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619949.905730:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 303 00000100:00100000:1.0:1788619949.905947:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571771392 00000100:00100000:1.0:1788619949.905951:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571771392 00000100:00100000:1.0:1788619949.905978:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 144 00000100:00100000:1.0:1788619949.905984:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe063b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571771392:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:1.0:1788619949.906015:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe063b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571771392:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 31us (504us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619949.906022:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 144 00000100:00100000:1.0:1788619949.906074:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571771008 00000100:00100000:1.0:1788619949.906078:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571771008 00000100:00100000:1.0:1788619949.906105:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571771264 00000100:00100000:1.0:1788619949.906108:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571771264 00000100:00100000:1.0:1788619949.906127:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 211 00000100:00100000:1.0:1788619949.906134:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe060e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571771008:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:1.0:1788619949.906171:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe060e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571771008:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 37us (790us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619949.906178:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 211 00000100:00100000:1.0:1788619949.906193:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 212 00000100:00100000:1.0:1788619949.906201:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571771264:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:1.0:1788619949.906240:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571771264:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 40us (699us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619949.906247:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 212 00000100:00100000:0.0:1788619949.906395:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 145 00000100:00100000:0.0:1788619949.906406:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe060000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571771136:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:0.0:1788619949.906437:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe060000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2355:x1875503571771136:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 33us (952us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619949.906445:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 145 00000100:00080000:3.0:1788619949.906471:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1788619949.906478:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1788619950.178726:0:3316:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1788619950.178761:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589287552:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1788619950.178841:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788619950.179673:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589287552 00000100:00100000:0.0:1788619950.179683:0:6699:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589287552 00000100:00100000:0.0:1788619950.179730:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 100 00000100:00100000:0.0:1788619950.179739:0:6699:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503589287552:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1788619950.179777:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3856662528 avail 3854565376 00000100:00100000:0.0:1788619950.179828:0:6699:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503589287552:12345-0@lo:13:osp-pre-1-0.0 Request processed in 90us (987us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619950.179834:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 100 00000100:00100000:0.0:1788619950.179879:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589287552:0@lo:13:osp-pre-1-0.0 00010000:00010000:0.0:1788619950.886006:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788619951.203659:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c75f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589287680:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1788619951.203763:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619951.204086:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589287680 00000100:00100000:1.0:1788619951.204098:0:6699:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589287680 00000100:00100000:1.0:1788619951.204184:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 101 00000100:00100000:1.0:1788619951.204209:0:6699:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589287680:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1788619951.204255:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 183959552 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1788619951.204315:0:6699:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589287680:12345-0@lo:13:osp-pre-0-0.0 Request processed in 110us (553us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619951.204379:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 101 00000100:00100000:0.0:1788619951.204537:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c75f4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589287680:0@lo:13:osp-pre-0-0.0 00000100:00080000:3.0:1788619951.204738:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:2.0:1788619951.913026:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788619952.933108:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788619953.954722:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788619954.792285:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df8b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3316:1875503589287936:0@lo:400:kworker.0 00000100:00100000:0.0:1788619954.792382:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619954.792594:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589288192:0@lo:400:kworker.0 00000100:00100000:1.0:1788619954.792627:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619954.792685:0:3318:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:1.0:1788619954.792692:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df8b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:3318:1875503589287808:0@lo:400:kworker.0 00000100:00100000:1.0:1788619954.792735:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619954.792761:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3318:1875503589288320:0@lo:400:kworker.0 00000100:00100000:1.0:1788619954.792802:0:3317:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df8b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3317:1875503589288064:0@lo:400:kworker.0 00000100:00100000:1.0:1788619954.792834:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589288192 00000100:00100000:1.0:1788619954.792842:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589288192 00000100:00100000:1.0:1788619954.792898:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 146 00000100:00100000:1.0:1788619954.792908:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589288192:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788619954.797193:0:3317:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619954.797414:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589288192:12345-0@lo:400:kworker.0 Request processed in 4505us (4783us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619954.797429:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 146 00000100:00100000:1.0:1788619954.797467:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589287808 00000100:00100000:1.0:1788619954.797474:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589287808 00000100:00100000:1.0:1788619954.797527:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 304 00000100:00100000:1.0:1788619954.797536:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5cac850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+19:3318:x1875503589287808:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788619954.797602:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5cac850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+19:3318:x1875503589287808:12345-0@lo:400:kworker.0 Request processed in 67us (4866us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619954.797608:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 304 00000100:00080000:1.0:1788619954.797649:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1788619954.797666:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619954.797709:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589288192:0@lo:400:kworker.0 00000100:00100000:1.0:1788619954.797740:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df8b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:3318:1875503589287808:0@lo:400:kworker.0 00000100:00100000:2.0:1788619954.798233:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589287936 00000100:00100000:2.0:1788619954.798239:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589287936 00000100:00100000:2.0:1788619954.798273:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589288064 00000100:00100000:2.0:1788619954.798277:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589288064 00000100:00100000:2.0:1788619954.798302:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589288320 00000100:00100000:2.0:1788619954.798305:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589288320 00000100:00100000:2.0:1788619954.798319:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 213 00000100:00100000:2.0:1788619954.798345:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69ffb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3316:x1875503589287936:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788619954.798402:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69ffb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3316:x1875503589287936:12345-0@lo:400:kworker.0 Request processed in 76us (6019us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619954.798409:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 213 00000100:00100000:2.0:1788619954.798424:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 214 00000100:00100000:2.0:1788619954.798430:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74a8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3317:x1875503589288064:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788619954.798464:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74a8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3317:x1875503589288064:12345-0@lo:400:kworker.0 Request processed in 34us (1276us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619954.798470:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 214 00000100:00100000:2.0:1788619954.798481:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 215 00000100:00100000:2.0:1788619954.798487:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3318:x1875503589288320:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788619954.798525:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3318:x1875503589288320:12345-0@lo:400:kworker.0 Request processed in 38us (858us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619954.798530:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 215 00000100:00080000:2.0:1788619954.798548:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1788619954.798571:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3318:1875503589288320:0@lo:400:kworker.0 00000100:00100000:2.0:1788619954.798599:0:3317:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df8b3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3317:1875503589288064:0@lo:400:kworker.0 00000100:00100000:2.0:1788619954.808222:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df8b0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3316:1875503589287936:0@lo:400:kworker.0 00000100:00100000:2.0:1788619954.808240:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69ff480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589288448:0@lo:400:kworker.0 00000100:00100000:2.0:1788619954.808308:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619954.808586:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589288448 00000100:00100000:3.0:1788619954.808595:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589288448 00000100:00100000:3.0:1788619954.808663:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 147 00000100:00100000:3.0:1788619954.808672:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74a8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503589288448:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788619954.808727:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74a8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503589288448:12345-0@lo:400:kworker.0 Request processed in 55us (418us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619954.808734:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 147 00000100:00080000:2.0:1788619954.809052:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1788619954.809078:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69ff480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589288448:0@lo:400:kworker.0 00010000:00010000:0.0:1788619954.980343:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788619955.025692:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619955.025741:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619955.025806:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571771520 00000100:00100000:0.0:1788619955.025817:0:5836:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571771520 00000100:00100000:0.0:1788619955.025876:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 305 00000100:00100000:0.0:1788619955.025888:0:5836:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5cad850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:597450a2-e90c-408c-b401-a0ce48eed204+13:2352:x1875503571771520:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:0.0:1788619955.025969:0:5836:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5cad850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:597450a2-e90c-408c-b401-a0ce48eed204+13:2352:x1875503571771520:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 83us (287us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619955.025976:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 305 00000100:00100000:2.0:1788619955.026818:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571771648 00000100:00100000:2.0:1788619955.026825:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571771648 00000100:00080000:1.0:1788619955.027071:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1788619955.027111:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 216 00000100:00100000:2.0:1788619955.027133:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2352:x1875503571771648:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:2.0:1788619955.027275:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2352:x1875503571771648:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 156us (1532us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619955.027283:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 216 00000100:00100000:1.0:1788619955.028871:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619955.030273:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571771776 00000100:00100000:3.0:1788619955.030287:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571771776 00000100:00080000:1.0:1788619955.030548:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1788619955.031439:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1371 00000100:00100000:3.0:1788619955.031455:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2351:x1875503571771776:12345-192.168.201.5@tcp:9:kworker.0 00000020:00000080:3.0:1788619955.031487:0:6696:0:(genops.c:1468:class_disconnect()) disconnect: cookie 0x1cd7fe5368731fbd 00000100:00080000:3.0:1788619955.031498:0:6696:0:(import.c:65:import_set_state_nolock()) ffff8e78e6776000 : changing import state from FULL to CLOSED 00000001:00000004:3.0:1788619955.031539:0:6696:0:(tgt_lastrcvd.c:727:tgt_server_data_update()) lustre-OST0000_UUID: mount_count is 1, last_transno is 4294967537 00000100:00100000:3.0:1788619955.032671:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+2:2351:x1875503571771776:12345-192.168.201.5@tcp:9:kworker.0 Request processed in 1215us (3803us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619955.032682:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1371 00000100:00100000:2.0:1788619955.032871:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00010000:00010000:2.0:1788619955.032959:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788619955.032974:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571771904 00000100:00100000:2.0:1788619955.032980:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571771904 00000100:00100000:2.0:1788619955.033025:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 217 00000100:00100000:2.0:1788619955.033034:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74a9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571771904:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:2.0:1788619955.033079:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74a9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2354:x1875503571771904:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 46us (209us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619955.033085:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 217 00000100:00080000:2.0:1788619955.033104:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1788619955.040233:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619955.040486:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571772032 00000100:00100000:3.0:1788619955.040496:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571772032 00000100:00100000:3.0:1788619955.040604:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 148 00000100:00100000:3.0:1788619955.040615:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74ab800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571772032:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:3.0:1788619955.040702:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74ab800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1b0df97a-4f84-4680-9108-88353da6ca92+6:2355:x1875503571772032:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 90us (482us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619955.040711:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 148 00000100:00080000:2.0:1788619955.041159:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1788619955.305614:0:3316:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1788619955.305637:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589288576:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1788619955.305707:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788619955.323419:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589288576 00000100:00100000:2.0:1788619955.323434:0:6699:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589288576 00000100:00100000:2.0:1788619955.323516:0:6700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 102 00000100:00100000:2.0:1788619955.323527:0:6700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503589288576:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1788619955.323562:0:6700:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3856662528 avail 3854565376 00000100:00100000:2.0:1788619955.323642:0:6700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503589288576:12345-0@lo:13:osp-pre-1-0.0 Request processed in 117us (17934us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619955.323649:0:6700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 102 00000100:00100000:2.0:1788619955.323706:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589288576:0@lo:13:osp-pre-1-0.0 00000001:00100000:3.0:1788619955.385508:0:6721:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0000: committing for initial connect of 92dda994-8428-47f5-a731-0b06f1aba2e4 00000020:00000080:3.0:1788619955.385518:0:6721:0:(genops.c:885:class_export_put()) final put ffff8e78c742b800/92dda994-8428-47f5-a731-0b06f1aba2e4 00000020:00000080:1.0:1788619955.385694:0:484:0:(genops.c:844:class_export_destroy()) destroying export ffff8e78c742b800/92dda994-8428-47f5-a731-0b06f1aba2e4 for lustre-OST0000 00000020:00000080:1.0:1788619955.385778:0:484:0:(genops.c:1183:obd_zombie_import_free()) destroying import ffff8e78e6776000 for lustre-OST0000 00010000:00010000:0.0:1788619956.004824:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788619956.322978:0:3319:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1788619956.323002:0:3319:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df8b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3319:1875503589288704:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1788619956.323072:0:3319:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619956.323188:0:6700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589288704 00000100:00100000:1.0:1788619956.323197:0:6700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589288704 00000100:00100000:1.0:1788619956.323256:0:6700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 103 00000100:00100000:1.0:1788619956.323265:0:6700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1875503589288704:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1788619956.323304:0:6700:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 100466688 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1788619956.323385:0:6700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1875503589288704:12345-0@lo:13:osp-pre-0-0.0 Request processed in 121us (312us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619956.323392:0:6700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 103 00000100:00080000:0.0:1788619956.323641:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1788619956.323682:0:3319:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df8b2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3319:1875503589288704:0@lo:13:osp-pre-0-0.0 00010000:00010000:0.0:1788619957.030539:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788619958.069627:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788619959.074793:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788619959.908727:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900e00380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589289216:0@lo:400:kworker.0 00000100:00100000:3.0:1788619959.908867:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619959.908952:0:3318:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:3.0:1788619959.908961:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:3318:1875503589288832:0@lo:400:kworker.0 00000100:00100000:3.0:1788619959.909003:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619959.909032:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900e01f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3318:1875503589289344:0@lo:400:kworker.0 00000100:00100000:3.0:1788619959.909082:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619959.909116:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589288832 00000100:00100000:1.0:1788619959.909206:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3316:1875503589288960:0@lo:400:kworker.0 00000100:00100000:3.0:1788619959.909228:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589288832 00000100:00100000:1.0:1788619959.909242:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619959.909265:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900e03100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589289472:0@lo:400:kworker.0 00000100:00100000:1.0:1788619959.909287:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619959.909290:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 306 00000100:00100000:3.0:1788619959.909300:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5cafc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+19:3318:x1875503589288832:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788619959.909316:0:3316:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1788619959.909323:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3316:1875503589289088:0@lo:400:kworker.0 00000100:00100000:1.0:1788619959.909367:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619959.909385:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5cafc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+19:3318:x1875503589288832:12345-0@lo:400:kworker.0 Request processed in 87us (381us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619959.909394:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 306 00000100:00100000:1.0:1788619959.909405:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589289216 00000100:00100000:1.0:1788619959.909409:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589289216 00000100:00100000:3.0:1788619959.909434:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589289472 00000100:00100000:3.0:1788619959.909437:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589289472 00000100:00100000:1.0:1788619959.909452:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 149 00000100:00100000:3.0:1788619959.909458:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 150 00000100:00100000:1.0:1788619959.909458:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e02a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589289216:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788619959.909463:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5019880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503589289472:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788619959.909490:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e02a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589289216:12345-0@lo:400:kworker.0 Request processed in 32us (624us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619959.909492:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5019880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503589289472:12345-0@lo:400:kworker.0 Request processed in 29us (204us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619959.909496:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 149 00000100:00100000:3.0:1788619959.909499:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 150 00000100:00100000:1.0:1788619959.909516:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589289344 00000100:00100000:1.0:1788619959.909519:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589289344 00000100:00100000:3.0:1788619959.909534:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:3318:1875503589288832:0@lo:400:kworker.0 00000100:00100000:1.0:1788619959.909548:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589288960 00000100:00100000:1.0:1788619959.909551:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589288960 00000100:00100000:3.0:1788619959.909553:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900e00380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589289216:0@lo:400:kworker.0 00000100:00100000:1.0:1788619959.909569:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589289088 00000100:00100000:1.0:1788619959.909572:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589289088 00000100:00100000:1.0:1788619959.909586:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 218 00000100:00100000:1.0:1788619959.909593:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e00a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3318:x1875503589289344:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788619959.909604:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 219 00000100:00100000:3.0:1788619959.909610:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df8b2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3316:x1875503589288960:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788619959.909628:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e00a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3318:x1875503589289344:12345-0@lo:400:kworker.0 Request processed in 36us (545us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619959.909635:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 218 00000100:00100000:3.0:1788619959.909636:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df8b2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3316:x1875503589288960:12345-0@lo:400:kworker.0 Request processed in 27us (394us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619959.909645:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 220 00000100:00100000:1.0:1788619959.909650:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5018a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3316:x1875503589289088:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788619959.910146:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5018a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3316:x1875503589289088:12345-0@lo:400:kworker.0 Request processed in 494us (776us total) trans 0 rc 0/0 00000100:00080000:0.0:1788619959.910537:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1788619959.910546:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1788619959.910550:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1788619959.910651:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3316:1875503589289088:0@lo:400:kworker.0 00000100:00100000:0.0:1788619959.910668:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3316:1875503589288960:0@lo:400:kworker.0 00000100:00100000:0.0:1788619959.910683:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900e03100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589289472:0@lo:400:kworker.0 00000100:00100000:0.0:1788619959.910912:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900e01f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3318:1875503589289344:0@lo:400:kworker.0 00000100:00100000:3.0:1788619959.911028:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 219 00000100:00100000:1.0:1788619959.913514:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 220 00010000:00010000:3.0:1788619960.098535:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788619960.137881:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619960.137921:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619960.137959:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619960.138005:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619960.138052:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571772544 00000100:00100000:2.0:1788619960.138063:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571772544 00000100:00100000:2.0:1788619960.138133:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 151 00000100:00100000:2.0:1788619960.138145:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010fd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571772544:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:2.0:1788619960.138222:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010fd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571772544:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 80us (216us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619960.138231:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 151 00000100:00100000:2.0:1788619960.138387:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571772416 00000100:00100000:2.0:1788619960.138391:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571772416 00000100:00100000:2.0:1788619960.138419:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571772288 00000100:00100000:2.0:1788619960.138422:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571772288 00000100:00100000:2.0:1788619960.138443:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 221 00000100:00100000:2.0:1788619960.138450:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c74a9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571772416:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:2.0:1788619960.138484:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c74a9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571772416:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 34us (562us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619960.138492:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 221 00000100:00100000:2.0:1788619960.138504:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 222 00000100:00100000:2.0:1788619960.138510:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010fca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2352:x1875503571772288:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:2.0:1788619960.138532:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010fca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2352:x1875503571772288:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 22us (572us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619960.138538:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 222 00000100:00080000:2.0:1788619960.138598:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1788619960.138604:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1788619960.148572:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571772160 00000100:00100000:1.0:1788619960.148590:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571772160 00000100:00100000:1.0:1788619960.148658:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 307 00000100:00100000:1.0:1788619960.148670:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5cac450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:597450a2-e90c-408c-b401-a0ce48eed204+13:2352:x1875503571772160:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:1.0:1788619960.148751:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5cac450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:597450a2-e90c-408c-b401-a0ce48eed204+13:2352:x1875503571772160:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 84us (10880us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619960.148759:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 307 00000100:00100000:2.0:1788619960.420634:0:3317:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5019180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3317:1875503589289600:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1788619960.420733:0:3317:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619960.421219:0:6700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589289600 00000100:00100000:3.0:1788619960.421231:0:6700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589289600 00000100:00100000:3.0:1788619960.421300:0:6700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 104 00000100:00100000:3.0:1788619960.421418:0:6700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3317:x1875503589289600:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1788619960.421982:0:6700:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3856662528 avail 3854565376 00000100:00100000:3.0:1788619960.428474:0:6700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3317:x1875503589289600:12345-0@lo:13:osp-pre-1-0.0 Request processed in 7147us (7723us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619960.428493:0:6700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 104 00000100:00100000:2.0:1788619960.435135:0:3317:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5019180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3317:1875503589289600:0@lo:13:osp-pre-1-0.0 00010000:00010000:0.0:1788619961.122509:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:2.0:1788619961.194517:0:14277:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: - need MDS1_VERSION <= 2.16.61-1-g89cf292a8c2 (34683395 <= 34618625) for LU-18938, skip 360 00000100:00100000:2.0:1788619961.442798:0:3319:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1788619961.442825:0:3319:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5018700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3319:1875503589289728:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1788619961.442899:0:3319:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619961.443068:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589289728 00000100:00100000:1.0:1788619961.443076:0:6699:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589289728 00000100:00100000:1.0:1788619961.443216:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 105 00000100:00100000:1.0:1788619961.443227:0:6699:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010ff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1875503589289728:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1788619961.443264:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 100466688 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1788619961.443322:0:6699:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010ff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1875503589289728:12345-0@lo:13:osp-pre-0-0.0 Request processed in 97us (423us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619961.443351:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 105 00000100:00100000:1.0:1788619961.443576:0:3319:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5018700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3319:1875503589289728:0@lo:13:osp-pre-0-0.0 00000100:00080000:2.0:1788619961.443687:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:3.0:1788619962.148306:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788619963.179180:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:0.0:1788619963.216120:0:14325:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: - need MDS1_VERSION < v2_14_55-100-g8a84c7f9c7 (34683395 < 34486116) for LU-14927, skip 0f 00010000:00010000:3.0:1788619964.194818:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788619965.033497:0:3317:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1788619965.033523:0:3317:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3317:1875503589289984:0@lo:400:kworker.0 00000100:00100000:2.0:1788619965.033594:0:3317:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788619965.033679:0:3317:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3317:1875503589290112:0@lo:400:kworker.0 00000100:00100000:2.0:1788619965.033704:0:3317:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619965.033828:0:3318:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1788619965.033839:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:3318:1875503589289856:0@lo:400:kworker.0 00000100:00100000:3.0:1788619965.033890:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619965.033975:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589289984 00000100:00100000:1.0:1788619965.033983:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589289984 00000100:00100000:2.0:1788619965.034026:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589290112 00000100:00100000:2.0:1788619965.034031:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589290112 00000100:00100000:0.0:1788619965.034308:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589289856 00000100:00100000:0.0:1788619965.034313:0:5836:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589289856 00000100:00100000:0.0:1788619965.034367:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 308 00000100:00100000:0.0:1788619965.034376:0:5836:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5cac050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+19:3318:x1875503589289856:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788619965.034426:0:5836:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5cac050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+19:3318:x1875503589289856:12345-0@lo:400:kworker.0 Request processed in 51us (536us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619965.034434:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 308 00000100:00100000:3.0:1788619965.038131:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:3318:1875503589289856:0@lo:400:kworker.0 00000100:00100000:1.0:1788619965.040066:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 223 00000100:00100000:1.0:1788619965.040084:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010fea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3317:x1875503589289984:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788619965.040179:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010fea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3317:x1875503589289984:12345-0@lo:400:kworker.0 Request processed in 102us (6585us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619965.040186:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 223 00000100:00100000:2.0:1788619965.041021:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 224 00000100:00100000:2.0:1788619965.041035:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010fc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3317:x1875503589290112:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788619965.041083:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010fc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3317:x1875503589290112:12345-0@lo:400:kworker.0 Request processed in 55us (7378us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619965.041089:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 224 00000100:00080000:2.0:1788619965.041114:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1788619965.041197:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5dc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589290496:0@lo:400:kworker.0 00000100:00100000:2.0:1788619965.041239:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788619965.041283:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589290496 00000100:00100000:2.0:1788619965.041290:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589290496 00000100:00100000:2.0:1788619965.041345:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 152 00000100:00100000:2.0:1788619965.041352:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dcbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503589290496:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788619965.041387:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dcbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503589290496:12345-0@lo:400:kworker.0 Request processed in 36us (147us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619965.041393:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 152 00000100:00080000:2.0:1788619965.041413:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1788619965.041440:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5dc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589290496:0@lo:400:kworker.0 00000100:00100000:3.0:1788619965.041829:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589290240:0@lo:400:kworker.0 00000100:00100000:3.0:1788619965.041876:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619965.041896:0:3318:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1788619965.041903:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3318:1875503589290368:0@lo:400:kworker.0 00000100:00100000:3.0:1788619965.041927:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619965.041966:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589290368 00000100:00100000:3.0:1788619965.041970:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589290368 00000100:00100000:3.0:1788619965.042004:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 225 00000100:00100000:3.0:1788619965.042011:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e00000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3318:x1875503589290368:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788619965.042048:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e00000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3318:x1875503589290368:12345-0@lo:400:kworker.0 Request processed in 38us (120us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619965.042054:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 225 00000100:00080000:3.0:1788619965.042070:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788619965.042089:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3318:1875503589290368:0@lo:400:kworker.0 00000100:00100000:2.0:1788619965.042820:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589290240 00000100:00100000:2.0:1788619965.042826:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589290240 00000100:00100000:2.0:1788619965.042879:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 153 00000100:00100000:2.0:1788619965.042887:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e00380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589290240:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788619965.042935:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e00380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589290240:12345-0@lo:400:kworker.0 Request processed in 49us (1071us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619965.042941:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 153 00000100:00080000:3.0:1788619965.046016:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1788619965.046076:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589290240:0@lo:400:kworker.0 00000100:00100000:1.0:1788619965.050091:0:3317:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3317:1875503589290112:0@lo:400:kworker.0 00000100:00100000:1.0:1788619965.050113:0:3317:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3317:1875503589289984:0@lo:400:kworker.0 00010000:00010000:3.0:1788619965.218618:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:0.0:1788619965.238823:0:14374:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: - need OST1_VERSION < v2_17_50-410-gbd8e7b9555 (34683395 < 34681754) for LU-12550, skip 216 00000100:00100000:2.0:1788619965.254745:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619965.254846:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619965.254879:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619965.254915:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619965.254956:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571772928 00000100:00100000:2.0:1788619965.254966:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571772928 00000100:00100000:0.0:1788619965.254978:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571772800 00000100:00100000:0.0:1788619965.254985:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571772800 00000100:00100000:2.0:1788619965.255030:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 226 00000100:00100000:2.0:1788619965.255043:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dc9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571772928:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:2.0:1788619965.255111:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dc9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571772928:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 72us (265us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619965.255120:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 226 00000100:00100000:2.0:1788619965.255137:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 227 00000100:00100000:2.0:1788619965.255144:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dcb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2352:x1875503571772800:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:2.0:1788619965.255172:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dcb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2352:x1875503571772800:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 29us (292us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619965.255178:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 227 00000100:00100000:2.0:1788619965.255199:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571773056 00000100:00100000:2.0:1788619965.255203:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571773056 00000100:00100000:2.0:1788619965.255229:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 154 00000100:00100000:2.0:1788619965.255235:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dcb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571773056:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:2.0:1788619965.255266:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dcb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571773056:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 32us (352us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619965.255272:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 154 00000100:00080000:2.0:1788619965.255288:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1788619965.255293:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788619965.258831:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571772672 00000100:00100000:3.0:1788619965.258844:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571772672 00000100:00100000:3.0:1788619965.258905:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 309 00000100:00100000:3.0:1788619965.258916:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5cad450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:597450a2-e90c-408c-b401-a0ce48eed204+13:2352:x1875503571772672:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:3.0:1788619965.259014:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5cad450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:597450a2-e90c-408c-b401-a0ce48eed204+13:2352:x1875503571772672:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 101us (4277us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619965.259021:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 309 00000100:00100000:2.0:1788619965.543553:0:3317:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3317:1875503589290624:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1788619965.543625:0:3317:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788619965.559804:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589290624 00000100:00100000:0.0:1788619965.559818:0:6699:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589290624 00000100:00100000:0.0:1788619965.559890:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 106 00000100:00100000:0.0:1788619965.559901:0:6699:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dc9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3317:x1875503589290624:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1788619965.559948:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3856662528 avail 3854565376 00000100:00100000:0.0:1788619965.560023:0:6699:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dc9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3317:x1875503589290624:12345-0@lo:13:osp-pre-1-0.0 Request processed in 124us (16397us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619965.560031:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 106 00000100:00100000:2.0:1788619965.563596:0:3317:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3317:1875503589290624:0@lo:13:osp-pre-1-0.0 00010000:00010000:1.0:1788619966.242979:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788619966.563136:0:3319:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1788619966.563166:0:3319:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5018700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3319:1875503589290752:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1788619966.563259:0:3319:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619966.563396:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589290752 00000100:00100000:1.0:1788619966.563406:0:6699:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589290752 00000100:00100000:1.0:1788619966.563463:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 107 00000100:00100000:1.0:1788619966.563473:0:6699:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5019f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1875503589290752:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1788619966.563519:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 100466688 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1788619966.563570:0:6699:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5019f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1875503589290752:12345-0@lo:13:osp-pre-0-0.0 Request processed in 99us (313us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619966.563578:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 107 00000100:00100000:1.0:1788619966.563641:0:3319:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5018700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3319:1875503589290752:0@lo:13:osp-pre-0-0.0 00000001:02000400:0.0:1788619966.934619:0:14422:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: excepting tests: 42a 42c 42b 118c 118d 407 119i 216 817 411a 130b 130c 130d 130e 130f 130g 00010000:00010000:3.0:1788619967.272039:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788619968.292780:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:0.0:1788619968.595567:0:14470:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: skipping tests SLOW=no: 27m 60i 64b 68 71 135 136 230d 300o 842 51b 51c 51e 834 00010000:00010000:3.0:1788619969.315100:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788619970.154668:0:3319:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:3319:1875503589290880:0@lo:400:kworker.0 00000100:00100000:0.0:1788619970.154754:0:3319:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788619970.154804:0:3319:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fefd5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3319:1875503589291392:0@lo:400:kworker.0 00000100:00100000:0.0:1788619970.154829:0:3319:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788619970.154866:0:3319:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1788619970.154872:0:3319:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3319:1875503589291264:0@lo:400:kworker.0 00000100:00100000:0.0:1788619970.154896:0:3319:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788619970.154925:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69ff480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3316:1875503589291008:0@lo:400:kworker.0 00000100:00100000:0.0:1788619970.154945:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788619970.154959:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fefd4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589291520:0@lo:400:kworker.0 00000100:00100000:0.0:1788619970.154978:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788619970.154992:0:3316:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1788619970.154996:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3316:1875503589291136:0@lo:400:kworker.0 00000100:00100000:0.0:1788619970.155014:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788619970.155043:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589290880 00000100:00100000:0.0:1788619970.155052:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589290880 00000100:00100000:0.0:1788619970.155102:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 310 00000100:00100000:0.0:1788619970.155111:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5cae850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+19:3319:x1875503589290880:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788619970.155167:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5cae850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+19:3319:x1875503589290880:12345-0@lo:400:kworker.0 Request processed in 56us (413us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619970.155174:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 310 00000100:00100000:0.0:1788619970.155210:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589291392 00000100:00100000:0.0:1788619970.155213:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589291392 00000100:00100000:0.0:1788619970.155246:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589291008 00000100:00100000:0.0:1788619970.155248:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589291008 00000100:00100000:0.0:1788619970.155261:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589291136 00000100:00100000:0.0:1788619970.155263:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589291136 00000100:00100000:0.0:1788619970.155276:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 228 00000100:00100000:0.0:1788619970.155282:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3319:x1875503589291392:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788619970.155311:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3319:x1875503589291392:12345-0@lo:400:kworker.0 Request processed in 29us (481us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619970.155317:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 228 00000100:00100000:0.0:1788619970.155356:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 229 00000100:00100000:0.0:1788619970.155362:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3316:x1875503589291008:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788619970.155387:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3316:x1875503589291008:12345-0@lo:400:kworker.0 Request processed in 26us (442us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619970.155393:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 229 00000100:00100000:0.0:1788619970.155402:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 230 00000100:00100000:0.0:1788619970.155407:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe001880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3316:x1875503589291136:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788619970.155427:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe001880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3316:x1875503589291136:12345-0@lo:400:kworker.0 Request processed in 20us (412us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619970.155432:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 230 00000100:00100000:0.0:1788619970.155447:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589291264 00000100:00100000:0.0:1788619970.155450:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589291264 00000100:00100000:0.0:1788619970.155473:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589291520 00000100:00100000:0.0:1788619970.155475:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589291520 00000100:00100000:0.0:1788619970.155490:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 155 00000100:00100000:0.0:1788619970.155495:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3319:x1875503589291264:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788619970.155521:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3319:x1875503589291264:12345-0@lo:400:kworker.0 Request processed in 26us (624us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619970.155526:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 155 00000100:00100000:0.0:1788619970.155534:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 156 00000100:00100000:0.0:1788619970.155539:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe002a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503589291520:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788619970.155558:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe002a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503589291520:12345-0@lo:400:kworker.0 Request processed in 19us (579us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619970.155563:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 156 00000100:00080000:0.0:1788619970.155591:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1788619970.155595:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1788619970.155597:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1788619970.155637:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3316:1875503589291136:0@lo:400:kworker.0 00000100:00100000:0.0:1788619970.155646:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69ff480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3316:1875503589291008:0@lo:400:kworker.0 00000100:00100000:0.0:1788619970.155653:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fefd4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589291520:0@lo:400:kworker.0 00000100:00100000:0.0:1788619970.155681:0:3319:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69fce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3319:1875503589291264:0@lo:400:kworker.0 00000100:00100000:0.0:1788619970.155689:0:3319:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:3319:1875503589290880:0@lo:400:kworker.0 00000100:00100000:0.0:1788619970.155698:0:3319:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fefd5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3319:1875503589291392:0@lo:400:kworker.0 00010000:00010000:0.0:1788619970.339377:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788619970.381341:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619970.381400:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619970.381434:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619970.381536:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619970.381588:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571773440 00000100:00100000:1.0:1788619970.381601:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571773440 00000100:00100000:1.0:1788619970.381683:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571773312 00000100:00100000:1.0:1788619970.381687:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571773312 00000100:00100000:1.0:1788619970.381713:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 231 00000100:00100000:1.0:1788619970.381724:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5018700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571773440:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:1.0:1788619970.381829:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5018700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1b0df97a-4f84-4680-9108-88353da6ca92+9:2352:x1875503571773440:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 107us (428us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619970.381837:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 231 00000100:00100000:1.0:1788619970.381867:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 232 00000100:00100000:1.0:1788619970.381874:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5019500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2352:x1875503571773312:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:1.0:1788619970.381897:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5019500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2352:x1875503571773312:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 25us (464us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619970.381901:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 232 00000100:00080000:1.0:1788619970.381916:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1788619970.382031:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571773568 00000100:00100000:1.0:1788619970.382036:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571773568 00000100:00100000:1.0:1788619970.382075:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 157 00000100:00100000:1.0:1788619970.382082:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571773568:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:1.0:1788619970.382119:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:1b0df97a-4f84-4680-9108-88353da6ca92+6:2352:x1875503571773568:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 37us (583us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619970.382126:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 157 00000100:00080000:1.0:1788619970.382162:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1788619970.382262:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571773184 00000100:00100000:0.0:1788619970.382266:0:5836:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571773184 00000100:00100000:0.0:1788619970.382298:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 311 00000100:00100000:0.0:1788619970.382303:0:5836:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c9bc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:597450a2-e90c-408c-b401-a0ce48eed204+13:2352:x1875503571773184:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:0.0:1788619970.382356:0:5836:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c9bc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:597450a2-e90c-408c-b401-a0ce48eed204+13:2352:x1875503571773184:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 54us (1052us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619970.382360:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 311 00000001:02000400:3.0:1788619970.583249:0:14518:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: === sanity: start setup 10:52:48 (1788619968) === 00000100:00100000:3.0:1788619970.658748:0:3316:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1788619970.658781:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900e01f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589291648:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1788619970.658888:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619970.659107:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589291648 00000100:00100000:1.0:1788619970.659126:0:6699:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589291648 00000100:00100000:1.0:1788619970.659206:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 108 00000100:00100000:1.0:1788619970.659218:0:6699:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e03100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503589291648:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1788619970.659266:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3856662528 avail 3854565376 00000100:00100000:1.0:1788619970.659348:0:6699:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e03100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503589291648:12345-0@lo:13:osp-pre-1-0.0 Request processed in 111us (438us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619970.659357:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 108 00000100:00100000:1.0:1788619970.659416:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900e01f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589291648:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1788619970.699105:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619970.699179:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571773696 00000100:00100000:3.0:1788619970.699189:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571773696 00000100:00100000:3.0:1788619970.699250:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1666 00000100:00100000:3.0:1788619970.699267:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dc8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+9:15483:x1875503571773696:12345-192.168.201.5@tcp:101:bash.0 00010000:00010000:3.0:1788619970.699286:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788619970.701584:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06a00/0x1cd7fe5368737083 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1e018 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619970.701671:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06000/0x1cd7fe536873708a lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619970.701686:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06000/0x1cd7fe536873708a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619970.701701:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06000/0x1cd7fe536873708a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788619970.701737:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:3.0:1788619970.701741:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1788619970.701764:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06000/0x1cd7fe536873708a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619970.701795:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06a00/0x1cd7fe5368737083 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e018 expref: 11 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619970.701816:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06000/0x1cd7fe536873708a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e018 expref: 10 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619970.701825:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78c7cbbe00) 00010000:00010000:3.0:1788619970.701828:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788619970.701831:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa06000, rc 0) 00000100:00100000:3.0:1788619970.701913:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dc8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:1b0df97a-4f84-4680-9108-88353da6ca92+10:15483:x1875503571773696:12345-192.168.201.5@tcp:101:bash.0 Request processed in 2647us (2815us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619970.701922:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1666 00010000:00010000:3.0:1788619971.362692:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788619971.684153:0:3319:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1788619971.684180:0:3319:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3319:1875503589291776:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1788619971.684280:0:3319:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619971.684780:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589291776 00000100:00100000:3.0:1788619971.684788:0:6699:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589291776 00000100:00100000:3.0:1788619971.684853:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 109 00000100:00100000:3.0:1788619971.684862:0:6699:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dc8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1875503589291776:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1788619971.685294:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 100466688 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1788619971.685391:0:6699:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dc8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1875503589291776:12345-0@lo:13:osp-pre-0-0.0 Request processed in 530us (1119us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619971.685399:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 109 00000100:00100000:2.0:1788619971.685491:0:3319:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3319:1875503589291776:0@lo:13:osp-pre-0-0.0 00010000:00010000:0.0:1788619972.386498:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788619973.410528:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788619973.414561:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619973.414746:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619973.421103:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619973.423682:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571773824 00000100:00100000:1.0:1788619973.423710:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571773824 00000100:00100000:1.0:1788619973.423895:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571773952 00000100:00100000:1.0:1788619973.423900:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571773952 00000100:00100000:2.0:1788619973.424055:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 9 00000100:00100000:2.0:1788619973.424073:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe000e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:597450a2-e90c-408c-b401-a0ce48eed204+14:2354:x1875503571773824:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619973.424096:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619973.424125:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff8e78c8078c00/0x1cd7fe53687314f0 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a194a7 expref: 14 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619973.424148:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619973.429478:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe000e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:597450a2-e90c-408c-b401-a0ce48eed204+13:2354:x1875503571773824:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 259us (9796us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619973.429503:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 9 00000100:00100000:2.0:1788619973.429572:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 10 00000100:00100000:2.0:1788619973.429666:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe003480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:597450a2-e90c-408c-b401-a0ce48eed204+12:2354:x1875503571773952:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619973.429694:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619973.429727:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff8e78c8078e00/0x1cd7fe53687314f7 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 5 type: PLN flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a194d1 expref: 12 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619973.429751:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619973.429834:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe003480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:597450a2-e90c-408c-b401-a0ce48eed204+11:2354:x1875503571773952:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 252us (15087us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619973.429841:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 10 00000100:00100000:0.0:1788619973.431867:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619973.431966:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571774336 00000100:00100000:1.0:1788619973.431979:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571774336 00000100:00100000:1.0:1788619973.432057:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1372 00000100:00100000:1.0:1788619973.432069:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe001f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+6:2351:x1875503571774336:12345-192.168.201.5@tcp:9:umount.0 00000020:00000080:1.0:1788619973.432108:0:7654:0:(genops.c:1468:class_disconnect()) disconnect: cookie 0x1cd7fe536873192d 00000020:00010000:1.0:1788619973.432118:0:7654:0:(genops.c:1921:obd_stale_export_put()) Put export ffff8e78cdce6800: total 0 00000100:00080000:1.0:1788619973.432123:0:7654:0:(import.c:65:import_set_state_nolock()) ffff8e78cdce0800 : changing import state from FULL to CLOSED 00000001:00000004:1.0:1788619973.432177:0:7654:0:(tgt_lastrcvd.c:727:tgt_server_data_update()) lustre-OST0000_UUID: mount_count is 1, last_transno is 4294967537 00000100:00100000:1.0:1788619973.433942:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe001f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1b0df97a-4f84-4680-9108-88353da6ca92+4:2351:x1875503571774336:12345-192.168.201.5@tcp:9:umount.0 Request processed in 1861us (2072us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619973.433958:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1372 00000020:00010000:1.0:1788619973.433995:0:5833:0:(genops.c:1907:obd_stale_export_get()) Get export ffff8e78cdce6800: total 0 00010000:00010000:1.0:1788619973.434028:0:5833:0:(ldlm_lock.c:2651:ldlm_export_cancel_blocked_locks()) Export ffff8e78cdce6800, canceled 0 locks, left on hash table 1. 00010000:00010000:1.0:1788619973.434098:0:5833:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000081ae252b/0x1cd7fe5368737021 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000020000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dfc4 expref: 3 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619973.434128:0:5833:0:(ldlm_lock.c:2679:ldlm_export_cancel_locks()) Export ffff8e78cdce6800, canceled 1 locks, left on hash table 0. 00010000:00010000:1.0:1788619973.434140:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788619973.434142:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788619973.434144:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1788619973.434233:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1788619973.445213:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571774080 00000100:00100000:2.0:1788619973.445237:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571774080 00000100:00100000:2.0:1788619973.445381:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 11 00000100:00100000:2.0:1788619973.445394:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe002680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:597450a2-e90c-408c-b401-a0ce48eed204+11:2355:x1875503571774080:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788619973.445416:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619973.445442:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff8e78ccca6800/0x1cd7fe5368731521 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 7 type: PLN flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a19502 expref: 11 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619973.445464:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619973.445552:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe002680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:597450a2-e90c-408c-b401-a0ce48eed204+10:2355:x1875503571774080:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 159us (24465us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619973.445560:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 11 00000100:00100000:0.0:1788619973.462072:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619973.462178:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571774848 00000100:00100000:1.0:1788619973.462197:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571774848 00000100:00080000:0.0:1788619973.462294:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788619973.462760:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1667 00000100:00100000:3.0:1788619973.462779:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe000700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+10:2351:x1875503571774848:12345-192.168.201.5@tcp:39:umount.0 00000020:00000080:3.0:1788619973.462838:0:5844:0:(genops.c:1468:class_disconnect()) disconnect: cookie 0x1cd7fe5368731505 00000020:00010000:3.0:1788619973.462858:0:5844:0:(genops.c:1921:obd_stale_export_put()) Put export ffff8e7900f0c800: total 0 00000100:00080000:3.0:1788619973.462863:0:5844:0:(import.c:65:import_set_state_nolock()) ffff8e7900f0b800 : changing import state from FULL to CLOSED 00000001:00000004:3.0:1788619973.462896:0:5844:0:(tgt_lastrcvd.c:727:tgt_server_data_update()) lustre-MDT0000_UUID: mount_count is 1, last_transno is 4294967888 00000020:00010000:0.0:1788619973.463640:0:5832:0:(genops.c:1907:obd_stale_export_get()) Get export ffff8e7900f0c800: total 0 00010000:00010000:0.0:1788619973.463647:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788619973.463712:0:5832:0:(ldlm_lock.c:2651:ldlm_export_cancel_blocked_locks()) Export ffff8e7900f0c800, canceled 0 locks, left on hash table 5. 00010000:00010000:0.0:1788619973.463755:0:5832:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb:0x0].0x0 (ffff8e78c3796b00) 00010000:00010000:0.0:1788619973.463760:0:5832:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788619973.463772:0:5832:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07000/0x1cd7fe5368731cd0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a19786 expref: 9 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619973.463809:0:5832:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619973.463812:0:5832:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1788619973.463820:0:5832:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78cdcc6c00/0x1cd7fe5368731fa8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xe:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a19866 expref: 8 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619973.463836:0:5832:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xe:0x0].0x0 (ffff8e78c67bcf00) 00010000:00010000:0.0:1788619973.463838:0:5832:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788619973.463842:0:5832:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe536873702f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1dfcb expref: 7 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619973.463857:0:5832:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78c7cbbe00) 00010000:00010000:0.0:1788619973.463860:0:5832:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788619973.463863:0:5832:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06000/0x1cd7fe536873708a lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e018 expref: 6 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1788619973.463918:0:5832:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xb:0x0] updating lvb blocks from disk: 2 -> 10 00010000:00010000:0.0:1788619973.463928:0:5832:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb:0x0].0x0 (ffff8e78c3796b00) 00010000:00010000:0.0:1788619973.463931:0:5832:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5b 00010000:00010000:0.0:1788619973.463936:0:5832:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e0296000/0x1cd7fe5368731d24 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb:0x0].0x0 bits 0x5b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a197b0 expref: 5 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619973.463948:0:5832:0:(ldlm_lock.c:2679:ldlm_export_cancel_locks()) Export ffff8e7900f0c800, canceled 5 locks, left on hash table 0. 00010000:00010000:0.0:1788619973.463959:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788619973.463961:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788619973.463963:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788619973.464603:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe000700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1b0df97a-4f84-4680-9108-88353da6ca92+3:2351:x1875503571774848:12345-192.168.201.5@tcp:39:umount.0 Request processed in 1827us (2535us total) trans 4294967889 rc 0/0 00000100:00100000:3.0:1788619973.464617:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1667 00010000:00010000:1.0:1788619974.435634:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788619975.266725:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3316:1875503589292032:0@lo:400:kworker.0 00000100:00100000:3.0:1788619975.266811:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619975.266860:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c4be2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589292544:0@lo:400:kworker.0 00000100:00100000:3.0:1788619975.266885:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619975.266912:0:3316:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1788619975.266917:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3316:1875503589292160:0@lo:400:kworker.0 00000100:00100000:3.0:1788619975.266938:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619975.266975:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589292032 00000100:00100000:3.0:1788619975.266983:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589292032 00000100:00100000:3.0:1788619975.267028:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589292160 00000100:00100000:3.0:1788619975.267031:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589292160 00000100:00100000:3.0:1788619975.267062:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 233 00000100:00100000:3.0:1788619975.267071:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3316:x1875503589292032:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788619975.267125:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3316:x1875503589292032:12345-0@lo:400:kworker.0 Request processed in 54us (313us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619975.267132:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 233 00000100:00100000:3.0:1788619975.267150:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 234 00000100:00100000:3.0:1788619975.267156:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd939180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3316:x1875503589292160:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788619975.267182:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd939180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3316:x1875503589292160:12345-0@lo:400:kworker.0 Request processed in 27us (243us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619975.267187:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 234 00000100:00080000:3.0:1788619975.267205:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788619975.267238:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3316:1875503589292160:0@lo:400:kworker.0 00000100:00100000:3.0:1788619975.267247:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3316:1875503589292032:0@lo:400:kworker.0 00000100:00100000:1.0:1788619975.267357:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589292288:0@lo:400:kworker.0 00000100:00100000:1.0:1788619975.267387:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619975.267406:0:3318:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:1.0:1788619975.267412:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e7900e01f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:3318:1875503589291904:0@lo:400:kworker.0 00000100:00100000:1.0:1788619975.267441:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619975.267465:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3318:1875503589292416:0@lo:400:kworker.0 00000100:00100000:1.0:1788619975.267484:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619975.267508:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589292416 00000100:00100000:1.0:1788619975.267512:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589292416 00000100:00100000:1.0:1788619975.267542:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 235 00000100:00100000:1.0:1788619975.267548:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5018000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3318:x1875503589292416:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788619975.267584:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5018000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3318:x1875503589292416:12345-0@lo:400:kworker.0 Request processed in 36us (98us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619975.267590:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 235 00000100:00100000:1.0:1788619975.267608:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589292544 00000100:00100000:1.0:1788619975.267612:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589292544 00000100:00100000:1.0:1788619975.267632:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589292288 00000100:00100000:1.0:1788619975.267635:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589292288 00000100:00100000:1.0:1788619975.267654:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 158 00000100:00100000:1.0:1788619975.267660:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503589292544:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788619975.267694:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503589292544:12345-0@lo:400:kworker.0 Request processed in 34us (808us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619975.267700:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 158 00000100:00100000:1.0:1788619975.267712:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 159 00000100:00100000:1.0:1788619975.267718:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589292288:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788619975.267747:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589292288:12345-0@lo:400:kworker.0 Request processed in 30us (360us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619975.267752:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 159 00000100:00100000:1.0:1788619975.267780:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589291904 00000100:00100000:1.0:1788619975.267784:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589291904 00000100:00100000:1.0:1788619975.267809:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 312 00000100:00100000:1.0:1788619975.267815:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c98450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+19:3318:x1875503589291904:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788619975.267842:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c98450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+19:3318:x1875503589291904:12345-0@lo:400:kworker.0 Request processed in 27us (401us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619975.267848:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 312 00000100:00080000:1.0:1788619975.267877:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1788619975.267881:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1788619975.267883:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1788619975.267902:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e7900e01f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:3318:1875503589291904:0@lo:400:kworker.0 00000100:00100000:1.0:1788619975.267913:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3318:1875503589292416:0@lo:400:kworker.0 00000100:00100000:1.0:1788619975.267921:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589292288:0@lo:400:kworker.0 00000100:00100000:1.0:1788619975.267948:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c4be2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589292544:0@lo:400:kworker.0 00000001:00100000:0.0:1788619975.339938:0:5867:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-MDT0000: committing for initial connect of 1b0df97a-4f84-4680-9108-88353da6ca92 00000001:00080000:0.0:1788619975.339955:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967889 is committed 00000020:00000080:0.0:1788619975.339961:0:5867:0:(genops.c:885:class_export_put()) final put ffff8e7900f0c800/1b0df97a-4f84-4680-9108-88353da6ca92 00000020:00000080:0.0:1788619975.339984:0:484:0:(genops.c:844:class_export_destroy()) destroying export ffff8e7900f0c800/1b0df97a-4f84-4680-9108-88353da6ca92 for lustre-MDT0000 00000020:00000080:0.0:1788619975.340056:0:484:0:(genops.c:1183:obd_zombie_import_free()) destroying import ffff8e7900f0b800 for lustre-MDT0000 00010000:00010000:1.0:1788619975.474401:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788619975.505008:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619975.505049:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619975.505078:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571775104 00000100:00100000:3.0:1788619975.505088:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571775104 00000100:00100000:3.0:1788619975.505154:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 236 00000100:00100000:3.0:1788619975.505164:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010fe300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2353:x1875503571775104:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:3.0:1788619975.505245:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010fe300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2353:x1875503571775104:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 82us (247us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619975.505253:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 236 00000100:00100000:3.0:1788619975.505273:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571774976 00000100:00100000:3.0:1788619975.505276:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571774976 00000100:00100000:3.0:1788619975.505299:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 313 00000100:00100000:3.0:1788619975.505305:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c98050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:597450a2-e90c-408c-b401-a0ce48eed204+10:2353:x1875503571774976:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:3.0:1788619975.505346:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c98050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:597450a2-e90c-408c-b401-a0ce48eed204+10:2353:x1875503571774976:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 41us (296us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619975.505352:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 313 00000100:00080000:3.0:1788619975.505375:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1788619975.779548:0:3317:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3317:1875503589292672:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1788619975.779619:0:3317:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788619975.779684:0:6700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589292672 00000100:00100000:2.0:1788619975.779691:0:6700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589292672 00000100:00100000:2.0:1788619975.779738:0:6700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 110 00000100:00100000:2.0:1788619975.779746:0:6700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dc8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3317:x1875503589292672:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1788619975.780222:0:6700:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3856662528 avail 3854565376 00000100:00100000:2.0:1788619975.780279:0:6700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dc8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3317:x1875503589292672:12345-0@lo:13:osp-pre-1-0.0 Request processed in 533us (660us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619975.780286:0:6700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 110 00000100:00100000:1.0:1788619975.780381:0:3317:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3317:1875503589292672:0@lo:13:osp-pre-1-0.0 00000001:00100000:2.0:1788619975.836675:0:6721:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0000: committing for initial connect of 1b0df97a-4f84-4680-9108-88353da6ca92 00000020:00000080:2.0:1788619975.836686:0:6721:0:(genops.c:885:class_export_put()) final put ffff8e78cdce6800/1b0df97a-4f84-4680-9108-88353da6ca92 00000020:00000080:1.0:1788619975.848168:0:37:0:(genops.c:844:class_export_destroy()) destroying export ffff8e78cdce6800/1b0df97a-4f84-4680-9108-88353da6ca92 for lustre-OST0000 00000020:00000080:1.0:1788619975.848221:0:37:0:(genops.c:1183:obd_zombie_import_free()) destroying import ffff8e78cdce0800 for lustre-OST0000 00010000:00010000:2.0:1788619976.504402:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788619976.805654:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589292800:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1788619976.805726:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619976.805882:0:6700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589292800 00000100:00100000:1.0:1788619976.805892:0:6700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589292800 00000100:00100000:1.0:1788619976.805942:0:6700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 111 00000100:00100000:1.0:1788619976.805952:0:6700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dc9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589292800:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1788619976.805990:0:6700:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1788619976.806056:0:6700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dc9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589292800:12345-0@lo:13:osp-pre-0-0.0 Request processed in 104us (328us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619976.806063:0:6700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 111 00000100:00100000:0.0:1788619976.806147:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589292800:0@lo:13:osp-pre-0-0.0 00010000:00010000:1.0:1788619977.506686:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788619978.531117:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788619979.556637:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:2.0:1788619980.000819:0:14715:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: oleg105-client.virtnet: executing check_config_client /mnt/lustre 00000100:00100000:2.0:1788619980.386790:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df384380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589293312:0@lo:400:kworker.0 00000100:00100000:0.0:1788619980.386843:0:3319:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:3319:1875503589292928:0@lo:400:kworker.0 00000100:00100000:0.0:1788619980.386891:0:3319:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619980.386964:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3316:1875503589293056:0@lo:400:kworker.0 00000100:00100000:2.0:1788619980.387289:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788619980.388200:0:3319:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df384e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3319:1875503589293440:0@lo:400:kworker.0 00000100:00100000:1.0:1788619980.390000:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619980.390072:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df386300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589293568:0@lo:400:kworker.0 00000100:00100000:1.0:1788619980.390109:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788619980.390199:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589293056 00000100:00100000:2.0:1788619980.390210:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589293056 00000100:00100000:0.0:1788619980.391184:0:3319:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788619980.391479:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589292928 00000100:00100000:0.0:1788619980.391486:0:5836:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589292928 00000100:00100000:0.0:1788619980.391531:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 314 00000100:00100000:0.0:1788619980.391543:0:5836:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c9a850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+19:3319:x1875503589292928:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788619980.391616:0:5836:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c9a850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+19:3319:x1875503589292928:12345-0@lo:400:kworker.0 Request processed in 75us (4725us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619980.391623:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 314 00000100:00100000:0.0:1788619980.391672:0:3317:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3317:1875503589293184:0@lo:400:kworker.0 00000100:00100000:0.0:1788619980.391717:0:3317:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788619980.391758:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589293440 00000100:00100000:0.0:1788619980.391763:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589293440 00000100:00100000:2.0:1788619980.392510:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589293184 00000100:00100000:2.0:1788619980.392517:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589293184 00000100:00100000:0.0:1788619980.394711:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 237 00000100:00100000:0.0:1788619980.394742:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5018380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3316:x1875503589293056:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788619980.395480:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5018380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3316:x1875503589293056:12345-0@lo:400:kworker.0 Request processed in 753us (5487us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619980.395490:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 237 00000100:00100000:0.0:1788619980.395522:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 238 00000100:00100000:0.0:1788619980.395531:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe000a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3319:x1875503589293440:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788619980.395570:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe000a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3319:x1875503589293440:12345-0@lo:400:kworker.0 Request processed in 42us (4390us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619980.395576:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 238 00000100:00080000:0.0:1788619980.395613:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1788619980.396471:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 239 00000100:00100000:2.0:1788619980.396486:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe001500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3317:x1875503589293184:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788619980.396551:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe001500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3317:x1875503589293184:12345-0@lo:400:kworker.0 Request processed in 72us (4833us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619980.396558:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 239 00000100:00100000:0.0:1788619980.397157:0:3317:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3317:1875503589293184:0@lo:400:kworker.0 00000100:00100000:1.0:1788619980.398453:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3316:1875503589293056:0@lo:400:kworker.0 00000100:00100000:3.0:1788619980.398568:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589293312 00000100:00100000:3.0:1788619980.398579:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589293312 00000100:00100000:2.0:1788619980.401263:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589293568 00000100:00100000:2.0:1788619980.401281:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589293568 00000100:00100000:2.0:1788619980.401314:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 160 00000100:00080000:3.0:1788619980.401465:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1788619980.401470:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1788619980.401484:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 161 00000100:00100000:3.0:1788619980.401496:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503589293568:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788619980.401621:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503589293568:12345-0@lo:400:kworker.0 Request processed in 129us (11510us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619980.401629:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 161 00000100:00100000:2.0:1788619980.403563:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5dcad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589293312:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788619980.403718:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5dcad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589293312:12345-0@lo:400:kworker.0 Request processed in 2390us (16425us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619980.403729:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 160 00000100:00100000:2.0:1788619980.403893:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df384380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589293312:0@lo:400:kworker.0 00000100:00100000:0.0:1788619980.409050:0:3319:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:3319:1875503589292928:0@lo:400:kworker.0 00000100:00100000:0.0:1788619980.409080:0:3319:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df384e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3319:1875503589293440:0@lo:400:kworker.0 00000100:00100000:0.0:1788619980.409167:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df386300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589293568:0@lo:400:kworker.0 00010000:00010000:2.0:1788619980.578837:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788619980.619564:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619980.619639:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619980.619778:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571775360 00000100:00100000:0.0:1788619980.619792:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571775360 00000100:00100000:0.0:1788619980.619898:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 240 00000100:00100000:0.0:1788619980.619910:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571775360:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:0.0:1788619980.620011:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571775360:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 104us (464us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619980.620019:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 240 00000100:00080000:1.0:1788619980.620323:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788619980.623894:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571775232 00000100:00100000:3.0:1788619980.623917:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571775232 00000100:00100000:3.0:1788619980.624016:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 315 00000100:00100000:3.0:1788619980.624027:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c99450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:597450a2-e90c-408c-b401-a0ce48eed204+10:2354:x1875503571775232:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:3.0:1788619980.624154:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c99450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:597450a2-e90c-408c-b401-a0ce48eed204+10:2354:x1875503571775232:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 128us (4514us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619980.624162:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 315 00000100:00100000:0.0:1788619980.898842:0:3316:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1788619980.898875:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fe003b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589293696:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1788619980.898989:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788619980.899080:0:6700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589293696 00000100:00100000:0.0:1788619980.899091:0:6700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589293696 00000100:00100000:0.0:1788619980.899154:0:6700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 112 00000100:00100000:0.0:1788619980.899165:0:6700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe001180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503589293696:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1788619980.899213:0:6700:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3856662528 avail 3854565376 00000100:00100000:0.0:1788619980.899263:0:6700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe001180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503589293696:12345-0@lo:13:osp-pre-1-0.0 Request processed in 100us (275us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619980.899272:0:6700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 112 00000100:00100000:0.0:1788619980.899354:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fe003b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589293696:0@lo:13:osp-pre-1-0.0 00010000:00010000:2.0:1788619981.619480:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788619981.923932:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589293824:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1788619981.924007:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788619981.924167:0:6700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589293824 00000100:00100000:0.0:1788619981.924296:0:6700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589293824 00000100:00100000:0.0:1788619981.924439:0:6700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 113 00000100:00100000:0.0:1788619981.924461:0:6700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe003b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589293824:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1788619981.924500:0:6700:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1788619981.924552:0:6700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe003b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589293824:12345-0@lo:13:osp-pre-0-0.0 Request processed in 104us (544us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619981.924561:0:6700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 113 00000100:00100000:0.0:1788619981.924786:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589293824:0@lo:13:osp-pre-0-0.0 00010000:00010000:1.0:1788619982.631136:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788619983.650500:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788619984.675506:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788619985.508707:0:3319:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df384700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:3319:1875503589293952:0@lo:400:kworker.0 00000100:00100000:3.0:1788619985.508779:0:3319:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619985.508824:0:3319:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df385c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3319:1875503589294464:0@lo:400:kworker.0 00000100:00100000:3.0:1788619985.508852:0:3319:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619985.508893:0:3319:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1788619985.508900:0:3319:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df386a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3319:1875503589294336:0@lo:400:kworker.0 00000100:00100000:3.0:1788619985.508927:0:3319:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619985.508957:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589294464 00000100:00100000:3.0:1788619985.508965:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589294464 00000100:00100000:3.0:1788619985.509017:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 241 00000100:00100000:3.0:1788619985.509026:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df384e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3319:x1875503589294464:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788619985.509215:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df384e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3319:x1875503589294464:12345-0@lo:400:kworker.0 Request processed in 189us (361us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619985.509223:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 241 00000100:00100000:3.0:1788619985.509246:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589294336 00000100:00100000:3.0:1788619985.509250:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589294336 00000100:00100000:3.0:1788619985.509278:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 162 00000100:00100000:3.0:1788619985.509285:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df384380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3319:x1875503589294336:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788619985.509316:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df384380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3319:x1875503589294336:12345-0@lo:400:kworker.0 Request processed in 32us (389us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619985.509322:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 162 00000100:00100000:3.0:1788619985.509391:0:3319:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df386a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3319:1875503589294336:0@lo:400:kworker.0 00000100:00100000:3.0:1788619985.509401:0:3319:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df385c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3319:1875503589294464:0@lo:400:kworker.0 00000100:00100000:3.0:1788619985.509430:0:3317:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df386680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3317:1875503589294208:0@lo:400:kworker.0 00000100:00100000:3.0:1788619985.509452:0:3317:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619985.509471:0:3317:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1788619985.509477:0:3317:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df385500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3317:1875503589294080:0@lo:400:kworker.0 00000100:00100000:3.0:1788619985.509494:0:3317:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619985.509507:0:3317:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df386300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3317:1875503589294592:0@lo:400:kworker.0 00000100:00100000:3.0:1788619985.509525:0:3317:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619985.509552:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589294592 00000100:00100000:3.0:1788619985.509555:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589294592 00000100:00100000:3.0:1788619985.509579:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 163 00000100:00100000:3.0:1788619985.509585:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3317:x1875503589294592:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788619985.509629:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3317:x1875503589294592:12345-0@lo:400:kworker.0 Request processed in 44us (103us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619985.509634:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 163 00000100:00100000:3.0:1788619985.509656:0:3317:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df386300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3317:1875503589294592:0@lo:400:kworker.0 00000100:00080000:3.0:1788619985.509677:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1788619985.509681:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1788619985.509684:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788619985.509696:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589294208 00000100:00100000:3.0:1788619985.509699:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589294208 00000100:00100000:3.0:1788619985.509723:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589294080 00000100:00100000:3.0:1788619985.509726:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589294080 00000100:00100000:3.0:1788619985.509741:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 242 00000100:00100000:3.0:1788619985.509746:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df385c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3317:x1875503589294208:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788619985.509779:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df385c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3317:x1875503589294208:12345-0@lo:400:kworker.0 Request processed in 33us (326us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619985.509785:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 242 00000100:00100000:3.0:1788619985.509795:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 243 00000100:00100000:3.0:1788619985.509801:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df386a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3317:x1875503589294080:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788619985.509825:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df386a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3317:x1875503589294080:12345-0@lo:400:kworker.0 Request processed in 25us (330us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619985.509830:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 243 00000100:00080000:3.0:1788619985.509844:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1788619985.509985:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589293952 00000100:00100000:0.0:1788619985.509990:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589293952 00000100:00100000:0.0:1788619985.510016:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 316 00000100:00100000:0.0:1788619985.510022:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c9b050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+19:3319:x1875503589293952:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788619985.510066:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c9b050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+19:3319:x1875503589293952:12345-0@lo:400:kworker.0 Request processed in 35us (1276us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619985.510071:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 316 00000100:00100000:0.0:1788619985.510368:0:3319:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df384700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:3319:1875503589293952:0@lo:400:kworker.0 00000100:00100000:3.0:1788619985.511141:0:3317:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df385500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3317:1875503589294080:0@lo:400:kworker.0 00000100:00100000:3.0:1788619985.511164:0:3317:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df386680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3317:1875503589294208:0@lo:400:kworker.0 00010000:00010000:2.0:1788619985.702143:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788619985.736155:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788619985.736216:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619985.736284:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571775488 00000100:00100000:0.0:1788619985.736298:0:5836:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571775488 00000100:00100000:0.0:1788619985.736392:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 317 00000100:00100000:0.0:1788619985.736403:0:5836:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c9a050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:597450a2-e90c-408c-b401-a0ce48eed204+10:2355:x1875503571775488:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:0.0:1788619985.736506:0:5836:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c9a050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:597450a2-e90c-408c-b401-a0ce48eed204+10:2355:x1875503571775488:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 104us (288us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619985.736514:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 317 00000100:00100000:3.0:1788619985.737547:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571775616 00000100:00100000:3.0:1788619985.737556:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571775616 00000100:00100000:3.0:1788619985.737636:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 244 00000100:00100000:3.0:1788619985.737645:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5019180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571775616:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:3.0:1788619985.737732:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5019180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571775616:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 88us (1591us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619985.737739:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 244 00000100:00080000:1.0:1788619985.737952:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1788619986.036112:0:3316:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1788619986.036233:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589294720:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1788619986.036357:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788619986.036612:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589294720 00000100:00100000:0.0:1788619986.036624:0:6699:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589294720 00000100:00100000:0.0:1788619986.037401:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 114 00000100:00100000:0.0:1788619986.037417:0:6699:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fe000000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503589294720:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1788619986.037469:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3856662528 avail 3854565376 00000100:00100000:0.0:1788619986.037541:0:6699:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fe000000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503589294720:12345-0@lo:13:osp-pre-1-0.0 Request processed in 127us (1184us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619986.037549:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 114 00000100:00100000:0.0:1788619986.037635:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589294720:0@lo:13:osp-pre-1-0.0 00010000:00010000:3.0:1788619986.723056:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788619987.045574:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c501a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589294848:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1788619987.045675:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619987.046498:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589294848 00000100:00100000:1.0:1788619987.046513:0:6699:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589294848 00000100:00100000:1.0:1788619987.046610:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 115 00000100:00100000:1.0:1788619987.046621:0:6699:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e6479f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589294848:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1788619987.046671:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00080000:2.0:1788619987.046812:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1788619987.047494:0:6699:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e6479f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589294848:12345-0@lo:13:osp-pre-0-0.0 Request processed in 871us (1815us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619987.047507:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 115 00000100:00100000:2.0:1788619987.047658:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c501a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589294848:0@lo:13:osp-pre-0-0.0 00010000:00010000:2.0:1788619987.751239:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788619988.770702:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788619989.796661:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788619990.630928:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fefd5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589295360:0@lo:400:kworker.0 00000100:00100000:3.0:1788619990.630958:0:3319:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:3319:1875503589294976:0@lo:400:kworker.0 00000100:00100000:3.0:1788619990.632441:0:3319:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619990.632935:0:3319:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78fefd4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3319:1875503589295488:0@lo:400:kworker.0 00000100:00100000:3.0:1788619990.632978:0:3319:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619990.633021:0:3317:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3317:1875503589295232:0@lo:400:kworker.0 00000100:00100000:3.0:1788619990.633046:0:3317:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619990.633065:0:3317:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1788619990.633072:0:3317:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3317:1875503589295104:0@lo:400:kworker.0 00000100:00100000:3.0:1788619990.633094:0:3317:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619990.633111:0:3317:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3317:1875503589295616:0@lo:400:kworker.0 00000100:00100000:3.0:1788619990.633132:0:3317:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619990.633160:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589294976 00000100:00100000:3.0:1788619990.633167:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589294976 00000100:00100000:3.0:1788619990.633210:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 318 00000100:00100000:3.0:1788619990.633219:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c99c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+19:3319:x1875503589294976:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788619990.633273:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c99c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+19:3319:x1875503589294976:12345-0@lo:400:kworker.0 Request processed in 54us (836us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619990.633280:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 318 00000100:00100000:3.0:1788619990.633311:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589295488 00000100:00100000:3.0:1788619990.633315:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589295488 00000100:00100000:3.0:1788619990.634185:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589295232 00000100:00100000:3.0:1788619990.634193:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589295232 00000100:00100000:3.0:1788619990.634214:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589295104 00000100:00100000:3.0:1788619990.634217:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589295104 00000100:00100000:3.0:1788619990.634231:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 245 00000100:00100000:3.0:1788619990.634237:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3319:x1875503589295488:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788619990.634282:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3319:x1875503589295488:12345-0@lo:400:kworker.0 Request processed in 45us (1303us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619990.634289:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 245 00000100:00100000:3.0:1788619990.634301:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 246 00000100:00100000:3.0:1788619990.634307:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3317:x1875503589295232:12345-0@lo:400:kworker.0 00000100:00080000:1.0:1788619990.634424:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788619990.634860:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3317:x1875503589295232:12345-0@lo:400:kworker.0 Request processed in 551us (1812us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619990.634868:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 246 00000100:00100000:3.0:1788619990.634881:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 247 00000100:00100000:3.0:1788619990.634888:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3317:x1875503589295104:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788619990.634915:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3317:x1875503589295104:12345-0@lo:400:kworker.0 Request processed in 29us (1821us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619990.634920:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 247 00000100:00100000:3.0:1788619990.634943:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589295616 00000100:00100000:3.0:1788619990.634948:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589295616 00000100:00100000:3.0:1788619990.634975:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 164 00000100:00100000:3.0:1788619990.634981:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3317:x1875503589295616:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788619990.635012:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3317:x1875503589295616:12345-0@lo:400:kworker.0 Request processed in 31us (1878us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619990.635017:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 164 00000100:00080000:3.0:1788619990.635044:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1788619990.635072:0:3317:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3317:1875503589295104:0@lo:400:kworker.0 00000100:00100000:3.0:1788619990.635084:0:3317:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3317:1875503589295616:0@lo:400:kworker.0 00000100:00100000:3.0:1788619990.635094:0:3317:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3317:1875503589295232:0@lo:400:kworker.0 00000100:00100000:3.0:1788619990.635128:0:3319:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:3319:1875503589294976:0@lo:400:kworker.0 00000100:00100000:3.0:1788619990.635138:0:3319:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fefd4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3319:1875503589295488:0@lo:400:kworker.0 00000100:00100000:0.0:1788619990.637297:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619990.637431:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589295360 00000100:00100000:3.0:1788619990.637441:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589295360 00000100:00100000:3.0:1788619990.638546:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 165 00000100:00100000:3.0:1788619990.638560:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589295360:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788619990.638645:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589295360:12345-0@lo:400:kworker.0 Request processed in 88us (1354us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619990.638652:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 165 00000100:00100000:0.0:1788619990.639416:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78fefd5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589295360:0@lo:400:kworker.0 00000100:00080000:1.0:1788619990.640882:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:3.0:1788619990.819559:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788619990.853488:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619990.853543:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619990.853573:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571775872 00000100:00100000:3.0:1788619990.853583:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571775872 00000100:00100000:3.0:1788619990.853642:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 248 00000100:00100000:3.0:1788619990.853652:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2352:x1875503571775872:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:3.0:1788619990.853724:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2352:x1875503571775872:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 73us (180us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619990.853731:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 248 00000100:00080000:3.0:1788619990.853752:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1788619990.856445:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571775744 00000100:00100000:0.0:1788619990.856460:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571775744 00000100:00100000:0.0:1788619990.856543:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 319 00000100:00100000:0.0:1788619990.856554:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c9b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:597450a2-e90c-408c-b401-a0ce48eed204+10:2352:x1875503571775744:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:0.0:1788619990.856631:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c9b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:597450a2-e90c-408c-b401-a0ce48eed204+10:2352:x1875503571775744:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 79us (3152us total) trans 0 rc 0/0 00000100:00100000:0.0:1788619990.856638:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 319 00000100:00100000:2.0:1788619991.140832:0:3316:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1788619991.140856:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e6479180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589295744:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1788619991.140923:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619991.141041:0:6700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589295744 00000100:00100000:3.0:1788619991.141049:0:6700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589295744 00000100:00100000:3.0:1788619991.141101:0:6700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 116 00000100:00100000:3.0:1788619991.141110:0:6700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e647b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503589295744:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1788619991.141144:0:6700:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3856662528 avail 3854565376 00000100:00100000:3.0:1788619991.141191:0:6700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e647b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503589295744:12345-0@lo:13:osp-pre-1-0.0 Request processed in 82us (268us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619991.141198:0:6700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 116 00000100:00100000:3.0:1788619991.141237:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e6479180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589295744:0@lo:13:osp-pre-1-0.0 00010000:00010000:2.0:1788619991.843454:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788619992.162541:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78e6479180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589295872:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1788619992.162614:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788619992.162747:0:6700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589295872 00000100:00100000:2.0:1788619992.162755:0:6700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589295872 00000100:00100000:2.0:1788619992.162804:0:6700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 117 00000100:00100000:2.0:1788619992.162813:0:6700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589295872:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1788619992.162846:0:6700:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1788619992.162884:0:6700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589295872:12345-0@lo:13:osp-pre-0-0.0 Request processed in 73us (270us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619992.162891:0:6700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 117 00000100:00100000:2.0:1788619992.163115:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78e6479180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589295872:0@lo:13:osp-pre-0-0.0 00000100:00080000:3.0:1788619992.165103:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:2.0:1788619992.866510:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788619993.895835:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788619994.914651:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788619995.747023:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78dfa41880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589296384:0@lo:400:kworker.0 00000100:00100000:1.0:1788619995.747113:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619995.747185:0:3318:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:1.0:1788619995.747192:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78dfa41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:3318:1875503589296000:0@lo:400:kworker.0 00000100:00100000:1.0:1788619995.747221:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619995.747258:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78dfa43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3318:1875503589296512:0@lo:400:kworker.0 00000100:00100000:1.0:1788619995.747286:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788619995.747429:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589296384 00000100:00100000:2.0:1788619995.747438:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589296384 00000100:00100000:2.0:1788619995.747504:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 166 00000100:00100000:2.0:1788619995.747513:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c501a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589296384:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788619995.747566:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c501a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589296384:12345-0@lo:400:kworker.0 Request processed in 54us (453us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619995.747572:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 166 00000100:00080000:1.0:1788619995.747909:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1788619995.747946:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78dfa41880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589296384:0@lo:400:kworker.0 00000100:00100000:3.0:1788619995.748201:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78dfa40a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3316:1875503589296128:0@lo:400:kworker.0 00000100:00100000:3.0:1788619995.748245:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619995.748280:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78dfa41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589296640:0@lo:400:kworker.0 00000100:00100000:3.0:1788619995.748304:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619995.748356:0:3316:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1788619995.748363:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78dfa41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3316:1875503589296256:0@lo:400:kworker.0 00000100:00100000:3.0:1788619995.748386:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619995.748428:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589296000 00000100:00100000:3.0:1788619995.748433:0:5836:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589296000 00000100:00100000:3.0:1788619995.748464:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 320 00000100:00100000:3.0:1788619995.748471:0:5836:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5d2bc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+19:3318:x1875503589296000:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788619995.748516:0:5836:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5d2bc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+19:3318:x1875503589296000:12345-0@lo:400:kworker.0 Request processed in 46us (1294us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619995.748523:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 320 00000100:00100000:3.0:1788619995.748552:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589296512 00000100:00100000:3.0:1788619995.748556:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589296512 00000100:00100000:3.0:1788619995.748583:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589296128 00000100:00100000:3.0:1788619995.748586:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589296128 00000100:00100000:3.0:1788619995.748599:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589296256 00000100:00100000:3.0:1788619995.748602:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589296256 00000100:00100000:3.0:1788619995.748618:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 249 00000100:00100000:3.0:1788619995.748624:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c4be2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3318:x1875503589296512:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788619995.748654:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c4be2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3318:x1875503589296512:12345-0@lo:400:kworker.0 Request processed in 30us (1368us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619995.748660:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 249 00000100:00100000:3.0:1788619995.748670:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 250 00000100:00100000:3.0:1788619995.748835:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5ff7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3316:x1875503589296128:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788619995.748874:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5ff7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3316:x1875503589296128:12345-0@lo:400:kworker.0 Request processed in 198us (629us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619995.748881:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 250 00000100:00100000:3.0:1788619995.748895:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 251 00000100:00100000:3.0:1788619995.748901:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3316:x1875503589296256:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788619995.748928:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3316:x1875503589296256:12345-0@lo:400:kworker.0 Request processed in 28us (541us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619995.748933:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 251 00000100:00100000:3.0:1788619995.748955:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589296640 00000100:00100000:3.0:1788619995.748959:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589296640 00000100:00100000:3.0:1788619995.748985:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 167 00000100:00100000:3.0:1788619995.748991:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fefd5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503589296640:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788619995.749021:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fefd5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503589296640:12345-0@lo:400:kworker.0 Request processed in 30us (715us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619995.749026:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 167 00000100:00100000:3.0:1788619995.749067:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78dfa41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3316:1875503589296256:0@lo:400:kworker.0 00000100:00100000:3.0:1788619995.749356:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78dfa40a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3316:1875503589296128:0@lo:400:kworker.0 00000100:00100000:3.0:1788619995.749372:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78dfa41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589296640:0@lo:400:kworker.0 00000100:00080000:3.0:1788619995.749403:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1788619995.749417:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788619995.749439:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78dfa41180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:3318:1875503589296000:0@lo:400:kworker.0 00000100:00100000:3.0:1788619995.749450:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78dfa43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3318:1875503589296512:0@lo:400:kworker.0 00000001:02000400:0.0:1788619995.937111:0:15418:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: Using TIMEOUT=20 00010000:00010000:2.0:1788619995.938617:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788619995.982248:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788619995.982284:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788619995.983837:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571776000 00000100:00100000:2.0:1788619995.983853:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571776000 00000100:00100000:2.0:1788619995.983920:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 321 00000100:00100000:2.0:1788619995.983933:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5d28450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:597450a2-e90c-408c-b401-a0ce48eed204+10:2355:x1875503571776000:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:2.0:1788619995.984012:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5d28450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:597450a2-e90c-408c-b401-a0ce48eed204+10:2355:x1875503571776000:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 83us (1775us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619995.984020:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 321 00000100:00100000:3.0:1788619995.991051:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571776128 00000100:00100000:3.0:1788619995.991066:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571776128 00000100:00100000:3.0:1788619995.991301:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 252 00000100:00100000:3.0:1788619995.991312:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69ff480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571776128:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:3.0:1788619995.991460:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69ff480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571776128:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 150us (9175us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619995.991468:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 252 00000100:00080000:1.0:1788619995.995926:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1788619996.263528:0:3317:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78dfa43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3317:1875503589296768:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1788619996.263623:0:3317:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788619996.263692:0:6700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589296768 00000100:00100000:2.0:1788619996.263701:0:6700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589296768 00000100:00100000:2.0:1788619996.263744:0:6700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 118 00000100:00100000:2.0:1788619996.263752:0:6700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa40700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3317:x1875503589296768:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1788619996.263786:0:6700:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3856662528 avail 3854565376 00000100:00100000:2.0:1788619996.263831:0:6700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa40700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3317:x1875503589296768:12345-0@lo:13:osp-pre-1-0.0 Request processed in 79us (208us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619996.263839:0:6700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 118 00000100:00100000:2.0:1788619996.263890:0:3317:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78dfa43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3317:1875503589296768:0@lo:13:osp-pre-1-0.0 00010000:00010000:2.0:1788619996.965804:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788619997.283544:0:3319:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1788619997.283567:0:3319:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78dfa43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3319:1875503589296896:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1788619997.283638:0:3319:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788619997.283740:0:6700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589296896 00000100:00100000:2.0:1788619997.283748:0:6700:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589296896 00000100:00100000:2.0:1788619997.283810:0:6700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 119 00000100:00100000:2.0:1788619997.283821:0:6700:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e01f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1875503589296896:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1788619997.283969:0:6700:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00080000:1.0:1788619997.284037:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1788619997.284043:0:6700:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e01f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1875503589296896:12345-0@lo:13:osp-pre-0-0.0 Request processed in 221us (401us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619997.284052:0:6700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 119 00000100:00100000:2.0:1788619997.284152:0:3319:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78dfa43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3319:1875503589296896:0@lo:13:osp-pre-0-0.0 00010000:00010000:2.0:1788619997.987440:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788619999.016153:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000080:3.0:1788619999.152938:0:15565:0:(class_obd.c:336:class_handle_ioctl()) obdclass: cmd=400866bb len=0 uarg=00007ffe13ae0a60 20000000:00000080:3.0:1788619999.152957:0:15565:0:(mgs_handler.c:1167:mgs_iocontrol()) MGS: cmd=400866bb len=576 karg=ffff8e77c777a400 uarg=0000000000000000 20000000:01000000:3.0:1788619999.153005:0:15565:0:(mgs_llog.c:6192:mgs_set_param2()) set_param2 fs='' device='general' param='lod.*.mdt_hash=crush' 20000000:01000000:3.0:1788619999.153011:0:15565:0:(mgs_llog.c:4647:mgs_write_log_param2()) next param 'lod.*.mdt_hash=crush' 20000000:01000000:3.0:1788619999.153017:0:15565:0:(mgs_llog.c:1379:mgs_modify_param()) modify params/general/lod.*.mdt_hash=crush del=0 00000040:00080000:0.0:1788619999.154863:0:15566:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x2:0x0] index 5 count 7 20000000:00000400:3.0:1788619999.155027:0:15565:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 10000000:01000000:3.0:1788619999.160804:0:15565:0:(mgc_request.c:68:mgc_name2resid()) log params to resid 0x736d61726170/0x3 (params) 00010000:00010000:3.0:1788619999.160838:0:15565:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: MGS lock: ffff8e78dfa06a00/0x1cd7fe5368737091 lrc: 3/0,1 mode: --/EX res: [0x736d61726170:0x3:0x0].0x0 rrc: 6 type: PLN flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15565 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619999.160858:0:15565:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff8e77c5ed0000/0x1cd7fe5368731528 lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 6 type: PLN flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a19509 expref: 9 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619999.160868:0:15565:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff8e77c5ed1800/0x1cd7fe5368731536 lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 6 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x1cd7fe536873152f expref: 18 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619999.160875:0:15565:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff8e78c8078000/0x1cd7fe5368731544 lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 6 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x1cd7fe536873153d expref: 18 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619999.160882:0:15565:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff8e78c8078800/0x1cd7fe5368731552 lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 6 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x1cd7fe536873154b expref: 18 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619999.160889:0:15565:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: MGS lock: ffff8e78dfa06a00/0x1cd7fe5368737091 lrc: 7/0,1 mode: --/EX res: [0x736d61726170:0x3:0x0].0x0 rrc: 6 type: PLN flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 15565 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619999.160931:0:15565:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff8e78c8078800/0x1cd7fe5368731552 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 6 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0x1cd7fe536873154b expref: 18 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619999.160946:0:15565:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff8e78c8078800/0x1cd7fe5368731552 lrc: 4/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 6 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0x1cd7fe536873154b expref: 18 pid: 5838 timeout: 641 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619999.160960:0:15565:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fd500 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:15565:1875503589297024:0@lo:104: 00000100:00100000:3.0:1788619999.161017:0:15565:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1788619999.161135:0:15565:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff8e78c8078000/0x1cd7fe5368731544 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 6 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0x1cd7fe536873153d expref: 18 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619999.161143:0:15565:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff8e78c8078000/0x1cd7fe5368731544 lrc: 4/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 6 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0x1cd7fe536873153d expref: 18 pid: 5837 timeout: 641 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619999.161151:0:15565:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fd180 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:15565:1875503589297152:0@lo:104: 00000100:00100000:3.0:1788619999.161171:0:15565:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1788619999.161767:0:15565:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff8e77c5ed1800/0x1cd7fe5368731536 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 6 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0x1cd7fe536873152f expref: 18 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619999.161778:0:15565:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff8e77c5ed1800/0x1cd7fe5368731536 lrc: 4/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 6 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0x1cd7fe536873152f expref: 18 pid: 5837 timeout: 641 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619999.161788:0:15565:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df385500 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:15565:1875503589297280:0@lo:104: 00000100:00100000:3.0:1788619999.161816:0:15565:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788619999.161930:0:5827:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589297024 00000100:00100000:1.0:1788619999.161939:0:5827:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589297024 00000100:00100000:1.0:1788619999.161977:0:5827:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589297152 00000100:00100000:1.0:1788619999.161980:0:5827:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589297152 00000100:00100000:1.0:1788619999.161992:0:5827:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 25 00000100:00100000:1.0:1788619999.162001:0:5827:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010fc380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+4:15565:x1875503589297024:12345-0@lo:104: 00010000:00010000:1.0:1788619999.162014:0:5827:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.201.105@tcp lock: ffff8e78c0031800/0x1cd7fe536873154b lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x1400000000000 nid: local remote: 0x1cd7fe5368731552 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788619999.162096:0:5827:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1788619999.162104:0:5827:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010fc380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+4:15565:x1875503589297024:12345-0@lo:104: Request processed in 104us (1087us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619999.162110:0:5827:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 25 00000100:00100000:1.0:1788619999.162127:0:5827:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 26 00000100:00100000:1.0:1788619999.162133:0:5827:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df386680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+3:15565:x1875503589297152:12345-0@lo:104: 00010000:00010000:1.0:1788619999.162141:0:5827:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.201.105@tcp lock: ffff8e78ccca6c00/0x1cd7fe536873153d lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x1400000000000 nid: local remote: 0x1cd7fe5368731544 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619999.162220:0:15565:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff8e77c5ed0000/0x1cd7fe5368731528 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 6 type: PLN flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a19509 expref: 9 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619999.162232:0:15565:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: MGS lock: ffff8e77c5ed0000/0x1cd7fe5368731528 lrc: 4/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 6 type: PLN flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a19509 expref: 9 pid: 5837 timeout: 591 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619999.162242:0:15565:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df384700 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:15565:1875503589297408:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788619999.162291:0:15565:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c56b3000 going to sleep for 11 seconds 00000100:00100000:3.0:1788619999.162312:0:15565:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fd500 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:15565:1875503589297024:0@lo:104: 00000100:00100000:3.0:1788619999.162320:0:15565:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fd180 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:15565:1875503589297152:0@lo:104: 00010000:00010000:0.0:1788619999.162525:0:5832:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1788619999.162532:0:5832:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.201.105@tcp lock: ffff8e78c0031800/0x1cd7fe536873154b lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x1400000000000 nid: local remote: 0x1cd7fe5368731552 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619999.162540:0:5832:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78c0031800 already unused, calling callback (ffffffffc1ddfc20) 10000000:00010000:0.0:1788619999.162543:0:5832:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.201.105@tcp lock: ffff8e78c0031800/0x1cd7fe536873154b lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x1400400000000 nid: local remote: 0x1cd7fe5368731552 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619999.162551:0:5832:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.201.105@tcp lock: ffff8e78c0031800/0x1cd7fe536873154b lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x1408400000000 nid: local remote: 0x1cd7fe5368731552 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1788619999.162558:0:5832:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.201.105@tcp lock: ffff8e78c0031800/0x1cd7fe536873154b lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x1409400000000 nid: local remote: 0x1cd7fe5368731552 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1788619999.162564:0:5832:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:0.0:1788619999.162601:0:5832:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.201.105@tcp lock: ffff8e78c0031800/0x1cd7fe536873154b lrc: 2/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x5c09400000020 nid: local remote: 0x1cd7fe5368731552 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619999.162607:0:5832:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1788619999.162696:0:5832:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.201.105@tcp lock: ffff8e78c0031800/0x1cd7fe536873154b lrc: 1/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x5c09400000020 nid: local remote: 0x1cd7fe5368731552 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619999.162703:0:5832:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.201.105@tcp lock: ffff8e78c0031800/0x1cd7fe536873154b lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x5c09400000020 nid: local remote: 0x1cd7fe5368731552 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619999.162715:0:5832:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1788619999.162718:0:5832:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.201.105@tcp lock: ffff8e78ccca6c00/0x1cd7fe536873153d lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x1400000000000 nid: local remote: 0x1cd7fe5368731544 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619999.162722:0:5832:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78ccca6c00 already unused, calling callback (ffffffffc1ddfc20) 10000000:00010000:0.0:1788619999.162725:0:5832:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.201.105@tcp lock: ffff8e78ccca6c00/0x1cd7fe536873153d lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x1400400000000 nid: local remote: 0x1cd7fe5368731544 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619999.162730:0:5832:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.201.105@tcp lock: ffff8e78ccca6c00/0x1cd7fe536873153d lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x1408400000000 nid: local remote: 0x1cd7fe5368731544 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1788619999.162736:0:5832:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.201.105@tcp lock: ffff8e78ccca6c00/0x1cd7fe536873153d lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x1409400000000 nid: local remote: 0x1cd7fe5368731544 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1788619999.162740:0:5832:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:0.0:1788619999.162755:0:5832:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.201.105@tcp lock: ffff8e78ccca6c00/0x1cd7fe536873153d lrc: 2/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x5c09400000020 nid: local remote: 0x1cd7fe5368731544 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619999.162761:0:5832:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1788619999.162776:0:5832:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.201.105@tcp lock: ffff8e78ccca6c00/0x1cd7fe536873153d lrc: 1/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x5c09400000020 nid: local remote: 0x1cd7fe5368731544 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619999.162786:0:5832:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.201.105@tcp lock: ffff8e78ccca6c00/0x1cd7fe536873153d lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x5c09400000020 nid: local remote: 0x1cd7fe5368731544 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788619999.162795:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788619999.162796:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788619999.162798:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788619999.163185:0:5828:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:2.0:1788619999.163198:0:5828:0:(service.c:3462:ptlrpc_start_thread()) ldlm_cbd[0] started 2 min 2 max 56 00000100:00100000:2.0:1788619999.163204:0:5828:0:(service.c:3521:ptlrpc_start_thread()) starting thread 'ldlm_cb00_002' 00000100:00100000:2.0:1788619999.164146:0:5828:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589297280 00000100:00100000:2.0:1788619999.164155:0:5828:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589297280 00000100:00100000:2.0:1788619999.164244:0:15567:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 27 00000100:00100000:2.0:1788619999.164254:0:15567:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78e6400450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+4:15565:x1875503589297280:12345-0@lo:104: 00010000:00010000:2.0:1788619999.164265:0:15567:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.201.105@tcp lock: ffff8e78c8079a00/0x1cd7fe536873152f lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1400000000000 nid: local remote: 0x1cd7fe5368731536 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619999.164366:0:15567:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1788619999.164372:0:15567:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78e6400450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+4:15565:x1875503589297280:12345-0@lo:104: Request processed in 121us (2556us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619999.164379:0:15567:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 27 00010000:00010000:2.0:1788619999.164398:0:5833:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1788619999.164403:0:5833:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.201.105@tcp lock: ffff8e78c8079a00/0x1cd7fe536873152f lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1400000000000 nid: local remote: 0x1cd7fe5368731536 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619999.164414:0:5833:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78c8079a00 already unused, calling callback (ffffffffc1ddfc20) 10000000:00010000:2.0:1788619999.164418:0:5833:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.201.105@tcp lock: ffff8e78c8079a00/0x1cd7fe536873152f lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1400400000000 nid: local remote: 0x1cd7fe5368731536 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619999.164426:0:5833:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.201.105@tcp lock: ffff8e78c8079a00/0x1cd7fe536873152f lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1408400000000 nid: local remote: 0x1cd7fe5368731536 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1788619999.164433:0:5833:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.201.105@tcp lock: ffff8e78c8079a00/0x1cd7fe536873152f lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1409400000000 nid: local remote: 0x1cd7fe5368731536 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1788619999.164438:0:5833:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:2.0:1788619999.164462:0:5833:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.201.105@tcp lock: ffff8e78c8079a00/0x1cd7fe536873152f lrc: 2/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0x1cd7fe5368731536 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619999.164469:0:5833:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1788619999.164479:0:5833:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.201.105@tcp lock: ffff8e78c8079a00/0x1cd7fe536873152f lrc: 1/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0x1cd7fe5368731536 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619999.164486:0:5833:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.201.105@tcp lock: ffff8e78c8079a00/0x1cd7fe536873152f lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0x1cd7fe5368731536 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619999.164499:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788619999.164500:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788619999.164502:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788619999.166097:0:3319:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:3319:1875503589297536:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1788619999.166157:0:3319:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788619999.166523:0:15565:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df385500 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:15565:1875503589297280:0@lo:104: 00000100:00100000:0.0:1788619999.167103:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c69fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:3316:1875503589297664:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1788619999.167139:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788619999.167319:0:3316:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1788619999.167344:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78dfa43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:3316:1875503589297792:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1788619999.167377:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1788619999.167440:0:5827:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/0 locks to priority blp list, 0 blwis in pool 00000100:00100000:1.0:1788619999.167450:0:5827:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df386680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+3:15565:x1875503589297152:12345-0@lo:104: Request processed in 5315us (6275us total) trans 0 rc 0/0 00000100:00100000:1.0:1788619999.167460:0:5827:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 26 00000100:00100000:2.0:1788619999.169382:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589297536 00000100:00100000:2.0:1788619999.169397:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589297536 00010000:00010000:2.0:1788619999.169447:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff8e78c8078800/0x1cd7fe5368731552 lrc: 4/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 6 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x1cd7fe536873154b expref: 19 pid: 5838 timeout: 641 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619999.169494:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589297664 00000100:00100000:2.0:1788619999.169498:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589297664 00010000:00010000:2.0:1788619999.169514:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff8e78c8078000/0x1cd7fe5368731544 lrc: 4/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 6 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x1cd7fe536873153d expref: 20 pid: 5837 timeout: 641 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788619999.169541:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 560 00000100:00100000:2.0:1788619999.169550:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+20:3319:x1875503589297536:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1788619999.169564:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619999.169570:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff8e78c8078800/0x1cd7fe5368731552 lrc: 4/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 7 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x1cd7fe536873154b expref: 20 pid: 5838 timeout: 641 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619999.169584:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff8e78c8078800/0x1cd7fe5368731552 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 7 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0x1cd7fe536873154b expref: 20 pid: 5838 timeout: 641 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619999.169602:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff8e78c8078800/0x1cd7fe5368731552 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 7 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0x1cd7fe536873154b expref: 20 pid: 5838 timeout: 641 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619999.169628:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619999.169691:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+19:3319:x1875503589297536:12345-0@lo:103:ldlm_bl.0 Request processed in 142us (3533us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619999.169697:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 560 00000100:00100000:2.0:1788619999.169714:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 561 00000100:00100000:2.0:1788619999.169720:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c80e8050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+18:3316:x1875503589297664:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1788619999.169728:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788619999.169732:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff8e78c8078000/0x1cd7fe5368731544 lrc: 4/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 6 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x1cd7fe536873153d expref: 18 pid: 5837 timeout: 641 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619999.169741:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff8e78c8078000/0x1cd7fe5368731544 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 6 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0x1cd7fe536873153d expref: 18 pid: 5837 timeout: 641 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619999.169751:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff8e78c8078000/0x1cd7fe5368731544 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 6 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0x1cd7fe536873153d expref: 18 pid: 5837 timeout: 641 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619999.169760:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788619999.169784:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c80e8050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+17:3316:x1875503589297664:12345-0@lo:103:ldlm_bl.0 Request processed in 64us (2644us total) trans 0 rc 0/0 00000100:00100000:2.0:1788619999.169789:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 561 00000100:00100000:2.0:1788619999.169958:0:15565:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df384700 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:15565:1875503589297408:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788619999.169990:0:15565:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: MGS lock: ffff8e78dfa06a00/0x1cd7fe5368737091 lrc: 3/0,1 mode: --/EX res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 15565 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788619999.170531:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788619999.170908:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69fdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:3316:1875503589297664:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1788619999.170939:0:3319:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c69ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:3319:1875503589297536:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1788619999.170998:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788619999.171114:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589297792 00000100:00100000:3.0:1788619999.171121:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589297792 00010000:00010000:3.0:1788619999.171148:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff8e77c5ed1800/0x1cd7fe5368731536 lrc: 4/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x1cd7fe536873152f expref: 17 pid: 5837 timeout: 641 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619999.171184:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571776256 00000100:00100000:3.0:1788619999.171188:0:5831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571776256 00010000:00010000:3.0:1788619999.171202:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff8e77c5ed0000/0x1cd7fe5368731528 lrc: 4/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a19509 expref: 10 pid: 5837 timeout: 591 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788619999.171214:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 562 00000100:00100000:3.0:1788619999.171221:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+17:3316:x1875503589297792:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1788619999.171232:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619999.171237:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff8e77c5ed1800/0x1cd7fe5368731536 lrc: 4/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 5 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x1cd7fe536873152f expref: 17 pid: 5837 timeout: 641 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619999.171247:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff8e77c5ed1800/0x1cd7fe5368731536 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 5 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0x1cd7fe536873152f expref: 17 pid: 5837 timeout: 641 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619999.171260:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff8e77c5ed1800/0x1cd7fe5368731536 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 5 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0x1cd7fe536873152f expref: 17 pid: 5837 timeout: 641 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619999.171271:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619999.171323:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+16:3316:x1875503589297792:12345-0@lo:103:ldlm_bl.0 Request processed in 101us (3944us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619999.171385:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 562 00000100:00100000:3.0:1788619999.171401:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 563 00000100:00100000:3.0:1788619999.171407:0:5831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900e03b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:597450a2-e90c-408c-b401-a0ce48eed204+10:2355:x1875503571776256:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788619999.171416:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788619999.171421:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff8e77c5ed0000/0x1cd7fe5368731528 lrc: 4/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a19509 expref: 10 pid: 5837 timeout: 591 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619999.171435:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff8e77c5ed0000/0x1cd7fe5368731528 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a19509 expref: 10 pid: 5837 timeout: 591 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619999.171445:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff8e77c5ed0000/0x1cd7fe5368731528 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a19509 expref: 10 pid: 5837 timeout: 591 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619999.171454:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: MGS lock: ffff8e78dfa06a00/0x1cd7fe5368737091 lrc: 3/0,1 mode: EX/EX res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x40290000000000 nid: local remote: 0x0 expref: -99 pid: 15565 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619999.171461:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff8e78dfa06a00/0x1cd7fe5368737091 lrc: 4/0,1 mode: EX/EX res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x40290000000000 nid: local remote: 0x0 expref: -99 pid: 15565 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619999.171474:0:5831:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: MGS lock: ffff8e78dfa06a00/0x1cd7fe5368737091 lrc: 5/0,1 mode: EX/EX res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 15565 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619999.171481:0:5831:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: MGS lock: ffff8e78dfa06a00/0x1cd7fe5368737091 lrc: 5/0,1 mode: EX/EX res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 15565 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619999.171488:0:5831:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGS lock: ffff8e78dfa06a00/0x1cd7fe5368737091 lrc: 4/0,0 mode: EX/EX res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 15565 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619999.171494:0:5831:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGS lock: ffff8e78dfa06a00/0x1cd7fe5368737091 lrc: 5/0,0 mode: EX/EX res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 15565 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619999.171500:0:5831:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78dfa06a00 already unused, calling callback (ffffffffc17c24f0) 00010000:00010000:3.0:1788619999.171504:0:5831:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: MGS lock: ffff8e78dfa06a00/0x1cd7fe5368737091 lrc: 5/0,0 mode: EX/EX res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 15565 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619999.171512:0:5831:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: MGS lock: ffff8e78dfa06a00/0x1cd7fe5368737091 lrc: 6/0,0 mode: EX/EX res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 15565 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619999.171532:0:5831:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGS lock: ffff8e78dfa06a00/0x1cd7fe5368737091 lrc: 4/0,0 mode: --/EX res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 15565 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619999.171542:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788619999.171578:0:5831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900e03b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:597450a2-e90c-408c-b401-a0ce48eed204+9:2355:x1875503571776256:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 171us (581us total) trans 0 rc 0/0 00000100:00100000:3.0:1788619999.171584:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 563 00010000:00010000:3.0:1788619999.172721:0:15565:0:(ldlm_request.c:190:ldlm_completion_tail()) ### client-side enqueue: destroyed ns: MGS lock: ffff8e78dfa06a00/0x1cd7fe5368737091 lrc: 1/0,0 mode: --/EX res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 15565 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619999.172735:0:15565:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: MGS lock: ffff8e78dfa06a00/0x1cd7fe5368737091 lrc: 1/0,0 mode: --/EX res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 15565 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788619999.172742:0:15565:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff8e78dfa06a00/0x1cd7fe5368737091 lrc: 0/0,0 mode: --/EX res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 15565 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788619999.173472:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78dfa43480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:3316:1875503589297792:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1788620000.034948:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788620000.517755:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788620000.517822:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571776384 00000100:00100000:2.0:1788620000.517832:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571776384 00000100:00100000:2.0:1788620000.517898:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 322 00000100:00100000:2.0:1788620000.517909:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5d28050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:597450a2-e90c-408c-b401-a0ce48eed204+9:5593:x1875503571776384:12345-192.168.201.5@tcp:101:ll_cfg_requeue.0 00010000:00010000:2.0:1788620000.517926:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788620000.517942:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff8e77c6a3ea00/0x1cd7fe5368737098 lrc: 2/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1e01f expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620000.517965:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff8e77c6a3ea00/0x1cd7fe5368737098 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e01f expref: 10 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620000.517973:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff8e77c6a3ea00/0x1cd7fe5368737098 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e01f expref: 10 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620000.517980:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6a3ea00, rc 0) 00000100:00100000:2.0:1788620000.518123:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5d28050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:597450a2-e90c-408c-b401-a0ce48eed204+10:5593:x1875503571776384:12345-192.168.201.5@tcp:101:ll_cfg_requeue.0 Request processed in 216us (373us total) trans 0 rc 0/0 00000100:00100000:2.0:1788620000.518127:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 322 00000100:00100000:3.0:1788620000.526125:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788620000.526210:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571776512 00000100:00100000:2.0:1788620000.526219:0:5836:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571776512 00000100:00100000:2.0:1788620000.526276:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 323 00000100:00100000:2.0:1788620000.526286:0:5836:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5d2a850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:597450a2-e90c-408c-b401-a0ce48eed204+10:5593:x1875503571776512:12345-192.168.201.5@tcp:501:ll_cfg_requeue.0 00000100:00100000:2.0:1788620000.526602:0:5836:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5d2a850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:597450a2-e90c-408c-b401-a0ce48eed204+10:5593:x1875503571776512:12345-192.168.201.5@tcp:501:ll_cfg_requeue.0 Request processed in 316us (486us total) trans 0 rc 0/0 00000100:00100000:2.0:1788620000.526610:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 323 00000100:00100000:1.0:1788620000.531986:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788620000.532072:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571776640 00000100:00100000:3.0:1788620000.532084:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571776640 00000100:00100000:3.0:1788620000.532141:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 324 00000100:00100000:3.0:1788620000.532151:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5d29450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:597450a2-e90c-408c-b401-a0ce48eed204+10:5593:x1875503571776640:12345-192.168.201.5@tcp:503:ll_cfg_requeue.0 00000100:00100000:3.0:1788620000.532390:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5d29450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:597450a2-e90c-408c-b401-a0ce48eed204+10:5593:x1875503571776640:12345-192.168.201.5@tcp:503:ll_cfg_requeue.0 Request processed in 240us (414us total) trans 0 rc 0/0 00000100:00100000:3.0:1788620000.532399:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 324 10000000:01000000:3.0:1788620000.802757:0:5841:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:3.0:1788620000.802765:0:5841:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff8e7900e12000 from 7 10000000:01000000:3.0:1788620000.802771:0:5841:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00010000:00010000:3.0:1788620000.802823:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.201.105@tcp lock: ffff8e78dfa06a00/0x1cd7fe536873709f lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620000.802835:0:5841:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.201.105@tcp lock: ffff8e78dfa06a00/0x1cd7fe536873709f lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620000.802848:0:5841:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.201.105@tcp lock: ffff8e78dfa06a00/0x1cd7fe536873709f lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788620000.802875:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df386300 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:5841:1875503589297920:0@lo:101:ll_cfg_requeue.0 00000100:00100000:3.0:1788620000.803024:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788620000.803139:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c56ebe40 going to sleep for 11 seconds 00000100:00100000:2.0:1788620000.804856:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589297920 00000100:00100000:2.0:1788620000.804870:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589297920 00000100:00100000:2.0:1788620000.812605:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 325 00000100:00100000:2.0:1788620000.812624:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5d2b050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+16:5841:x1875503589297920:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:2.0:1788620000.812646:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788620000.812982:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff8e77c6a3f800/0x1cd7fe53687370a6 lrc: 2/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x40000000000000 nid: local remote: 0x1cd7fe536873709f expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620000.813016:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff8e77c6a3f800/0x1cd7fe53687370a6 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0x1cd7fe536873709f expref: 17 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620000.813028:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff8e77c6a3f800/0x1cd7fe53687370a6 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x1cd7fe536873709f expref: 17 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620000.813038:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6a3f800, rc 0) 00000100:00100000:2.0:1788620000.813132:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5d2b050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+17:5841:x1875503589297920:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 513us (10107us total) trans 0 rc 0/0 00000100:00100000:2.0:1788620000.813140:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 325 00000100:00100000:3.0:1788620000.813437:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df386300 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:5841:1875503589297920:0@lo:101:ll_cfg_requeue.0 00010000:00010000:3.0:1788620000.813457:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.201.105@tcp lock: ffff8e78dfa06a00/0x1cd7fe536873709f lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x1cd7fe53687370a6 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620000.813468:0:5841:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.201.105@tcp lock: ffff8e78dfa06a00/0x1cd7fe536873709f lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x1cd7fe53687370a6 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788620000.813555:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df386300 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:5841:1875503589298048:0@lo:501:ll_cfg_requeue.0 00000100:00100000:3.0:1788620000.813594:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788620000.813634:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c56ebe40 going to sleep for 11 seconds 00000100:00100000:2.0:1788620000.813758:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589298048 00000100:00100000:2.0:1788620000.813764:0:5836:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589298048 00000100:00100000:2.0:1788620000.813801:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 326 00000100:00100000:2.0:1788620000.813810:0:5836:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5d2a050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+17:5841:x1875503589298048:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1788620000.815157:0:5836:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5d2a050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+17:5841:x1875503589298048:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 1343us (1558us total) trans 0 rc 0/0 00000100:00100000:2.0:1788620000.815168:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 326 00000100:00100000:3.0:1788620000.825888:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df386300 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:5841:1875503589298048:0@lo:501:ll_cfg_requeue.0 00000100:00100000:3.0:1788620000.825959:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df386300 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:5841:1875503589298176:0@lo:503:ll_cfg_requeue.0 00000100:00100000:3.0:1788620000.826034:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788620000.826085:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c56ebe40 going to sleep for 11 seconds 00000100:00100000:2.0:1788620000.826181:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589298176 00000100:00100000:2.0:1788620000.826188:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589298176 00000100:00100000:2.0:1788620000.826247:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 327 00000100:00100000:2.0:1788620000.826257:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5ce6450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+17:5841:x1875503589298176:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:2.0:1788620000.826688:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5ce6450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+17:5841:x1875503589298176:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 432us (652us total) trans 0 rc 0/0 00000100:00100000:2.0:1788620000.826696:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 327 00000100:00100000:0.0:1788620000.840019:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df386300 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:5841:1875503589298176:0@lo:503:ll_cfg_requeue.0 00000040:00080000:3.0:1788620000.853639:0:15623:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x2:0x0] index 64768 count 7 00000020:01000000:0.0:1788620000.853779:0:5841:0:(obd_config.c:2145:class_config_parse_llog()) Processed log params gen 7-6 (rc=0) 10000000:01000000:0.0:1788620000.853802:0:5841:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.201.105@tcp: configuration from log 'params' succeeded (0). 00010000:00010000:0.0:1788620000.853813:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.201.105@tcp lock: ffff8e78dfa06a00/0x1cd7fe536873709f lrc: 3/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x1cd7fe53687370a6 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620000.853836:0:5841:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.201.105@tcp lock: ffff8e78dfa06a00/0x1cd7fe536873709f lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x1cd7fe53687370a6 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1788620000.853848:0:5841:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:0.0:1788620000.853851:0:5841:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff8e78fedf8800 from 7 10000000:01000000:0.0:1788620000.853854:0:5841:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00010000:00010000:0.0:1788620000.853903:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.201.105@tcp lock: ffff8e77c777dc00/0x1cd7fe53687370ad lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620000.853911:0:5841:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.201.105@tcp lock: ffff8e77c777dc00/0x1cd7fe53687370ad lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620000.853922:0:5841:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.201.105@tcp lock: ffff8e77c777dc00/0x1cd7fe53687370ad lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788620000.853946:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a58e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:5841:1875503589298304:0@lo:101:ll_cfg_requeue.0 00000100:00100000:0.0:1788620000.854017:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788620000.854061:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5eafe40 going to sleep for 11 seconds 00000100:00100000:1.0:1788620000.855423:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589298304 00000100:00100000:1.0:1788620000.855433:0:5836:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589298304 00000100:00100000:1.0:1788620000.855509:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 328 00000100:00100000:1.0:1788620000.855519:0:5836:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5ce4850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+17:5841:x1875503589298304:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:1.0:1788620000.855533:0:5836:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788620000.855550:0:5836:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff8e77c683c600/0x1cd7fe53687370b4 lrc: 2/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x40000000000000 nid: local remote: 0x1cd7fe53687370ad expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620000.855574:0:5836:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff8e77c683c600/0x1cd7fe53687370b4 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0x1cd7fe53687370ad expref: 18 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620000.855585:0:5836:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff8e77c683c600/0x1cd7fe53687370b4 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x1cd7fe53687370ad expref: 18 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620000.855600:0:5836:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c600, rc 0) 00000100:00100000:1.0:1788620000.855663:0:5836:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5ce4850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+18:5841:x1875503589298304:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 146us (1646us total) trans 0 rc 0/0 00000100:00100000:1.0:1788620000.855670:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 328 00000100:00100000:1.0:1788620000.855722:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a58e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:5841:1875503589298304:0@lo:101:ll_cfg_requeue.0 00010000:00010000:1.0:1788620000.855739:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.201.105@tcp lock: ffff8e77c777dc00/0x1cd7fe53687370ad lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x1cd7fe53687370b4 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620000.855749:0:5841:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.201.105@tcp lock: ffff8e77c777dc00/0x1cd7fe53687370ad lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x1cd7fe53687370b4 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788620000.855812:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c75f6d80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:5841:1875503589298432:0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1788620000.855840:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788620000.855867:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fdd87180 going to sleep for 11 seconds 00000100:00100000:1.0:1788620000.855881:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589298432 00000100:00100000:1.0:1788620000.855903:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589298432 00000100:00100000:1.0:1788620000.855928:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 329 00000100:00100000:1.0:1788620000.855934:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5ce5850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+18:5841:x1875503589298432:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1788620000.856187:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5ce5850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+18:5841:x1875503589298432:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 252us (345us total) trans 0 rc 0/0 00000100:00100000:1.0:1788620000.856195:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 329 00000100:00100000:1.0:1788620000.856232:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c75f6d80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:5841:1875503589298432:0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1788620000.856259:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c75f6d80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:5841:1875503589298560:0@lo:503:ll_cfg_requeue.0 00000100:00100000:1.0:1788620000.856299:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788620000.856313:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fdd87180 going to sleep for 11 seconds 00000100:00100000:1.0:1788620000.856358:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589298560 00000100:00100000:1.0:1788620000.856362:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589298560 00000100:00100000:1.0:1788620000.856383:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 330 00000100:00100000:1.0:1788620000.856389:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5ce7c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+18:5841:x1875503589298560:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:1.0:1788620000.856524:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5ce7c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+18:5841:x1875503589298560:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 134us (224us total) trans 0 rc 0/0 00000100:00100000:1.0:1788620000.856530:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 330 00000100:00100000:1.0:1788620000.856567:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c75f6d80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:5841:1875503589298560:0@lo:503:ll_cfg_requeue.0 00000040:00080000:1.0:1788620000.857151:0:15624:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x2:0x0] index 64768 count 7 00000020:01000000:1.0:1788620000.857194:0:5841:0:(obd_config.c:2145:class_config_parse_llog()) Processed log params gen 7-6 (rc=0) 10000000:01000000:1.0:1788620000.857204:0:5841:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.201.105@tcp: configuration from log 'params' succeeded (0). 00010000:00010000:1.0:1788620000.857211:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.201.105@tcp lock: ffff8e77c777dc00/0x1cd7fe53687370ad lrc: 3/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x1cd7fe53687370b4 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620000.857217:0:5841:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.201.105@tcp lock: ffff8e77c777dc00/0x1cd7fe53687370ad lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x1cd7fe53687370b4 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1788620000.857225:0:5841:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:1.0:1788620000.857226:0:5841:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff8e78c5e49000 from 7 10000000:01000000:1.0:1788620000.857228:0:5841:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00010000:00010000:1.0:1788620000.857247:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.201.105@tcp lock: ffff8e77c683c400/0x1cd7fe53687370bb lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620000.857253:0:5841:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.201.105@tcp lock: ffff8e77c683c400/0x1cd7fe53687370bb lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620000.857259:0:5841:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.201.105@tcp lock: ffff8e77c683c400/0x1cd7fe53687370bb lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788620000.857273:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c75f6d80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:5841:1875503589298688:0@lo:101:ll_cfg_requeue.0 00000100:00100000:1.0:1788620000.857303:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788620000.857349:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fdd87c00 going to sleep for 11 seconds 00000100:00100000:1.0:1788620000.857363:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589298688 00000100:00100000:1.0:1788620000.857367:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589298688 00000100:00100000:1.0:1788620000.857391:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 331 00000100:00100000:1.0:1788620000.857398:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5ce4450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+18:5841:x1875503589298688:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:1.0:1788620000.857407:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788620000.857418:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff8e77c683dc00/0x1cd7fe53687370c2 lrc: 2/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 5 type: PLN flags: 0x40000000000000 nid: local remote: 0x1cd7fe53687370bb expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620000.857436:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff8e77c683dc00/0x1cd7fe53687370c2 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 5 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0x1cd7fe53687370bb expref: 19 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620000.857447:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff8e77c683dc00/0x1cd7fe53687370c2 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 5 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x1cd7fe53687370bb expref: 19 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620000.857455:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683dc00, rc 0) 00000100:00100000:1.0:1788620000.857485:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5ce4450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+19:5841:x1875503589298688:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 88us (181us total) trans 0 rc 0/0 00000100:00100000:1.0:1788620000.857490:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 331 00000100:00100000:1.0:1788620000.857514:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c75f6d80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:5841:1875503589298688:0@lo:101:ll_cfg_requeue.0 00010000:00010000:1.0:1788620000.857523:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.201.105@tcp lock: ffff8e77c683c400/0x1cd7fe53687370bb lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x11000000000000 nid: local remote: 0x1cd7fe53687370c2 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620000.857529:0:5841:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.201.105@tcp lock: ffff8e77c683c400/0x1cd7fe53687370bb lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x1000000000000 nid: local remote: 0x1cd7fe53687370c2 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788620000.857556:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c75f6d80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:5841:1875503589298816:0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1788620000.857571:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788620000.857579:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fdd87c00 going to sleep for 11 seconds 00000100:00100000:1.0:1788620000.857586:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589298816 00000100:00100000:1.0:1788620000.857588:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589298816 00000100:00100000:1.0:1788620000.857601:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 332 00000100:00100000:1.0:1788620000.857604:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5ce4050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+19:5841:x1875503589298816:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1788620000.857756:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5ce4050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+19:5841:x1875503589298816:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 149us (182us total) trans 0 rc 0/0 00000100:00100000:1.0:1788620000.857763:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 332 00000100:00100000:1.0:1788620000.857793:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c75f6d80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:5841:1875503589298816:0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1788620000.857816:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c75f6d80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:5841:1875503589298944:0@lo:503:ll_cfg_requeue.0 00000100:00100000:1.0:1788620000.857847:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788620000.857856:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78fdd87c00 going to sleep for 11 seconds 00000100:00100000:1.0:1788620000.857864:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589298944 00000100:00100000:1.0:1788620000.857866:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589298944 00000100:00100000:1.0:1788620000.857879:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 333 00000100:00100000:1.0:1788620000.857882:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5ce6850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+19:5841:x1875503589298944:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:1.0:1788620000.857966:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5ce6850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+19:5841:x1875503589298944:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 83us (118us total) trans 0 rc 0/0 00000100:00100000:1.0:1788620000.857971:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 333 00000100:00100000:1.0:1788620000.857997:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c75f6d80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:5841:1875503589298944:0@lo:503:ll_cfg_requeue.0 00000040:00080000:1.0:1788620000.869092:0:15625:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x2:0x0] index 64768 count 7 00000020:01000000:0.0:1788620000.869306:0:5841:0:(obd_config.c:2145:class_config_parse_llog()) Processed log params gen 7-6 (rc=0) 10000000:01000000:0.0:1788620000.869362:0:5841:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.201.105@tcp: configuration from log 'params' succeeded (0). 00010000:00010000:0.0:1788620000.869373:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.201.105@tcp lock: ffff8e77c683c400/0x1cd7fe53687370bb lrc: 3/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x11000000000000 nid: local remote: 0x1cd7fe53687370c2 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620000.869385:0:5841:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.201.105@tcp lock: ffff8e77c683c400/0x1cd7fe53687370bb lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x11000000000000 nid: local remote: 0x1cd7fe53687370c2 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788620000.879358:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df386300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3318:1875503589299072:0@lo:400:kworker.0 00000100:00100000:0.0:1788620000.879430:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788620000.879468:0:3318:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1788620000.879474:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df384700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3318:1875503589299200:0@lo:400:kworker.0 00000100:00100000:0.0:1788620000.879496:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788620000.882787:0:3317:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1788620000.884558:0:3316:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1788620000.884583:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df386680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3316:1875503589299456:0@lo:400:kworker.0 00000100:00100000:1.0:1788620000.884650:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788620000.884706:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589299072 00000100:00100000:1.0:1788620000.884713:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589299072 00000100:00100000:1.0:1788620000.884759:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589299200 00000100:00100000:1.0:1788620000.884763:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589299200 00000100:00100000:1.0:1788620000.884789:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589299456 00000100:00100000:1.0:1788620000.884791:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589299456 00000100:00100000:1.0:1788620000.884806:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 253 00000100:00100000:1.0:1788620000.884813:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3318:x1875503589299072:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788620000.884869:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3318:x1875503589299072:12345-0@lo:400:kworker.0 Request processed in 55us (5439us total) trans 0 rc 0/0 00000100:00100000:1.0:1788620000.885058:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 253 00000100:00100000:1.0:1788620000.885076:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 254 00000100:00100000:1.0:1788620000.885083:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a58700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3318:x1875503589299200:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788620000.885114:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a58700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3318:x1875503589299200:12345-0@lo:400:kworker.0 Request processed in 32us (5617us total) trans 0 rc 0/0 00000100:00100000:1.0:1788620000.885119:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 254 00000100:00100000:1.0:1788620000.885128:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 255 00000100:00100000:1.0:1788620000.885133:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c75f6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3316:x1875503589299456:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788620000.885158:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c75f6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3316:x1875503589299456:12345-0@lo:400:kworker.0 Request processed in 24us (508us total) trans 0 rc 0/0 00000100:00100000:1.0:1788620000.885164:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 255 00000100:00080000:1.0:1788620000.885178:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1788620000.885212:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df386680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3316:1875503589299456:0@lo:400:kworker.0 00000100:00100000:1.0:1788620000.885244:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df384700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3318:1875503589299200:0@lo:400:kworker.0 00000100:00100000:1.0:1788620000.885257:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df386300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3318:1875503589299072:0@lo:400:kworker.0 00000100:00100000:3.0:1788620000.888284:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df386300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589299584:0@lo:400:kworker.0 00000100:00100000:3.0:1788620000.888621:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788620000.888686:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589299584 00000100:00100000:2.0:1788620000.888693:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589299584 00000100:00100000:2.0:1788620000.888757:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 168 00000100:00100000:2.0:1788620000.888766:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df384700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589299584:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788620000.888805:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df384700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589299584:12345-0@lo:400:kworker.0 Request processed in 42us (185us total) trans 0 rc 0/0 00000100:00100000:2.0:1788620000.888810:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 168 00000100:00080000:3.0:1788620000.888916:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1788620000.888951:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df386300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589299584:0@lo:400:kworker.0 00000100:00100000:1.0:1788620000.889316:0:3317:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df385500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3317:1875503589299328:0@lo:400:kworker.0 00000100:00100000:1.0:1788620000.889375:0:3317:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788620000.889436:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589299328 00000100:00100000:2.0:1788620000.889441:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589299328 00000100:00100000:2.0:1788620000.889493:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 169 00000100:00100000:2.0:1788620000.889501:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c08380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3317:x1875503589299328:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788620000.889542:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c08380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3317:x1875503589299328:12345-0@lo:400:kworker.0 Request processed in 41us (166us total) trans 0 rc 0/0 00000100:00100000:2.0:1788620000.889548:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 169 00000100:00100000:1.0:1788620000.889568:0:3317:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df385500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3317:1875503589299328:0@lo:400:kworker.0 00000100:00080000:3.0:1788620000.889632:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:3.0:1788620001.067400:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788620001.100439:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788620001.101226:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788620001.101897:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571776896 00000100:00100000:0.0:1788620001.101908:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571776896 00000100:00100000:0.0:1788620001.102032:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 256 00000100:00100000:0.0:1788620001.102043:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df386300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571776896:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:0.0:1788620001.102360:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df386300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571776896:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 317us (1929us total) trans 0 rc 0/0 00000100:00100000:0.0:1788620001.102369:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 256 00000100:00100000:0.0:1788620001.102402:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571776768 00000100:00100000:0.0:1788620001.102407:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571776768 00000100:00100000:0.0:1788620001.102433:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 334 00000100:00100000:0.0:1788620001.102441:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5ce5450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:597450a2-e90c-408c-b401-a0ce48eed204+10:2355:x1875503571776768:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:0.0:1788620001.102469:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5ce5450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:597450a2-e90c-408c-b401-a0ce48eed204+10:2355:x1875503571776768:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 29us (1244us total) trans 0 rc 0/0 00000100:00100000:0.0:1788620001.102551:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 334 00000100:00080000:3.0:1788620001.102816:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1788620001.379755:0:3318:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1788620001.379791:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c5c08000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589299712:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1788620001.379862:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788620001.380005:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589299712 00000100:00100000:1.0:1788620001.380013:0:6699:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589299712 00000100:00100000:1.0:1788620001.380061:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 120 00000100:00100000:1.0:1788620001.380070:0:6699:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589299712:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1788620001.380119:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3856662528 avail 3854565376 00000100:00100000:1.0:1788620001.380167:0:6699:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589299712:12345-0@lo:13:osp-pre-1-0.0 Request processed in 97us (306us total) trans 0 rc 0/0 00000100:00100000:1.0:1788620001.380175:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 120 00000100:00100000:1.0:1788620001.380224:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c5c08000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589299712:0@lo:13:osp-pre-1-0.0 00010000:00010000:1.0:1788620002.082592:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788620002.404217:0:3317:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1788620002.404241:0:3317:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e79010fd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3317:1875503589299840:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1788620002.404320:0:3317:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788620002.404419:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589299840 00000100:00100000:1.0:1788620002.404426:0:6699:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589299840 00000100:00100000:1.0:1788620002.404493:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 121 00000100:00100000:1.0:1788620002.404504:0:6699:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010fc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3317:x1875503589299840:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1788620002.404541:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1788620002.404588:0:6699:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010fc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3317:x1875503589299840:12345-0@lo:13:osp-pre-0-0.0 Request processed in 86us (267us total) trans 0 rc 0/0 00000100:00100000:1.0:1788620002.404595:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 121 00000100:00100000:1.0:1788620002.404650:0:3317:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e79010fd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3317:1875503589299840:0@lo:13:osp-pre-0-0.0 00000100:00080000:3.0:1788620002.404743:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:02000400:1.0:1788620003.079938:0:15764:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: oleg105-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 00010000:00010000:1.0:1788620003.119172:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788620004.131104:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788620005.158514:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788620005.986820:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df385500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3318:1875503589300096:0@lo:400:kworker.0 00000100:00100000:1.0:1788620005.986834:0:3317:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df386680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:3317:1875503589299968:0@lo:400:kworker.0 00000100:00100000:1.0:1788620005.986937:0:3317:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788620005.986973:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788620005.987003:0:3317:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df19f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3317:1875503589300480:0@lo:400:kworker.0 00000100:00100000:0.0:1788620005.987003:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df19d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589300608:0@lo:400:kworker.0 00000100:00100000:1.0:1788620005.987027:0:3317:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788620005.987027:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788620005.987054:0:3317:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1788620005.987054:0:3318:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1788620005.987060:0:3317:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df19c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3317:1875503589300352:0@lo:400:kworker.0 00000100:00100000:0.0:1788620005.987060:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df19e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3318:1875503589300224:0@lo:400:kworker.0 00000100:00100000:0.0:1788620005.987080:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788620005.987081:0:3317:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788620005.987113:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589299968 00000100:00100000:1.0:1788620005.987114:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589300096 00000100:00100000:1.0:1788620005.987122:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589300096 00000100:00100000:0.0:1788620005.987122:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589299968 00000100:00100000:0.0:1788620005.987174:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 335 00000100:00100000:0.0:1788620005.987197:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5ce7050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+19:3317:x1875503589299968:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788620005.987238:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589300608 00000100:00100000:2.0:1788620005.987244:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589300608 00000100:00100000:0.0:1788620005.987283:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5ce7050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+19:3317:x1875503589299968:12345-0@lo:400:kworker.0 Request processed in 88us (346us total) trans 0 rc 0/0 00000100:00100000:2.0:1788620005.987288:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589300352 00000100:00100000:2.0:1788620005.987291:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589300352 00000100:00100000:0.0:1788620005.987292:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 335 00000100:00100000:2.0:1788620005.987311:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 170 00000100:00100000:2.0:1788620005.987319:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a59f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589300608:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788620005.987385:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a59f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589300608:12345-0@lo:400:kworker.0 Request processed in 67us (356us total) trans 0 rc 0/0 00000100:00100000:2.0:1788620005.987392:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 170 00000100:00100000:2.0:1788620005.987406:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 171 00000100:00100000:2.0:1788620005.987413:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5c08000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3317:x1875503589300352:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788620005.987443:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5c08000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3317:x1875503589300352:12345-0@lo:400:kworker.0 Request processed in 30us (361us total) trans 0 rc 0/0 00000100:00100000:2.0:1788620005.987448:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 171 00000100:00100000:2.0:1788620005.987509:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df19d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589300608:0@lo:400:kworker.0 00000100:00080000:3.0:1788620005.987630:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1788620005.987635:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1788620005.996732:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589300480 00000100:00100000:1.0:1788620005.996760:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589300480 00000100:00100000:2.0:1788620006.003891:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589300224 00000100:00100000:2.0:1788620006.003916:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589300224 00000100:00100000:2.0:1788620006.005171:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 257 00000100:00100000:2.0:1788620006.005197:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a58a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3318:x1875503589300096:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788620006.005373:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a58a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3318:x1875503589300096:12345-0@lo:400:kworker.0 Request processed in 187us (18398us total) trans 0 rc 0/0 00000100:00100000:3.0:1788620006.006662:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 258 00000100:00100000:3.0:1788620006.006684:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e79010fd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3317:x1875503589300480:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788620006.006770:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e79010fd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3317:x1875503589300480:12345-0@lo:400:kworker.0 Request processed in 97us (19742us total) trans 0 rc 0/0 00000100:00100000:3.0:1788620006.006778:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 258 00000100:00100000:3.0:1788620006.006811:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 259 00000100:00100000:3.0:1788620006.006820:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a59500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3318:x1875503589300224:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788620006.006867:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a59500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3318:x1875503589300224:12345-0@lo:400:kworker.0 Request processed in 49us (19786us total) trans 0 rc 0/0 00000100:00100000:3.0:1788620006.006874:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 259 00000100:00100000:3.0:1788620006.006967:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df19e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3318:1875503589300224:0@lo:400:kworker.0 00000100:00100000:3.0:1788620006.006983:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df385500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3318:1875503589300096:0@lo:400:kworker.0 00000100:00080000:3.0:1788620006.007035:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788620006.007068:0:3317:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df19c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3317:1875503589300352:0@lo:400:kworker.0 00000100:00100000:3.0:1788620006.007082:0:3317:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df386680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:3317:1875503589299968:0@lo:400:kworker.0 00000100:00100000:3.0:1788620006.007097:0:3317:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df19f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3317:1875503589300480:0@lo:400:kworker.0 00000100:00100000:2.0:1788620006.007702:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 257 00010000:00010000:1.0:1788620006.181291:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788620006.216425:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788620006.216474:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788620006.216514:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571777152 00000100:00100000:0.0:1788620006.216524:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571777152 00000100:00100000:0.0:1788620006.216596:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 260 00000100:00100000:0.0:1788620006.216606:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571777152:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:0.0:1788620006.216675:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2355:x1875503571777152:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 70us (260us total) trans 0 rc 0/0 00000100:00100000:0.0:1788620006.216683:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 260 00000100:00100000:0.0:1788620006.216707:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571777024 00000100:00100000:0.0:1788620006.216710:0:5836:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571777024 00000100:00100000:0.0:1788620006.216733:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 336 00000100:00100000:0.0:1788620006.216739:0:5836:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5ce6050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:597450a2-e90c-408c-b401-a0ce48eed204+10:2355:x1875503571777024:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:0.0:1788620006.216766:0:5836:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5ce6050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:597450a2-e90c-408c-b401-a0ce48eed204+10:2355:x1875503571777024:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 26us (292us total) trans 0 rc 0/0 00000100:00100000:0.0:1788620006.216772:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 336 00000100:00080000:0.0:1788620006.216796:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1788620006.504121:0:3319:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6a59880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3319:1875503589300736:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1788620006.504192:0:3319:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788620006.504273:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589300736 00000100:00100000:0.0:1788620006.504280:0:6699:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589300736 00000100:00100000:0.0:1788620006.504340:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 122 00000100:00100000:0.0:1788620006.504350:0:6699:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1875503589300736:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1788620006.504386:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3856662528 avail 3854565376 00000100:00100000:0.0:1788620006.504431:0:6699:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1875503589300736:12345-0@lo:13:osp-pre-1-0.0 Request processed in 83us (239us total) trans 0 rc 0/0 00000100:00100000:0.0:1788620006.504439:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 122 00000100:00100000:0.0:1788620006.504488:0:3319:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6a59880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3319:1875503589300736:0@lo:13:osp-pre-1-0.0 00000001:02000400:3.0:1788620006.873991:0:15864:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: === sanity: finish setup 10:53:25 (1788620005) === 00000100:00100000:3.0:1788620007.003606:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788620007.004612:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571777280 00000100:00100000:1.0:1788620007.004624:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571777280 00000100:00100000:1.0:1788620007.004685:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1668 00000100:00100000:1.0:1788620007.004696:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+14:15483:x1875503571777280:12345-192.168.201.5@tcp:101:bash.0 00010000:00010000:1.0:1788620007.004716:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788620007.004736:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687370c9 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1e026 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.004803:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe53687370d0 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.004819:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe53687370d0 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.004832:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe53687370d0 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.004847:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687370d7 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x54ef8a6e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.004854:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687370d7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x54ef8a6e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.004861:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687370d7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x54ef8a6e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.005054:0:11397:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe53687370d0 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.005062:0:11397:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe53687370d0 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.005068:0:11397:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe53687370d0 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.005073:0:11397:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c683ca00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:1.0:1788620007.005077:0:11397:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe53687370d0 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.005088:0:11397:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe53687370d0 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.005101:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78c7cbbe00) 00010000:00010000:1.0:1788620007.005104:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788620007.005109:0:11397:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe53687370d0 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.005115:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe53687370d0 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.005126:0:11397:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687370d7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x54ef8a6e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.005132:0:11397:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687370d7 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x54ef8a6e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.005138:0:11397:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687370d7 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x54ef8a6e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.005144:0:11397:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c683d800 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:1.0:1788620007.005147:0:11397:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687370d7 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x54ef8a6e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.005153:0:11397:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687370d7 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x54ef8a6e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.005161:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x54ef8a6e (ffff8e78c67bce00) 00010000:00010000:1.0:1788620007.005164:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788620007.005167:0:11397:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687370d7 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x54ef8a6e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.005178:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687370d7 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x54ef8a6e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.005208:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687370c9 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e026 expref: 15 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.005233:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78c7cbbe00) 00010000:00010000:1.0:1788620007.005235:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1788620007.005239:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d200/0x1cd7fe53687370c9 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e026 expref: 15 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.005250:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d200, rc 0) 00000100:00100000:1.0:1788620007.005312:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+14:15483:x1875503571777280:12345-192.168.201.5@tcp:101:bash.0 Request processed in 619us (1715us total) trans 0 rc 301/301 00000100:00100000:1.0:1788620007.005320:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1668 00000100:00100000:0.0:1788620007.014187:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788620007.014270:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571777408 00000100:00100000:2.0:1788620007.014280:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571777408 00000100:00100000:2.0:1788620007.014371:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1669 00000100:00100000:2.0:1788620007.014382:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a59880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:15483:x1875503571777408:12345-192.168.201.5@tcp:101:bash.0 00010000:00010000:2.0:1788620007.014403:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788620007.014423:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3ec00/0x1cd7fe53687370de lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1e02d expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.014494:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e000/0x1cd7fe53687370e5 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.014508:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e000/0x1cd7fe53687370e5 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.014519:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e000/0x1cd7fe53687370e5 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.014597:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e400/0x1cd7fe53687370ec lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x77bdd2c7 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.014606:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e400/0x1cd7fe53687370ec lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x77bdd2c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.014613:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e400/0x1cd7fe53687370ec lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x77bdd2c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.014806:0:5846:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e000/0x1cd7fe53687370e5 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.014814:0:5846:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e000/0x1cd7fe53687370e5 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.014821:0:5846:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e000/0x1cd7fe53687370e5 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.014827:0:5846:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c6a3e000 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:2.0:1788620007.014832:0:5846:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e000/0x1cd7fe53687370e5 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.014843:0:5846:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e000/0x1cd7fe53687370e5 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.014854:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78c7cbbe00) 00010000:00010000:2.0:1788620007.014857:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788620007.014862:0:5846:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e000/0x1cd7fe53687370e5 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.014868:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e000/0x1cd7fe53687370e5 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.014879:0:5846:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e400/0x1cd7fe53687370ec lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x77bdd2c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.014884:0:5846:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e400/0x1cd7fe53687370ec lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x77bdd2c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.014890:0:5846:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e400/0x1cd7fe53687370ec lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x77bdd2c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.014894:0:5846:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c6a3e400 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:2.0:1788620007.014897:0:5846:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e400/0x1cd7fe53687370ec lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x77bdd2c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.014902:0:5846:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e400/0x1cd7fe53687370ec lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x77bdd2c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.014908:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x77bdd2c7 (ffff8e78c7cbb600) 00010000:00010000:2.0:1788620007.014911:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788620007.014914:0:5846:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e400/0x1cd7fe53687370ec lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x77bdd2c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.014924:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e400/0x1cd7fe53687370ec lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x77bdd2c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.014954:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3ec00/0x1cd7fe53687370de lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e02d expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.014964:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78c7cbbe00) 00010000:00010000:2.0:1788620007.014966:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1788620007.014969:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3ec00/0x1cd7fe53687370de lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e02d expref: 15 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.014978:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6a3ec00, rc 0) 00000100:00100000:2.0:1788620007.015037:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a59880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:15483:x1875503571777408:12345-192.168.201.5@tcp:101:bash.0 Request processed in 657us (859us total) trans 0 rc 301/301 00000100:00100000:2.0:1788620007.015044:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1669 00000100:00100000:0.0:1788620007.024070:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788620007.024861:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571777536 00000100:00100000:2.0:1788620007.024872:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571777536 00000100:00100000:2.0:1788620007.024926:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 42 00000100:00100000:2.0:1788620007.024937:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a59180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:15483:x1875503571777536:12345-192.168.201.5@tcp:37:bash.0 00000100:00100000:0.0:1788620007.030828:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a59180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:15483:x1875503571777536:12345-192.168.201.5@tcp:37:bash.0 Request processed in 5882us (6756us total) trans 0 rc 0/0 00000100:00100000:0.0:1788620007.030841:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 42 00000100:00100000:0.0:1788620007.048171:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788620007.048267:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571777664 00000100:00100000:3.0:1788620007.048278:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571777664 00000100:00100000:3.0:1788620007.048364:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1670 00000100:00100000:3.0:1788620007.048375:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a58380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:18223:x1875503571777664:12345-192.168.201.5@tcp:101:rm.0 00010000:00010000:3.0:1788620007.048395:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788620007.048414:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06200/0x1cd7fe53687370f3 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1e034 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.048500:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06c00/0x1cd7fe53687370fa lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.048515:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06c00/0x1cd7fe53687370fa lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.048526:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06c00/0x1cd7fe53687370fa lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.048599:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe5368737101 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x98980ae bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.048607:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe5368737101 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x98980ae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.048612:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe5368737101 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x98980ae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.048993:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06e00/0x1cd7fe5368737108 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.049003:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06e00/0x1cd7fe5368737108 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.049010:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06e00/0x1cd7fe5368737108 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788620007.049044:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x5:0x0]: returning size 5477732 00000004:00200000:3.0:1788620007.049049:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788620007.049068:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06e00/0x1cd7fe5368737108 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1788620007.049087:0:5845:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x5:0x0] updating lvb size from disk: 0 -> 0 00000004:00010000:3.0:1788620007.049091:0:5845:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x5:0x0] updating lvb mtime from disk: 0 -> 1788619837 00000004:00010000:3.0:1788620007.049093:0:5845:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x5:0x0] updating lvb atime from disk: 0 -> 1788619837 00000004:00010000:3.0:1788620007.049096:0:5845:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x5:0x0] updating lvb ctime from disk: 0 -> 1788619837 00000004:00010000:3.0:1788620007.049098:0:5845:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x5:0x0] updating lvb blocks from disk: 18427282363563114435 -> 2 00002000:00010000:3.0:1788620007.049102:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 0 00010000:00010000:3.0:1788620007.049107:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06c00/0x1cd7fe53687370fa lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.049114:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06c00/0x1cd7fe53687370fa lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.049120:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06c00/0x1cd7fe53687370fa lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.049126:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78dfa06c00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:3.0:1788620007.049131:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06c00/0x1cd7fe53687370fa lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.049139:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06c00/0x1cd7fe53687370fa lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.049150:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78c7cbbe00) 00010000:00010000:3.0:1788620007.049154:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788620007.049158:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06c00/0x1cd7fe53687370fa lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.049165:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06c00/0x1cd7fe53687370fa lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.049174:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe5368737101 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x98980ae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.049180:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe5368737101 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x98980ae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.049186:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe5368737101 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x98980ae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.049192:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78dfa07a00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:3.0:1788620007.049194:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe5368737101 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x98980ae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.049200:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe5368737101 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x98980ae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.049208:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x98980ae (ffff8e78c3796f00) 00010000:00010000:3.0:1788620007.049210:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788620007.049223:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe5368737101 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x98980ae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.049229:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe5368737101 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x98980ae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.049263:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06200/0x1cd7fe53687370f3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e034 expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.049276:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06e00/0x1cd7fe5368737108 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e034 expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.049283:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5:0x0].0x0 (ffff8e78c3796800) 00010000:00010000:3.0:1788620007.049285:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5b 00010000:00010000:3.0:1788620007.049287:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa06e00, rc 0) 00000100:00100000:3.0:1788620007.049421:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a58380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:18223:x1875503571777664:12345-192.168.201.5@tcp:101:rm.0 Request processed in 1047us (1258us total) trans 0 rc 0/0 00000100:00100000:3.0:1788620007.049429:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1670 00000100:00100000:1.0:1788620007.072494:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788620007.073301:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571777792 00000100:00100000:3.0:1788620007.073312:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571777792 00000100:00100000:3.0:1788620007.073366:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1373 00000100:00100000:3.0:1788620007.073374:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df964000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:0+-99:2351:x1875503571777792:12345-192.168.201.5@tcp:8:rm.0 00010000:00080000:3.0:1788620007.073405:0:6697:0:(ldlm_lib.c:1444:target_handle_connect()) lustre-OST0001: connection from 92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp t0 exp 0000000000000000 cur 499 last 0 00000020:00000080:3.0:1788620007.073432:0:6697:0:(genops.c:1385:class_connect()) connect: client 92dda994-8428-47f5-a731-0b06f1aba2e4, cookie 0x1cd7fe536873710f 00002000:00100000:3.0:1788620007.073445:0:6697:0:(ofd_obd.c:121:ofd_parse_connect_data()) lustre-OST0001: cli 92dda994-8428-47f5-a731-0b06f1aba2e4/ffff8e77c577c000 ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 2113a03 ocd_grant: 8519680 ocd_index: 0 ocd_group 0 00002000:00100000:3.0:1788620007.073457:0:6697:0:(ofd_obd.c:219:ofd_parse_connect_data()) lustre-OST0001: cli supports cksum type f7, return f7 00002000:00080000:3.0:1788620007.073697:0:6697:0:(ofd_obd.c:385:ofd_obd_connect()) lustre-OST0001: get connection from MDS 0 00000100:00080000:3.0:1788620007.073726:0:6697:0:(import.c:65:import_set_state_nolock()) ffff8e77c577d000 : changing import state from RECOVER to FULL 00000100:00100000:3.0:1788620007.073799:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df964000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2351:x1875503571777792:12345-192.168.201.5@tcp:8:rm.0 Request processed in 424us (1314us total) trans 0 rc 0/0 00000100:00100000:3.0:1788620007.073806:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1373 00000100:00100000:3.0:1788620007.077590:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788620007.077624:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571777920 00000100:00100000:3.0:1788620007.077633:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571777920 00000100:00100000:3.0:1788620007.077696:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1374 00000100:00100000:3.0:1788620007.077706:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:0:x1875503571777920:12345-192.168.201.5@tcp:101:rm.0 00010000:00010000:3.0:1788620007.077723:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788620007.077743:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005cb4402d/0x1cd7fe5368737116 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1e03b expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788620007.077769:0:6696:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x2:0x0] initial LVB size: 5477732, mtime: 0x6a9c2c3d, atime: 0x6a9c2c3e, ctime: 0x6a9c2c3d, blocks: 0x1007 00010000:00010000:3.0:1788620007.077787:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x2:0x0].0x0 since=499s v=(0 0) 00010000:00010000:3.0:1788620007.077793:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788620007.077804:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005cb4402d/0x1cd7fe5368737116 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e03b expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.077824:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005cb4402d/0x1cd7fe5368737116 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e03b expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.077838:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa06200, rc 0) 00000100:00100000:3.0:1788620007.077889:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:0:x1875503571777920:12345-192.168.201.5@tcp:101:rm.0 Request processed in 184us (308us total) trans 0 rc 0/0 00000100:00100000:3.0:1788620007.077895:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1374 00000100:00080000:0.0:1788620007.078387:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1788620007.089537:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788620007.089847:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571778176 00000100:00100000:3.0:1788620007.089858:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571778176 00000100:00100000:3.0:1788620007.090015:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1671 00000100:00100000:3.0:1788620007.090027:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a58e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+15:18223:x1875503571778176:12345-192.168.201.5@tcp:36:rm.0 00010000:00010000:3.0:1788620007.090152:0:11397:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:3.0:1788620007.090287:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06e00/0x1cd7fe5368737108 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e034 expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.090308:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5:0x0].0x0 (ffff8e78c3796800) 00010000:00010000:3.0:1788620007.090343:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c5f72200/0x1cd7fe5368735ddd lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1cfe1 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.090355:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78c7cbbe00) 00010000:00010000:3.0:1788620007.090363:0:11397:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788620007.090385:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe536873711d lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.090399:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe536873711d lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.090410:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe536873711d lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.090424:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06c00/0x1cd7fe5368737124 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x98980ae bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.090431:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06c00/0x1cd7fe5368737124 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x98980ae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.090438:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06c00/0x1cd7fe5368737124 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x98980ae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.091451:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe536873712b lrc: 3/0,1 mode: --/EX res: [0x200000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.091467:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe536873712b lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.091476:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe536873712b lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1788620007.096134:0:11397:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].2, 64 off8320 00000004:00080000:3.0:1788620007.096156:0:11397:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 499765829302ns, next at 500765825147ns, rc = 0 00010000:00010000:3.0:1788620007.096284:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe5368737132 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.096304:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe5368737132 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.096318:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe5368737132 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.096341:0:11397:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe5368737132 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.096349:0:11397:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe5368737132 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.096355:0:11397:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe5368737132 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.096378:0:11397:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1788620007.096400:0:11397:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe536873712b lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.096405:0:11397:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe536873712b lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.096411:0:11397:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe536873712b lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.096417:0:11397:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78dfa07e00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:3.0:1788620007.096422:0:11397:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe536873712b lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.096429:0:11397:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe536873712b lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.096441:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5:0x0].0x0 (ffff8e78c3796c00) 00010000:00010000:3.0:1788620007.096444:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1788620007.096448:0:11397:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe536873712b lrc: 2/0,0 mode: --/EX res: [0x200000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.096454:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe536873712b lrc: 0/0,0 mode: --/EX res: [0x200000401:0x5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.096472:0:11397:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe536873711d lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.096488:0:11397:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe536873711d lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.096494:0:11397:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe536873711d lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.096499:0:11397:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78dfa07a00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:3.0:1788620007.096502:0:11397:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe536873711d lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.096511:0:11397:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe536873711d lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.096518:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78c3796200) 00010000:00010000:3.0:1788620007.096520:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788620007.096523:0:11397:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe536873711d lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.096529:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe536873711d lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1788620007.096544:0:11397:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000544a3c47 reply state 0000000011d2362c transno 4294967890 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06c00/0x1cd7fe5368737124 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x98980ae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788620007.096553:0:11397:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788620007.096647:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a58e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+15:18223:x1875503571778176:12345-192.168.201.5@tcp:36:rm.0 Request processed in 6618us (7114us total) trans 4294967890 rc 0/0 00000100:00100000:3.0:1788620007.096656:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1671 00010000:00010000:1.0:1788620007.099098:0:5832:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1788620007.099114:0:5832:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe5368737132 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.099125:0:5832:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78dfa06800 already unused, calling callback (ffffffffc17c24f0) 00010000:00010000:1.0:1788620007.099131:0:5832:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe5368737132 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.099140:0:5832:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe5368737132 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.099157:0:5832:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5:0x0].0x0 (ffff8e78c3796c00) 00010000:00010000:1.0:1788620007.099161:0:5832:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788620007.099167:0:5832:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe5368737132 lrc: 1/0,0 mode: --/PW res: [0x200000401:0x5:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.099173:0:5832:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe5368737132 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x5:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.099190:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788620007.099192:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788620007.099193:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788620007.112807:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06c00/0x1cd7fe5368737124 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x98980ae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.112824:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06c00/0x1cd7fe5368737124 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x98980ae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.112833:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06c00/0x1cd7fe5368737124 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x98980ae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.112840:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78dfa06c00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:1.0:1788620007.112846:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06c00/0x1cd7fe5368737124 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x98980ae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.112853:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06c00/0x1cd7fe5368737124 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x98980ae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.112871:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x98980ae (ffff8e78c3796e00) 00010000:00010000:1.0:1788620007.112875:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788620007.112880:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06c00/0x1cd7fe5368737124 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x98980ae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.112887:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06c00/0x1cd7fe5368737124 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x98980ae bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788620007.116871:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788620007.116964:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571778304 00000100:00100000:0.0:1788620007.116975:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571778304 00000100:00100000:0.0:1788620007.117034:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1672 00000100:00100000:0.0:1788620007.117045:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa40380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+14:18223:x1875503571778304:12345-192.168.201.5@tcp:101:rm.0 00010000:00010000:0.0:1788620007.117068:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788620007.117090:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777ca00/0x1cd7fe5368737139 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1e042 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.117155:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777da00/0x1cd7fe5368737140 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.117170:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777da00/0x1cd7fe5368737140 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.117181:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777da00/0x1cd7fe5368737140 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788620007.117245:0:9417:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:0.0:1788620007.117249:0:9417:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1788620007.117268:0:9417:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777da00/0x1cd7fe5368737140 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.117304:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777ca00/0x1cd7fe5368737139 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e042 expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.117321:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777da00/0x1cd7fe5368737140 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e042 expref: 15 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.117358:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78cfedf700) 00010000:00010000:0.0:1788620007.117360:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788620007.117364:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c777da00, rc 0) 00000100:00100000:0.0:1788620007.117420:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa40380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:18223:x1875503571778304:12345-192.168.201.5@tcp:101:rm.0 Request processed in 377us (559us total) trans 0 rc 0/0 00000100:00100000:0.0:1788620007.117428:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1672 00000100:00100000:2.0:1788620007.129617:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788620007.129932:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571778432 00000100:00100000:3.0:1788620007.129943:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571778432 00000100:00100000:3.0:1788620007.130004:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1673 00000100:00100000:3.0:1788620007.130015:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa40e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+15:18223:x1875503571778432:12345-192.168.201.5@tcp:101:rm.0 00010000:00010000:3.0:1788620007.130046:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788620007.130066:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe5368737147 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1e049 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.130128:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe536873714e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.130147:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe536873714e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.130171:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe536873714e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.130186:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06e00/0x1cd7fe5368737155 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xcf86d744 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.130193:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06e00/0x1cd7fe5368737155 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xcf86d744 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.130199:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06e00/0x1cd7fe5368737155 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xcf86d744 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.130300:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06c00/0x1cd7fe536873715c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.130308:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06c00/0x1cd7fe536873715c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.130315:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06c00/0x1cd7fe536873715c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788620007.130375:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788620007.130395:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06c00/0x1cd7fe536873715c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1788620007.130417:0:5845:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x6:0x0] updating lvb size from disk: 0 -> 234852 00000004:00010000:3.0:1788620007.130420:0:5845:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x6:0x0] updating lvb mtime from disk: 0 -> 1788619838 00000004:00010000:3.0:1788620007.130423:0:5845:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x6:0x0] updating lvb atime from disk: 0 -> 1788619837 00000004:00010000:3.0:1788620007.130426:0:5845:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x6:0x0] updating lvb ctime from disk: 0 -> 1788619838 00000004:00010000:3.0:1788620007.130428:0:5845:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x6:0x0] updating lvb blocks from disk: 18427282363563114435 -> 518 00002000:00010000:3.0:1788620007.130432:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 234852 00010000:00010000:3.0:1788620007.130438:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe536873714e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.130445:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe536873714e lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.130451:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe536873714e lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.130457:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78dfa07e00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:3.0:1788620007.130461:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe536873714e lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.130469:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe536873714e lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.130481:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78cfedf700) 00010000:00010000:3.0:1788620007.130484:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788620007.130489:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe536873714e lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.130496:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe536873714e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.130508:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06e00/0x1cd7fe5368737155 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xcf86d744 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.130514:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06e00/0x1cd7fe5368737155 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xcf86d744 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.130524:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06e00/0x1cd7fe5368737155 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xcf86d744 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.130530:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78dfa06e00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:3.0:1788620007.130533:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06e00/0x1cd7fe5368737155 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xcf86d744 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.130539:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06e00/0x1cd7fe5368737155 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xcf86d744 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.130546:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xcf86d744 (ffff8e78c3796200) 00010000:00010000:3.0:1788620007.130549:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788620007.130552:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06e00/0x1cd7fe5368737155 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xcf86d744 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.130557:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06e00/0x1cd7fe5368737155 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xcf86d744 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.130588:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe5368737147 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e049 expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.130602:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06c00/0x1cd7fe536873715c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e049 expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.130609:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6:0x0].0x0 (ffff8e78c3796800) 00010000:00010000:3.0:1788620007.130612:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5b 00010000:00010000:3.0:1788620007.130614:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa06c00, rc 0) 00000100:00100000:3.0:1788620007.130678:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa40e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:18223:x1875503571778432:12345-192.168.201.5@tcp:101:rm.0 Request processed in 664us (1070us total) trans 0 rc 0/0 00000100:00100000:3.0:1788620007.130689:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1673 00000100:00100000:3.0:1788620007.137057:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788620007.137207:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571778688 00000100:00100000:1.0:1788620007.137225:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571778688 00000100:00100000:1.0:1788620007.137290:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1674 00000100:00100000:1.0:1788620007.137301:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+16:18223:x1875503571778688:12345-192.168.201.5@tcp:36:rm.0 00010000:00010000:1.0:1788620007.137391:0:11397:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:1.0:1788620007.137442:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06c00/0x1cd7fe536873715c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x6:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e049 expref: 16 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.137461:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6:0x0].0x0 (ffff8e78c3796800) 00010000:00010000:1.0:1788620007.137477:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777da00/0x1cd7fe5368737140 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e042 expref: 15 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.137501:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78cfedf700) 00010000:00010000:1.0:1788620007.137509:0:11397:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788620007.137531:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368737163 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.137545:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368737163 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.137557:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368737163 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.137571:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe536873716a lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xcf86d744 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.137579:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe536873716a lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xcf86d744 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.137590:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe536873716a lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xcf86d744 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.137694:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368737171 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x6:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.137703:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368737171 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.137712:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368737171 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.138302:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368737178 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x6:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.138314:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368737178 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x6:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.138323:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368737178 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x6:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.138365:0:11397:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368737178 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x6:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.138372:0:11397:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368737178 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x6:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.138378:0:11397:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368737178 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x6:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.138401:0:11397:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1788620007.138421:0:11397:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368737171 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.138427:0:11397:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368737171 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.138434:0:11397:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368737171 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.138440:0:11397:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c683ce00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:1.0:1788620007.138444:0:11397:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368737171 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.138452:0:5833:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1788620007.138452:0:11397:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368737171 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.138458:0:5833:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368737178 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x6:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.138462:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6:0x0].0x0 (ffff8e78c67bcd00) 00010000:00010000:1.0:1788620007.138465:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1788620007.138467:0:5833:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c683c200 already unused, calling callback (ffffffffc17c24f0) 00010000:00010000:1.0:1788620007.138469:0:11397:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368737171 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.138471:0:5833:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368737178 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x6:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.138475:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368737171 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x6:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.138478:0:5833:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368737178 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x6:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40018480000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.138486:0:5833:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6:0x0].0x0 (ffff8e78c67bcd00) 00010000:00010000:2.0:1788620007.138489:0:5833:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788620007.138489:0:11397:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368737163 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.138492:0:5833:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368737178 lrc: 1/0,0 mode: --/PW res: [0x200000401:0x6:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.138495:0:11397:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368737163 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.138498:0:5833:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe5368737178 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x6:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.138502:0:11397:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368737163 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.138506:0:11397:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c683ca00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:2.0:1788620007.138510:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788620007.138510:0:11397:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368737163 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.138512:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788620007.138514:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788620007.138516:0:11397:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368737163 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.138522:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78c67bce00) 00010000:00010000:1.0:1788620007.138525:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788620007.138528:0:11397:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368737163 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.138533:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368737163 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1788620007.138544:0:11397:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000f7121b29 reply state 00000000b387e2fb transno 4294967891 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe536873716a lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xcf86d744 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788620007.138553:0:11397:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788620007.138635:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+16:18223:x1875503571778688:12345-192.168.201.5@tcp:36:rm.0 Request processed in 1334us (1587us total) trans 4294967891 rc 0/0 00000100:00100000:1.0:1788620007.138643:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1674 00010000:00010000:0.0:1788620007.144954:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe536873716a lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xcf86d744 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.144971:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe536873716a lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xcf86d744 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.144978:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe536873716a lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xcf86d744 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.144984:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c683cc00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788620007.144989:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe536873716a lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xcf86d744 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.144997:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe536873716a lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xcf86d744 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.145014:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xcf86d744 (ffff8e78c67bc600) 00010000:00010000:0.0:1788620007.145017:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788620007.145022:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe536873716a lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xcf86d744 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.145028:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe536873716a lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xcf86d744 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788620007.153878:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788620007.154513:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571778816 00000100:00100000:0.0:1788620007.154524:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571778816 00000100:00100000:0.0:1788620007.154591:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1675 00000100:00100000:0.0:1788620007.154603:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:18223:x1875503571778816:12345-192.168.201.5@tcp:101:rm.0 00010000:00010000:0.0:1788620007.154627:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788620007.154651:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777ca00/0x1cd7fe536873717f lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1e050 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.154721:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d000/0x1cd7fe5368737186 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.154735:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d000/0x1cd7fe5368737186 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.154745:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d000/0x1cd7fe5368737186 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788620007.154779:0:9417:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:0.0:1788620007.154783:0:9417:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1788620007.154802:0:9417:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d000/0x1cd7fe5368737186 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.154826:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777ca00/0x1cd7fe536873717f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e050 expref: 17 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.154844:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d000/0x1cd7fe5368737186 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e050 expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.154853:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78cfedfb00) 00010000:00010000:0.0:1788620007.154856:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788620007.154859:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c777d000, rc 0) 00000100:00100000:0.0:1788620007.154926:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:18223:x1875503571778816:12345-192.168.201.5@tcp:101:rm.0 Request processed in 325us (1057us total) trans 0 rc 0/0 00000100:00100000:0.0:1788620007.154933:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1675 00000100:00100000:2.0:1788620007.172186:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788620007.172295:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571778944 00000100:00100000:3.0:1788620007.172305:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571778944 00000100:00100000:3.0:1788620007.172407:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1676 00000100:00100000:3.0:1788620007.172418:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa43b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:18223:x1875503571778944:12345-192.168.201.5@tcp:101:rm.0 00010000:00010000:3.0:1788620007.172450:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788620007.172471:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe536873718d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1e057 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.172535:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06e00/0x1cd7fe5368737194 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.172550:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06e00/0x1cd7fe5368737194 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.172561:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06e00/0x1cd7fe5368737194 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.172630:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe536873719b lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x32368eb3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.172638:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe536873719b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x32368eb3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.172645:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe536873719b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x32368eb3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.172726:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe53687371a2 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.172735:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe53687371a2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.172742:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe53687371a2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788620007.172777:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x7:0x0]: returning size 1049076 00000004:00200000:3.0:1788620007.172781:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788620007.172799:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe53687371a2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x7:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1788620007.172819:0:5845:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x7:0x0] updating lvb size from disk: 0 -> 1048476 00000004:00010000:3.0:1788620007.172822:0:5845:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x7:0x0] updating lvb mtime from disk: 0 -> 1788619845 00000004:00010000:3.0:1788620007.172825:0:5845:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x7:0x0] updating lvb atime from disk: 0 -> 1788619845 00000004:00010000:3.0:1788620007.172827:0:5845:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x7:0x0] updating lvb ctime from disk: 0 -> 1788619845 00000004:00010000:3.0:1788620007.172829:0:5845:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x7:0x0] updating lvb blocks from disk: 18427282363563114435 -> 2054 00002000:00010000:3.0:1788620007.172833:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048476 00010000:00010000:3.0:1788620007.172838:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06e00/0x1cd7fe5368737194 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.172845:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06e00/0x1cd7fe5368737194 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.172852:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06e00/0x1cd7fe5368737194 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.172857:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78dfa06e00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:3.0:1788620007.172861:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06e00/0x1cd7fe5368737194 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.172873:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06e00/0x1cd7fe5368737194 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.172884:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78cfedfb00) 00010000:00010000:3.0:1788620007.172887:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788620007.172891:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06e00/0x1cd7fe5368737194 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.172897:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06e00/0x1cd7fe5368737194 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.172909:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe536873719b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x32368eb3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.172915:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe536873719b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x32368eb3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.172921:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe536873719b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x32368eb3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.172925:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78dfa07e00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:3.0:1788620007.172928:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe536873719b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x32368eb3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.172934:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe536873719b lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x32368eb3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.172942:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x32368eb3 (ffff8e78c3796200) 00010000:00010000:3.0:1788620007.172944:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788620007.172947:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe536873719b lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x32368eb3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.172956:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe536873719b lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x32368eb3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.172984:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07a00/0x1cd7fe536873718d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e057 expref: 18 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.172997:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe53687371a2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x7:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e057 expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.173005:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x7:0x0].0x0 (ffff8e78c3796400) 00010000:00010000:3.0:1788620007.173007:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5b 00010000:00010000:3.0:1788620007.173010:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa06800, rc 0) 00000100:00100000:3.0:1788620007.173063:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa43b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+17:18223:x1875503571778944:12345-192.168.201.5@tcp:101:rm.0 Request processed in 647us (886us total) trans 0 rc 0/0 00000100:00100000:3.0:1788620007.173071:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1676 00000100:00100000:1.0:1788620007.177096:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788620007.179312:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571779072 00000100:00100000:2.0:1788620007.179342:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571779072 00000100:00100000:2.0:1788620007.179397:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1375 00000100:00100000:2.0:1788620007.179406:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df967480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:0+-99:2351:x1875503571779072:12345-192.168.201.5@tcp:8:rm.0 00010000:00080000:2.0:1788620007.179440:0:6697:0:(ldlm_lib.c:1444:target_handle_connect()) lustre-OST0000: connection from 92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp t0 exp 0000000000000000 cur 499 last 0 00000020:00000080:2.0:1788620007.179470:0:6697:0:(genops.c:1385:class_connect()) connect: client 92dda994-8428-47f5-a731-0b06f1aba2e4, cookie 0x1cd7fe53687371a9 00002000:00100000:2.0:1788620007.179983:0:6697:0:(ofd_obd.c:121:ofd_parse_connect_data()) lustre-OST0000: cli 92dda994-8428-47f5-a731-0b06f1aba2e4/ffff8e77c6b97800 ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 2113a03 ocd_grant: 83492864 ocd_index: 0 ocd_group 0 00002000:00100000:2.0:1788620007.180009:0:6697:0:(ofd_obd.c:219:ofd_parse_connect_data()) lustre-OST0000: cli supports cksum type f7, return f7 00002000:00080000:2.0:1788620007.181746:0:6697:0:(ofd_obd.c:385:ofd_obd_connect()) lustre-OST0000: get connection from MDS 0 00000100:00080000:2.0:1788620007.181792:0:6697:0:(import.c:65:import_set_state_nolock()) ffff8e77c6b91800 : changing import state from RECOVER to FULL 00000100:00100000:2.0:1788620007.181881:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df967480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2351:x1875503571779072:12345-192.168.201.5@tcp:8:rm.0 Request processed in 2472us (4791us total) trans 0 rc 0/0 00000100:00100000:2.0:1788620007.181895:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1375 00000100:00100000:0.0:1788620007.184594:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788620007.187410:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571779200 00000100:00100000:3.0:1788620007.187426:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571779200 00000100:00100000:3.0:1788620007.187524:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1376 00000100:00100000:3.0:1788620007.187536:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e7900984c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+6:0:x1875503571779200:12345-192.168.201.5@tcp:101:rm.0 00010000:00010000:3.0:1788620007.187556:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788620007.187580:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000a9a342/0x1cd7fe53687371b0 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1e05e expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788620007.187608:0:6696:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x3:0x0] initial LVB size: 1049076, mtime: 0x6a9c2c45, atime: 0x0, ctime: 0x6a9c2c45, blocks: 0x3 00010000:00010000:3.0:1788620007.187630:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x3:0x0].0x0 since=499s v=(0 0) 00010000:00010000:3.0:1788620007.187636:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788620007.187648:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000a9a342/0x1cd7fe53687371b0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e05e expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.187668:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000000a9a342/0x1cd7fe53687371b0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e05e expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.187681:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa07a00, rc 0) 00000100:00100000:3.0:1788620007.187755:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e7900984c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+7:0:x1875503571779200:12345-192.168.201.5@tcp:101:rm.0 Request processed in 222us (3168us total) trans 0 rc 0/0 00000100:00100000:3.0:1788620007.187762:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1376 00000100:00080000:0.0:1788620007.188322:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1788620007.199870:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788620007.200375:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571779456 00000100:00100000:0.0:1788620007.200385:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571779456 00000100:00100000:0.0:1788620007.200448:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1677 00000100:00100000:0.0:1788620007.200459:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+17:18223:x1875503571779456:12345-192.168.201.5@tcp:36:rm.0 00010000:00010000:0.0:1788620007.200517:0:5844:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:0.0:1788620007.200564:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe53687371a2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x7:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e057 expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.200586:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x7:0x0].0x0 (ffff8e78c3796400) 00010000:00010000:0.0:1788620007.200605:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d000/0x1cd7fe5368737186 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e050 expref: 16 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.200614:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78cfedfb00) 00010000:00010000:0.0:1788620007.200620:0:5844:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788620007.200641:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c200/0x1cd7fe53687371b7 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.200653:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c200/0x1cd7fe53687371b7 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.200663:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c200/0x1cd7fe53687371b7 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.200673:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777cc00/0x1cd7fe53687371be lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x32368eb3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.200679:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777cc00/0x1cd7fe53687371be lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x32368eb3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.200685:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777cc00/0x1cd7fe53687371be lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x32368eb3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.200776:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777ce00/0x1cd7fe53687371c5 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.200783:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777ce00/0x1cd7fe53687371c5 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.200790:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777ce00/0x1cd7fe53687371c5 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.203488:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1788620007.204237:0:5844:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].3, 64 off8384 00000004:00080000:1.0:1788620007.204254:0:5844:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 499873926962ns, next at 500873923007ns, rc = 0 00010000:00010000:1.0:1788620007.204401:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe53687371cc lrc: 3/0,1 mode: --/PW res: [0x200000401:0x7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.204421:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe53687371cc lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x7:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.204431:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe53687371cc lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x7:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.204444:0:5844:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe53687371cc lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x7:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.204451:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe53687371cc lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x7:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.204457:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe53687371cc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x7:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.204521:0:5844:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1788620007.204546:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777ce00/0x1cd7fe53687371c5 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.204553:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777ce00/0x1cd7fe53687371c5 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.204560:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777ce00/0x1cd7fe53687371c5 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.204565:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c777ce00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:1.0:1788620007.204570:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777ce00/0x1cd7fe53687371c5 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.204577:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777ce00/0x1cd7fe53687371c5 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.204589:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x7:0x0].0x0 (ffff8e78cfedfc00) 00010000:00010000:1.0:1788620007.204592:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1788620007.204596:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777ce00/0x1cd7fe53687371c5 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.204606:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777ce00/0x1cd7fe53687371c5 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.204624:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c200/0x1cd7fe53687371b7 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.204629:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c200/0x1cd7fe53687371b7 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.204635:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c200/0x1cd7fe53687371b7 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.204640:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c777c200 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:1.0:1788620007.204643:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c200/0x1cd7fe53687371b7 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.204648:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c200/0x1cd7fe53687371b7 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.204655:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78cfedf400) 00010000:00010000:1.0:1788620007.204657:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788620007.204660:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c200/0x1cd7fe53687371b7 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.204666:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c200/0x1cd7fe53687371b7 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1788620007.204680:0:5844:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000070f4bcfa reply state 000000002bcb98e5 transno 4294967892 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777cc00/0x1cd7fe53687371be lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x32368eb3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788620007.204690:0:5844:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788620007.204780:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+17:18223:x1875503571779456:12345-192.168.201.5@tcp:36:rm.0 Request processed in 4318us (4914us total) trans 4294967892 rc 0/0 00000100:00100000:1.0:1788620007.204789:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1677 00010000:00010000:0.0:1788620007.210555:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777cc00/0x1cd7fe53687371be lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x32368eb3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.210578:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777cc00/0x1cd7fe53687371be lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x32368eb3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.210586:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777cc00/0x1cd7fe53687371be lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x32368eb3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.210591:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c777cc00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788620007.210596:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777cc00/0x1cd7fe53687371be lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x32368eb3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.210604:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777cc00/0x1cd7fe53687371be lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x32368eb3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.210620:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x32368eb3 (ffff8e78cfedf800) 00010000:00010000:0.0:1788620007.210624:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788620007.210629:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777cc00/0x1cd7fe53687371be lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x32368eb3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.210636:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777cc00/0x1cd7fe53687371be lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x32368eb3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788620007.215510:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788620007.215619:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571779584 00000100:00100000:1.0:1788620007.215630:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571779584 00000100:00100000:1.0:1788620007.215680:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1678 00000100:00100000:1.0:1788620007.215690:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:18223:x1875503571779584:12345-192.168.201.5@tcp:101:rm.0 00010000:00010000:1.0:1788620007.215715:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788620007.215738:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe53687371d3 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1e065 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.215804:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687371da lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.215821:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687371da lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.215831:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687371da lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788620007.215869:0:5846:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:1.0:1788620007.215873:0:5846:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1788620007.215893:0:5846:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687371da lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.215935:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe53687371d3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e065 expref: 18 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.215954:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687371da lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e065 expref: 17 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.215968:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78c67bce00) 00010000:00010000:1.0:1788620007.215971:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1788620007.215975:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683da00, rc 0) 00000100:00100000:1.0:1788620007.216040:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+17:18223:x1875503571779584:12345-192.168.201.5@tcp:101:rm.0 Request processed in 351us (539us total) trans 0 rc 0/0 00000100:00100000:1.0:1788620007.216047:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1678 00010000:00010000:2.0:1788620007.216790:0:5833:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1788620007.216799:0:5833:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe53687371cc lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x7:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.216806:0:5833:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c683ce00 already unused, calling callback (ffffffffc17c24f0) 00010000:00010000:2.0:1788620007.216810:0:5833:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe53687371cc lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x7:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.216818:0:5833:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe53687371cc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x7:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.216831:0:5833:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x7:0x0].0x0 (ffff8e78cfedfc00) 00010000:00010000:2.0:1788620007.216834:0:5833:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788620007.216838:0:5833:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe53687371cc lrc: 1/0,0 mode: --/PW res: [0x200000401:0x7:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.216844:0:5833:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe53687371cc lrc: 0/0,0 mode: --/PW res: [0x200000401:0x7:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.216863:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788620007.216865:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788620007.216867:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788620007.220594:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788620007.220688:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571779712 00000100:00100000:3.0:1788620007.220699:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571779712 00000100:00100000:3.0:1788620007.220789:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1679 00000100:00100000:3.0:1788620007.220800:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df965500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+17:18223:x1875503571779712:12345-192.168.201.5@tcp:101:rm.0 00010000:00010000:3.0:1788620007.220821:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788620007.220845:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe53687371e1 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1e06c expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.220924:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06e00/0x1cd7fe53687371e8 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.220940:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06e00/0x1cd7fe53687371e8 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.220952:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06e00/0x1cd7fe53687371e8 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.220967:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06000/0x1cd7fe53687371ef lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xaa03ddaa bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.220974:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06000/0x1cd7fe53687371ef lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xaa03ddaa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.220980:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06000/0x1cd7fe53687371ef lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xaa03ddaa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.221236:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07000/0x1cd7fe53687371f6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.221245:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07000/0x1cd7fe53687371f6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.221252:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07000/0x1cd7fe53687371f6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788620007.221287:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x8:0x0]: returning size 2097152 00000004:00200000:3.0:1788620007.221292:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788620007.221310:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07000/0x1cd7fe53687371f6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x8:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1788620007.221441:0:5845:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x8:0x0] updating lvb size from disk: 0 -> 1048576 00000004:00010000:3.0:1788620007.221445:0:5845:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x8:0x0] updating lvb mtime from disk: 0 -> 1788619852 00000004:00010000:3.0:1788620007.221447:0:5845:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x8:0x0] updating lvb atime from disk: 0 -> 1788619852 00000004:00010000:3.0:1788620007.221449:0:5845:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x8:0x0] updating lvb ctime from disk: 0 -> 1788619852 00000004:00010000:3.0:1788620007.221451:0:5845:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x8:0x0] updating lvb blocks from disk: 18427282363563114435 -> 2054 00002000:00010000:3.0:1788620007.221456:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 1048576 00010000:00010000:3.0:1788620007.221462:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06e00/0x1cd7fe53687371e8 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.221469:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06e00/0x1cd7fe53687371e8 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.221555:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06e00/0x1cd7fe53687371e8 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.221562:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78dfa06e00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:3.0:1788620007.221566:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06e00/0x1cd7fe53687371e8 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.221574:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06e00/0x1cd7fe53687371e8 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.221586:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78c67bce00) 00010000:00010000:3.0:1788620007.221590:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788620007.221595:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06e00/0x1cd7fe53687371e8 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.221601:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06e00/0x1cd7fe53687371e8 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.221613:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06000/0x1cd7fe53687371ef lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xaa03ddaa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.221618:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06000/0x1cd7fe53687371ef lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xaa03ddaa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.221624:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06000/0x1cd7fe53687371ef lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xaa03ddaa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.221630:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78dfa06000 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:3.0:1788620007.221633:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06000/0x1cd7fe53687371ef lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xaa03ddaa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.221639:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06000/0x1cd7fe53687371ef lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xaa03ddaa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.221646:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xaa03ddaa (ffff8e78c3796500) 00010000:00010000:3.0:1788620007.221648:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788620007.221651:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06000/0x1cd7fe53687371ef lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xaa03ddaa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.221657:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06000/0x1cd7fe53687371ef lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xaa03ddaa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.221690:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe53687371e1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e06c expref: 19 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.221703:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07000/0x1cd7fe53687371f6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x8:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e06c expref: 18 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.221710:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8:0x0].0x0 (ffff8e78c3796a00) 00010000:00010000:3.0:1788620007.221712:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5b 00010000:00010000:3.0:1788620007.221715:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa07000, rc 0) 00000100:00100000:3.0:1788620007.221824:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df965500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+18:18223:x1875503571779712:12345-192.168.201.5@tcp:101:rm.0 Request processed in 1026us (1507us total) trans 0 rc 0/0 00000100:00100000:3.0:1788620007.221832:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1679 00000100:00100000:1.0:1788620007.225758:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788620007.225838:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571779840 00000100:00100000:3.0:1788620007.225852:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571779840 00000100:00100000:3.0:1788620007.226001:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1377 00000100:00100000:3.0:1788620007.226013:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df965880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2354:x1875503571779840:12345-192.168.201.5@tcp:101:rm.0 00010000:00010000:3.0:1788620007.226043:0:7654:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788620007.226064:0:7654:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000081ae252b/0x1cd7fe53687371fd lrc: 2/0,0 mode: --/PR res: [0x280000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1e073 expref: -99 pid: 7654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1788620007.226091:0:7654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x3:0x0] initial LVB size: 2097152, mtime: 0x6a9c2c4c, atime: 0x0, ctime: 0x6a9c2c4c, blocks: 0x807 00010000:00010000:3.0:1788620007.226111:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x3:0x0].0x0 since=499s v=(0 0) 00010000:00010000:3.0:1788620007.226115:0:7654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788620007.226126:0:7654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000081ae252b/0x1cd7fe53687371fd lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e073 expref: 8 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.226144:0:7654:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000081ae252b/0x1cd7fe53687371fd lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e073 expref: 8 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.226171:0:7654:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa07c00, rc 0) 00000100:00100000:3.0:1788620007.226240:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df965880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2354:x1875503571779840:12345-192.168.201.5@tcp:101:rm.0 Request processed in 230us (492us total) trans 0 rc 0/0 00000100:00100000:3.0:1788620007.226247:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1377 00000100:00100000:0.0:1788620007.235319:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788620007.235817:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571780096 00000100:00100000:3.0:1788620007.235828:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571780096 00000100:00100000:3.0:1788620007.236004:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1680 00000100:00100000:3.0:1788620007.236017:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a5ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+18:18223:x1875503571780096:12345-192.168.201.5@tcp:36:rm.0 00010000:00010000:3.0:1788620007.236143:0:5844:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:3.0:1788620007.236265:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07000/0x1cd7fe53687371f6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x8:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e06c expref: 18 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.236286:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8:0x0].0x0 (ffff8e78c3796a00) 00010000:00010000:3.0:1788620007.236305:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe53687371da lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e065 expref: 17 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.236314:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78c67bce00) 00010000:00010000:3.0:1788620007.236320:0:5844:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788620007.237402:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07600/0x1cd7fe5368737204 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.237422:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07600/0x1cd7fe5368737204 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.237432:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07600/0x1cd7fe5368737204 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.237447:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe536873720b lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xaa03ddaa bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.237454:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe536873720b lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xaa03ddaa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.237461:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe536873720b lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xaa03ddaa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.238807:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06600/0x1cd7fe5368737212 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.238825:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06600/0x1cd7fe5368737212 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.238835:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06600/0x1cd7fe5368737212 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1788620007.244780:0:5844:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].3, 64 off8384 00010000:00010000:3.0:1788620007.244881:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07800/0x1cd7fe5368737219 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.244899:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07800/0x1cd7fe5368737219 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x8:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.244923:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07800/0x1cd7fe5368737219 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x8:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.244932:0:5844:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07800/0x1cd7fe5368737219 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x8:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.244938:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07800/0x1cd7fe5368737219 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x8:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.244945:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07800/0x1cd7fe5368737219 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x8:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.244965:0:5844:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1788620007.244986:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06600/0x1cd7fe5368737212 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.244991:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06600/0x1cd7fe5368737212 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.244997:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06600/0x1cd7fe5368737212 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.245002:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78dfa06600 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:3.0:1788620007.245006:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06600/0x1cd7fe5368737212 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.245013:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06600/0x1cd7fe5368737212 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.245025:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8:0x0].0x0 (ffff8e78c5f50d00) 00010000:00010000:3.0:1788620007.245028:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1788620007.245404:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06600/0x1cd7fe5368737212 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.245412:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06600/0x1cd7fe5368737212 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.245433:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07600/0x1cd7fe5368737204 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.245439:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07600/0x1cd7fe5368737204 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.245444:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07600/0x1cd7fe5368737204 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.245449:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78dfa07600 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:3.0:1788620007.245451:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07600/0x1cd7fe5368737204 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.245456:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07600/0x1cd7fe5368737204 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.245464:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78c3796e00) 00010000:00010000:3.0:1788620007.245466:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788620007.245469:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07600/0x1cd7fe5368737204 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.245486:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07600/0x1cd7fe5368737204 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1788620007.245505:0:5844:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000007c2b4a6a reply state 00000000d29a02f5 transno 4294967893 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe536873720b lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xaa03ddaa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788620007.245515:0:5844:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788620007.245608:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a5ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+18:18223:x1875503571780096:12345-192.168.201.5@tcp:36:rm.0 Request processed in 9590us (10293us total) trans 4294967893 rc 0/0 00000100:00100000:3.0:1788620007.245617:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1680 00010000:00010000:2.0:1788620007.246012:0:5832:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1788620007.246023:0:5832:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07800/0x1cd7fe5368737219 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x8:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.246029:0:5832:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78dfa07800 already unused, calling callback (ffffffffc17c24f0) 00010000:00010000:2.0:1788620007.246042:0:5832:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07800/0x1cd7fe5368737219 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x8:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.246054:0:5832:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07800/0x1cd7fe5368737219 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x8:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.246064:0:5832:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8:0x0].0x0 (ffff8e78c5f50d00) 00010000:00010000:2.0:1788620007.246066:0:5832:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788620007.246070:0:5832:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07800/0x1cd7fe5368737219 lrc: 1/0,0 mode: --/PW res: [0x200000401:0x8:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.246075:0:5832:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07800/0x1cd7fe5368737219 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x8:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.246089:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788620007.246090:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788620007.246092:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788620007.248676:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe536873720b lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xaa03ddaa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.248693:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe536873720b lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xaa03ddaa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.248700:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe536873720b lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xaa03ddaa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.248709:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78dfa06400 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788620007.248714:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe536873720b lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xaa03ddaa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.248721:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe536873720b lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xaa03ddaa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.248736:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xaa03ddaa (ffff8e78c3796c00) 00010000:00010000:0.0:1788620007.248739:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788620007.248743:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe536873720b lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xaa03ddaa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.248749:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe536873720b lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xaa03ddaa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788620007.249950:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788620007.250021:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571780224 00000100:00100000:3.0:1788620007.250032:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571780224 00000100:00100000:3.0:1788620007.250097:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1681 00000100:00100000:3.0:1788620007.250107:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6a59f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+17:18223:x1875503571780224:12345-192.168.201.5@tcp:101:rm.0 00010000:00010000:3.0:1788620007.250130:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788620007.250151:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07200/0x1cd7fe5368737220 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1e07a expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.250306:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe5368737227 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.250320:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe5368737227 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.250428:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe5368737227 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788620007.250468:0:11397:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:3.0:1788620007.250473:0:11397:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1788620007.250498:0:11397:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe5368737227 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.250527:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07200/0x1cd7fe5368737220 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e07a expref: 19 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.250546:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe5368737227 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e07a expref: 18 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.250554:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78c5f50500) 00010000:00010000:3.0:1788620007.250557:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788620007.250561:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa06800, rc 0) 00000100:00100000:3.0:1788620007.250630:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6a59f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+18:18223:x1875503571780224:12345-192.168.201.5@tcp:101:rm.0 Request processed in 524us (688us total) trans 0 rc 0/0 00000100:00100000:3.0:1788620007.250637:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1681 00000100:00100000:2.0:1788620007.254088:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788620007.254558:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571780352 00000100:00100000:0.0:1788620007.254568:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571780352 00000100:00100000:0.0:1788620007.254678:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1682 00000100:00100000:0.0:1788620007.254690:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa43480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+18:18223:x1875503571780352:12345-192.168.201.5@tcp:101:rm.0 00010000:00010000:0.0:1788620007.254711:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788620007.254730:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d000/0x1cd7fe536873722e lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1e081 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.254797:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777da00/0x1cd7fe5368737235 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.254813:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777da00/0x1cd7fe5368737235 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.254828:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777da00/0x1cd7fe5368737235 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.254843:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d400/0x1cd7fe536873723c lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5fca220a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.254850:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d400/0x1cd7fe536873723c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5fca220a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.254857:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d400/0x1cd7fe536873723c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5fca220a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.254951:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d600/0x1cd7fe5368737243 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.254959:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d600/0x1cd7fe5368737243 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x9:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.254966:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d600/0x1cd7fe5368737243 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x9:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788620007.254999:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788620007.255017:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d600/0x1cd7fe5368737243 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x9:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1788620007.255051:0:5845:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x9:0x0] updating lvb size from disk: 0 -> 10 00000004:00010000:0.0:1788620007.255055:0:5845:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x9:0x0] updating lvb mtime from disk: 0 -> 1788619863 00000004:00010000:0.0:1788620007.255057:0:5845:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x9:0x0] updating lvb atime from disk: 0 -> 1788619862 00000004:00010000:0.0:1788620007.255059:0:5845:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x9:0x0] updating lvb ctime from disk: 0 -> 1788619863 00000004:00010000:0.0:1788620007.255061:0:5845:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x9:0x0] updating lvb blocks from disk: 18427282363563114435 -> 10 00002000:00010000:0.0:1788620007.255065:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 10 00010000:00010000:0.0:1788620007.255071:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777da00/0x1cd7fe5368737235 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.255077:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777da00/0x1cd7fe5368737235 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.255083:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777da00/0x1cd7fe5368737235 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.255088:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c777da00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788620007.255092:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777da00/0x1cd7fe5368737235 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.255100:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777da00/0x1cd7fe5368737235 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.255111:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78c5f50500) 00010000:00010000:0.0:1788620007.255114:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788620007.255118:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777da00/0x1cd7fe5368737235 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.255124:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777da00/0x1cd7fe5368737235 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.255135:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d400/0x1cd7fe536873723c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5fca220a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.255140:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d400/0x1cd7fe536873723c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5fca220a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.255144:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d400/0x1cd7fe536873723c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5fca220a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.255152:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c777d400 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788620007.255154:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d400/0x1cd7fe536873723c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5fca220a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.255171:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d400/0x1cd7fe536873723c lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5fca220a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.255178:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5fca220a (ffff8e78cfedf800) 00010000:00010000:0.0:1788620007.255180:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788620007.255182:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d400/0x1cd7fe536873723c lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5fca220a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.255187:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d400/0x1cd7fe536873723c lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5fca220a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.255227:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d000/0x1cd7fe536873722e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e081 expref: 20 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.255239:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d600/0x1cd7fe5368737243 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x9:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e081 expref: 19 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.255246:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x9:0x0].0x0 (ffff8e78cfedfb00) 00010000:00010000:0.0:1788620007.255248:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5b 00010000:00010000:0.0:1788620007.255251:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c777d600, rc 0) 00000100:00100000:0.0:1788620007.255385:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa43480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+19:18223:x1875503571780352:12345-192.168.201.5@tcp:101:rm.0 Request processed in 697us (1304us total) trans 0 rc 0/0 00000100:00100000:0.0:1788620007.255392:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1682 00000100:00100000:0.0:1788620007.259080:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788620007.259269:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571780608 00000100:00100000:3.0:1788620007.259279:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571780608 00000100:00100000:3.0:1788620007.259373:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1683 00000100:00100000:3.0:1788620007.259386:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+19:18223:x1875503571780608:12345-192.168.201.5@tcp:36:rm.0 00010000:00010000:3.0:1788620007.259445:0:5844:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:3.0:1788620007.259507:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d600/0x1cd7fe5368737243 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x9:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e081 expref: 19 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.259528:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x9:0x0].0x0 (ffff8e78cfedfb00) 00010000:00010000:3.0:1788620007.259546:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe5368737227 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e07a expref: 18 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.259555:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78c5f50500) 00010000:00010000:3.0:1788620007.259561:0:5844:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788620007.259583:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07000/0x1cd7fe536873724a lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.259597:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07000/0x1cd7fe536873724a lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.259607:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07000/0x1cd7fe536873724a lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.259619:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe5368737251 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5fca220a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.259625:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe5368737251 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5fca220a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.259632:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe5368737251 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5fca220a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.259726:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06000/0x1cd7fe5368737258 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.259735:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06000/0x1cd7fe5368737258 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.259741:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06000/0x1cd7fe5368737258 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.260858:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07600/0x1cd7fe536873725f lrc: 3/0,1 mode: --/PW res: [0x200000401:0x9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.260874:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07600/0x1cd7fe536873725f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x9:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.260884:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07600/0x1cd7fe536873725f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x9:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.260892:0:5844:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07600/0x1cd7fe536873725f lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x9:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.260899:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07600/0x1cd7fe536873725f lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x9:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.260906:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07600/0x1cd7fe536873725f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x9:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.260961:0:5844:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1788620007.260984:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06000/0x1cd7fe5368737258 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.260990:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06000/0x1cd7fe5368737258 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.260997:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06000/0x1cd7fe5368737258 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.261003:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78dfa06000 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:3.0:1788620007.261008:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06000/0x1cd7fe5368737258 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.261015:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06000/0x1cd7fe5368737258 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.261025:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x9:0x0].0x0 (ffff8e78c5f50c00) 00010000:00010000:3.0:1788620007.261028:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1788620007.261043:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06000/0x1cd7fe5368737258 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.261054:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06000/0x1cd7fe5368737258 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.261070:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07000/0x1cd7fe536873724a lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.261076:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07000/0x1cd7fe536873724a lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.261081:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07000/0x1cd7fe536873724a lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.261086:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78dfa07000 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:3.0:1788620007.261089:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07000/0x1cd7fe536873724a lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.261094:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07000/0x1cd7fe536873724a lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.261103:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78c5f50f00) 00010000:00010000:3.0:1788620007.261105:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788620007.261108:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07000/0x1cd7fe536873724a lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.261113:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07000/0x1cd7fe536873724a lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1788620007.261126:0:5844:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000006cb24813 reply state 00000000899da346 transno 4294967894 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe5368737251 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5fca220a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788620007.261135:0:5844:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788620007.261387:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+19:18223:x1875503571780608:12345-192.168.201.5@tcp:36:rm.0 Request processed in 2000us (2314us total) trans 4294967894 rc 0/0 00000100:00100000:3.0:1788620007.261396:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1683 00010000:00010000:2.0:1788620007.264350:0:5833:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1788620007.264364:0:5833:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07600/0x1cd7fe536873725f lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x9:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.264375:0:5833:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78dfa07600 already unused, calling callback (ffffffffc17c24f0) 00010000:00010000:2.0:1788620007.264380:0:5833:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07600/0x1cd7fe536873725f lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x9:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.264389:0:5833:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07600/0x1cd7fe536873725f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x9:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.264407:0:5833:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x9:0x0].0x0 (ffff8e78c5f50c00) 00010000:00010000:2.0:1788620007.264410:0:5833:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788620007.264414:0:5833:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07600/0x1cd7fe536873725f lrc: 1/0,0 mode: --/PW res: [0x200000401:0x9:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.264425:0:5833:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07600/0x1cd7fe536873725f lrc: 0/0,0 mode: --/PW res: [0x200000401:0x9:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.264522:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788620007.264525:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788620007.264527:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788620007.265831:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788620007.265891:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571780736 00000100:00100000:3.0:1788620007.265967:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571780736 00000100:00100000:3.0:1788620007.266074:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1684 00000100:00100000:3.0:1788620007.266086:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa43100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+19:18223:x1875503571780736:12345-192.168.201.5@tcp:101:rm.0 00010000:00010000:3.0:1788620007.266110:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788620007.266131:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07200/0x1cd7fe5368737266 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1e088 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.266218:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06600/0x1cd7fe536873726d lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.266232:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06600/0x1cd7fe536873726d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.266243:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06600/0x1cd7fe536873726d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788620007.266280:0:11397:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:3.0:1788620007.266284:0:11397:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1788620007.266303:0:11397:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06600/0x1cd7fe536873726d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.266360:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07200/0x1cd7fe5368737266 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e088 expref: 21 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.266377:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06600/0x1cd7fe536873726d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e088 expref: 20 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.266390:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78c5f50700) 00010000:00010000:3.0:1788620007.266393:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788620007.266396:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa06600, rc 0) 00000100:00100000:3.0:1788620007.266453:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa43100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+20:18223:x1875503571780736:12345-192.168.201.5@tcp:101:rm.0 Request processed in 370us (629us total) trans 0 rc 0/0 00000100:00100000:3.0:1788620007.266469:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1684 00010000:00010000:0.0:1788620007.267443:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe5368737251 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5fca220a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.267456:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe5368737251 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5fca220a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.267463:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe5368737251 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5fca220a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.267469:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78dfa07e00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788620007.267483:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe5368737251 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5fca220a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.267492:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe5368737251 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5fca220a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.267508:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5fca220a (ffff8e78c5f50b00) 00010000:00010000:0.0:1788620007.267511:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788620007.267515:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe5368737251 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5fca220a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.267522:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07e00/0x1cd7fe5368737251 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5fca220a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788620007.270117:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788620007.270415:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571780864 00000100:00100000:3.0:1788620007.270427:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571780864 00000100:00100000:3.0:1788620007.270491:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1685 00000100:00100000:3.0:1788620007.270503:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df964e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+19:18223:x1875503571780864:12345-192.168.201.5@tcp:101:rm.0 00010000:00010000:3.0:1788620007.270525:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788620007.270545:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06e00/0x1cd7fe5368737274 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1e08f expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.270637:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06c00/0x1cd7fe536873727b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.270651:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06c00/0x1cd7fe536873727b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.270663:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06c00/0x1cd7fe536873727b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.270720:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe5368737282 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x584b25ba bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.270729:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe5368737282 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x584b25ba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.270739:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe5368737282 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x584b25ba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.270834:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07800/0x1cd7fe5368737289 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xa:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.270843:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07800/0x1cd7fe5368737289 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.270850:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07800/0x1cd7fe5368737289 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788620007.270886:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1788620007.270920:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07800/0x1cd7fe5368737289 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xa:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1788620007.270942:0:5845:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xa:0x0] updating lvb size from disk: 0 -> 500 00000004:00010000:3.0:1788620007.270946:0:5845:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xa:0x0] updating lvb mtime from disk: 0 -> 1788619870 00000004:00010000:3.0:1788620007.270948:0:5845:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xa:0x0] updating lvb atime from disk: 0 -> 1788619869 00000004:00010000:3.0:1788620007.270951:0:5845:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xa:0x0] updating lvb ctime from disk: 0 -> 1788619870 00000004:00010000:3.0:1788620007.270954:0:5845:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0xa:0x0] updating lvb blocks from disk: 18427282363563114435 -> 10 00002000:00010000:3.0:1788620007.270958:0:5845:0:(mdt_io.c:1550:mdt_lvb2reply()) size 500 00010000:00010000:3.0:1788620007.270964:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06c00/0x1cd7fe536873727b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.270971:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06c00/0x1cd7fe536873727b lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.270976:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06c00/0x1cd7fe536873727b lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.270985:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78dfa06c00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:3.0:1788620007.270989:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06c00/0x1cd7fe536873727b lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.270995:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06c00/0x1cd7fe536873727b lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.271007:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78c5f50700) 00010000:00010000:3.0:1788620007.271010:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788620007.271014:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06c00/0x1cd7fe536873727b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.271020:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06c00/0x1cd7fe536873727b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.271044:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe5368737282 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x584b25ba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.271051:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe5368737282 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x584b25ba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.271056:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe5368737282 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x584b25ba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.271061:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78dfa06400 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:3.0:1788620007.271063:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe5368737282 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x584b25ba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.271070:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe5368737282 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x584b25ba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.271082:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x584b25ba (ffff8e78c5f50400) 00010000:00010000:3.0:1788620007.271084:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788620007.271087:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe5368737282 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x584b25ba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.271093:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06400/0x1cd7fe5368737282 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x584b25ba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.271125:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06e00/0x1cd7fe5368737274 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e08f expref: 21 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.271139:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07800/0x1cd7fe5368737289 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xa:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e08f expref: 20 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.271147:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa:0x0].0x0 (ffff8e78c5f50a00) 00010000:00010000:3.0:1788620007.271149:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5b 00010000:00010000:3.0:1788620007.271151:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78dfa07800, rc 0) 00000100:00100000:3.0:1788620007.271715:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df964e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+20:18223:x1875503571780864:12345-192.168.201.5@tcp:101:rm.0 Request processed in 1212us (1607us total) trans 0 rc 0/0 00000100:00100000:3.0:1788620007.271724:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1685 00000100:00100000:2.0:1788620007.275664:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788620007.275749:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571781120 00000100:00100000:3.0:1788620007.275760:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571781120 00000100:00100000:3.0:1788620007.275855:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1686 00000100:00100000:3.0:1788620007.275867:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa41180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+20:18223:x1875503571781120:12345-192.168.201.5@tcp:36:rm.0 00010000:00010000:3.0:1788620007.275933:0:5844:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:3.0:1788620007.275981:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07800/0x1cd7fe5368737289 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e08f expref: 20 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.276007:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa:0x0].0x0 (ffff8e78c5f50a00) 00010000:00010000:3.0:1788620007.276028:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06600/0x1cd7fe536873726d lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e088 expref: 19 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.276050:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78c5f50700) 00010000:00010000:3.0:1788620007.276056:0:5844:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788620007.276075:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07000/0x1cd7fe5368737290 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.276088:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07000/0x1cd7fe5368737290 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.276098:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07000/0x1cd7fe5368737290 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.276110:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06000/0x1cd7fe5368737297 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x584b25ba bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.276117:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06000/0x1cd7fe5368737297 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x584b25ba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.276123:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06000/0x1cd7fe5368737297 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x584b25ba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.276237:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe536873729e lrc: 3/0,1 mode: --/EX res: [0x200000401:0xa:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.276247:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe536873729e lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xa:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.276258:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe536873729e lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xa:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.276770:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64800/0x1cd7fe53687372a5 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xa:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.276781:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64800/0x1cd7fe53687372a5 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.276789:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64800/0x1cd7fe53687372a5 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xa:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.276796:0:5844:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64800/0x1cd7fe53687372a5 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xa:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.276802:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64800/0x1cd7fe53687372a5 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xa:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.276809:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64800/0x1cd7fe53687372a5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xa:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.276828:0:5844:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1788620007.276846:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe536873729e lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xa:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.276853:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe536873729e lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xa:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.276859:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe536873729e lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xa:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.276864:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78dfa06800 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:3.0:1788620007.276869:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe536873729e lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xa:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.276877:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe536873729e lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xa:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.276886:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa:0x0].0x0 (ffff8e78c5f50200) 00010000:00010000:3.0:1788620007.276889:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1788620007.276892:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe536873729e lrc: 2/0,0 mode: --/EX res: [0x200000401:0xa:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.276898:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06800/0x1cd7fe536873729e lrc: 0/0,0 mode: --/EX res: [0x200000401:0xa:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.276913:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07000/0x1cd7fe5368737290 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.276919:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07000/0x1cd7fe5368737290 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.276924:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07000/0x1cd7fe5368737290 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.276929:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78dfa07000 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:3.0:1788620007.276931:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07000/0x1cd7fe5368737290 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.276937:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07000/0x1cd7fe5368737290 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.276944:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78c5f50f00) 00010000:00010000:3.0:1788620007.276947:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788620007.276950:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07000/0x1cd7fe5368737290 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.276956:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa07000/0x1cd7fe5368737290 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1788620007.276969:0:5844:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000f4680388 reply state 00000000d19b623f transno 4294967895 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06000/0x1cd7fe5368737297 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x584b25ba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788620007.276977:0:5844:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788620007.277055:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa41180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+20:18223:x1875503571781120:12345-192.168.201.5@tcp:36:rm.0 Request processed in 1188us (1398us total) trans 4294967895 rc 0/0 00000100:00100000:3.0:1788620007.277063:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1686 00010000:00010000:2.0:1788620007.277135:0:5832:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1788620007.277142:0:5832:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64800/0x1cd7fe53687372a5 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xa:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.277149:0:5832:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78e6f64800 already unused, calling callback (ffffffffc17c24f0) 00010000:00010000:2.0:1788620007.277153:0:5832:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64800/0x1cd7fe53687372a5 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xa:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.277159:0:5832:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64800/0x1cd7fe53687372a5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xa:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.277166:0:5832:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa:0x0].0x0 (ffff8e78c5f50200) 00010000:00010000:2.0:1788620007.277169:0:5832:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1788620007.277172:0:5832:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64800/0x1cd7fe53687372a5 lrc: 1/0,0 mode: --/PW res: [0x200000401:0xa:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.277177:0:5832:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64800/0x1cd7fe53687372a5 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xa:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.277190:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788620007.277191:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788620007.277193:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788620007.283702:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788620007.283803:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571781248 00000100:00100000:0.0:1788620007.283813:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571781248 00000100:00100000:0.0:1788620007.283872:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1687 00000100:00100000:0.0:1788620007.283883:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df967b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+20:18223:x1875503571781248:12345-192.168.201.5@tcp:101:rm.0 00010000:00010000:0.0:1788620007.283909:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788620007.283930:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d400/0x1cd7fe53687372ac lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1e096 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.283999:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777da00/0x1cd7fe53687372b3 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.284015:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777da00/0x1cd7fe53687372b3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.284026:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777da00/0x1cd7fe53687372b3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788620007.284074:0:11397:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:0.0:1788620007.284078:0:11397:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1788620007.284097:0:11397:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777da00/0x1cd7fe53687372b3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.284122:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d400/0x1cd7fe53687372ac lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e096 expref: 22 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.284142:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777da00/0x1cd7fe53687372b3 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e096 expref: 21 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.284151:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78cfedf800) 00010000:00010000:0.0:1788620007.284154:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788620007.284171:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c777da00, rc 0) 00000100:00100000:0.0:1788620007.284279:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df967b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+21:18223:x1875503571781248:12345-192.168.201.5@tcp:101:rm.0 Request processed in 396us (585us total) trans 0 rc 0/0 00000100:00100000:0.0:1788620007.284287:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1687 00010000:00010000:3.0:1788620007.284649:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06000/0x1cd7fe5368737297 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x584b25ba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.284659:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06000/0x1cd7fe5368737297 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x584b25ba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.284666:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06000/0x1cd7fe5368737297 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x584b25ba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.284817:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78dfa06000 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:3.0:1788620007.284822:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06000/0x1cd7fe5368737297 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x584b25ba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.284830:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06000/0x1cd7fe5368737297 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x584b25ba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.284842:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x584b25ba (ffff8e78c5f50500) 00010000:00010000:3.0:1788620007.284845:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788620007.284849:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06000/0x1cd7fe5368737297 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x584b25ba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.284855:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78dfa06000/0x1cd7fe5368737297 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x584b25ba bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788620007.288965:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788620007.289029:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571781504 00000100:00100000:3.0:1788620007.289047:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571781504 00000100:00100000:3.0:1788620007.289105:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1688 00000100:00100000:3.0:1788620007.289115:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69ff100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+20:18223:x1875503571781504:12345-192.168.201.5@tcp:36:rm.0 00010000:00010000:3.0:1788620007.289180:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 4 locks, starting at 0 00010000:00010000:3.0:1788620007.289237:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e7902044e00/0x1cd7fe5368731cfa lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb:0x0].0x0 bits 0x40/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a197a2 expref: 20 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.289258:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e7902044000/0x1cd7fe5368731ce5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a19794 expref: 19 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.289269:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e7902045600/0x1cd7fe5368731c8a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb:0x0].0x0 bits 0x19/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000010000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a19771 expref: 18 pid: 5844 timeout: 469 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.289282:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb:0x0].0x0 (ffff8e78c3796b00) 00010000:00010000:3.0:1788620007.289296:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777da00/0x1cd7fe53687372b3 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e096 expref: 17 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.289305:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78cfedf800) 00010000:00010000:3.0:1788620007.289311:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788620007.289675:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d800/0x1cd7fe53687372ba lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.289690:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d800/0x1cd7fe53687372ba lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.289701:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d800/0x1cd7fe53687372ba lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.289714:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c600/0x1cd7fe53687372c1 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x248233be bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.289721:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c600/0x1cd7fe53687372c1 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x248233be bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.289727:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c600/0x1cd7fe53687372c1 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x248233be bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.289821:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c800/0x1cd7fe53687372c8 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xb:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.289831:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c800/0x1cd7fe53687372c8 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.289837:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c800/0x1cd7fe53687372c8 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.290354:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c000/0x1cd7fe53687372cf lrc: 3/0,1 mode: --/PW res: [0x200000401:0xb:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.290365:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c000/0x1cd7fe53687372cf lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.290376:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c000/0x1cd7fe53687372cf lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.290383:0:5845:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c000/0x1cd7fe53687372cf lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xb:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.290389:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c000/0x1cd7fe53687372cf lrc: 4/0,1 mode: PW/PW res: [0x200000401:0xb:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.290394:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c000/0x1cd7fe53687372cf lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.290409:0:5845:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1788620007.290428:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c800/0x1cd7fe53687372c8 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.290435:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c800/0x1cd7fe53687372c8 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.290441:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c800/0x1cd7fe53687372c8 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.290446:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c777c800 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788620007.290451:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c800/0x1cd7fe53687372c8 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.290457:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c800/0x1cd7fe53687372c8 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.290467:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb:0x0].0x0 (ffff8e78cfedff00) 00010000:00010000:0.0:1788620007.290470:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1788620007.290484:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c800/0x1cd7fe53687372c8 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.290491:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c800/0x1cd7fe53687372c8 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.290504:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d800/0x1cd7fe53687372ba lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.290509:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d800/0x1cd7fe53687372ba lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.290514:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d800/0x1cd7fe53687372ba lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.290519:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c777d800 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788620007.290521:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d800/0x1cd7fe53687372ba lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.290527:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d800/0x1cd7fe53687372ba lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.290533:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78c5f50700) 00010000:00010000:0.0:1788620007.290536:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788620007.290538:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d800/0x1cd7fe53687372ba lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.290543:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d800/0x1cd7fe53687372ba lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1788620007.290556:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000da8256f9 reply state 000000005a1e017c transno 4294967896 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c600/0x1cd7fe53687372c1 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x248233be bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788620007.290569:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788620007.290651:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69ff100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+18:18223:x1875503571781504:12345-192.168.201.5@tcp:36:rm.0 Request processed in 1536us (1695us total) trans 4294967896 rc 0/0 00000100:00100000:0.0:1788620007.290660:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1688 00010000:00010000:1.0:1788620007.291103:0:5833:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1788620007.291111:0:5833:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c000/0x1cd7fe53687372cf lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xb:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.291117:0:5833:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c777c000 already unused, calling callback (ffffffffc17c24f0) 00010000:00010000:1.0:1788620007.291121:0:5833:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c000/0x1cd7fe53687372cf lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xb:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.291127:0:5833:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c000/0x1cd7fe53687372cf lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.291136:0:5833:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb:0x0].0x0 (ffff8e78cfedff00) 00010000:00010000:1.0:1788620007.291139:0:5833:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788620007.291142:0:5833:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c000/0x1cd7fe53687372cf lrc: 1/0,0 mode: --/PW res: [0x200000401:0xb:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.291147:0:5833:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c000/0x1cd7fe53687372cf lrc: 0/0,0 mode: --/PW res: [0x200000401:0xb:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.291178:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788620007.291180:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788620007.291182:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1788620007.296365:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c600/0x1cd7fe53687372c1 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x248233be bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.296382:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c600/0x1cd7fe53687372c1 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x248233be bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.296390:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c600/0x1cd7fe53687372c1 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x248233be bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.296396:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c777c600 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788620007.296401:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c600/0x1cd7fe53687372c1 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x248233be bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.296408:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c600/0x1cd7fe53687372c1 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x248233be bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.296424:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x248233be (ffff8e78cfedf000) 00010000:00010000:0.0:1788620007.296427:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788620007.296432:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c600/0x1cd7fe53687372c1 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x248233be bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.296438:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c600/0x1cd7fe53687372c1 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x248233be bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788620007.341612:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788620007.341711:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571781632 00000100:00100000:3.0:1788620007.341722:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571781632 00000100:00100000:3.0:1788620007.341779:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1689 00000100:00100000:3.0:1788620007.341790:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c69fdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+17:18232:x1875503571781632:12345-192.168.201.5@tcp:101:mkdir.0 00010000:00010000:3.0:1788620007.341812:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788620007.341835:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64a00/0x1cd7fe53687372d6 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1e09d expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.341900:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64e00/0x1cd7fe53687372dd lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.341931:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64e00/0x1cd7fe53687372dd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.341945:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64e00/0x1cd7fe53687372dd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788620007.341982:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:3.0:1788620007.341986:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1788620007.342004:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64e00/0x1cd7fe53687372dd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.342028:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64a00/0x1cd7fe53687372d6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e09d expref: 19 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.342059:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64e00/0x1cd7fe53687372dd lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e09d expref: 18 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.342067:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78c5f50500) 00010000:00010000:3.0:1788620007.342069:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788620007.342073:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e6f64e00, rc 0) 00000100:00100000:3.0:1788620007.342136:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c69fdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+18:18232:x1875503571781632:12345-192.168.201.5@tcp:101:mkdir.0 Request processed in 348us (534us total) trans 0 rc 0/0 00000100:00100000:3.0:1788620007.342142:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1689 00000100:00100000:2.0:1788620007.345810:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788620007.345927:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571781760 00000100:00100000:1.0:1788620007.345936:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571781760 00000100:00100000:1.0:1788620007.345995:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1690 00000100:00100000:1.0:1788620007.346006:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa41c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2355:x1875503571781760:12345-192.168.201.5@tcp:41:mkdir.0 00000004:00000024:1.0:1788620007.346058:0:9417:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 143523840 pending 0 free 2259156992 avail 2257059840 00000100:00100000:1.0:1788620007.346134:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa41c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+18:2355:x1875503571781760:12345-192.168.201.5@tcp:41:mkdir.0 Request processed in 129us (336us total) trans 0 rc 0/0 00000100:00100000:1.0:1788620007.346141:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1690 00000100:00100000:2.0:1788620007.348792:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788620007.348888:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571782016 00000100:00100000:0.0:1788620007.348899:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571782016 00000100:00100000:0.0:1788620007.348970:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1691 00000100:00100000:0.0:1788620007.348980:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa40a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+18:18232:x1875503571782016:12345-192.168.201.5@tcp:36:mkdir.0 00010000:00010000:0.0:1788620007.349056:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788620007.349084:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64e00/0x1cd7fe53687372dd lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e09d expref: 18 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.349103:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78c5f50500) 00010000:00010000:0.0:1788620007.349113:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788620007.349929:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d800/0x1cd7fe53687372e4 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.349947:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d800/0x1cd7fe53687372e4 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.349958:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d800/0x1cd7fe53687372e4 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.349971:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c800/0x1cd7fe53687372eb lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.349977:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c800/0x1cd7fe53687372eb lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.349984:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c800/0x1cd7fe53687372eb lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788620007.351442:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x11:0x0]: returning size 16384 00010000:00010000:0.0:1788620007.351458:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d800/0x1cd7fe53687372e4 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.351466:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d800/0x1cd7fe53687372e4 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.351487:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d800/0x1cd7fe53687372e4 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.351493:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c777d800 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788620007.351498:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d800/0x1cd7fe53687372e4 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.351506:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d800/0x1cd7fe53687372e4 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.351519:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78cfedf000) 00010000:00010000:0.0:1788620007.351521:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788620007.351525:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d800/0x1cd7fe53687372e4 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.351532:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d800/0x1cd7fe53687372e4 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1788620007.351549:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000033787ba7 reply state 00000000a2829f77 transno 4294967897 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c800/0x1cd7fe53687372eb lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788620007.351559:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788620007.351666:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa40a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+19:18232:x1875503571782016:12345-192.168.201.5@tcp:36:mkdir.0 Request processed in 2683us (2880us total) trans 4294967897 rc 0/0 00000100:00100000:0.0:1788620007.351674:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1691 00010000:00010000:2.0:1788620007.354198:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c800/0x1cd7fe53687372eb lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.354227:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c800/0x1cd7fe53687372eb lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.354235:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c800/0x1cd7fe53687372eb lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.354240:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c777c800 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:2.0:1788620007.354246:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c800/0x1cd7fe53687372eb lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.354254:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c800/0x1cd7fe53687372eb lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.354270:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x792b1d9 (ffff8e78cfedf200) 00010000:00010000:2.0:1788620007.354273:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788620007.354278:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c800/0x1cd7fe53687372eb lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.354284:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c800/0x1cd7fe53687372eb lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788620007.366626:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788620007.366733:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571782144 00000100:00100000:0.0:1788620007.366743:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571782144 00000100:00100000:0.0:1788620007.366806:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1692 00000100:00100000:0.0:1788620007.366816:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa41f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+18:18233:x1875503571782144:12345-192.168.201.5@tcp:101:chmod.0 00010000:00010000:0.0:1788620007.366844:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788620007.366865:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d800/0x1cd7fe53687372f2 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1e0a4 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.366945:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d400/0x1cd7fe53687372f9 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.366960:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d400/0x1cd7fe53687372f9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.366972:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d400/0x1cd7fe53687372f9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788620007.367011:0:11397:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:0.0:1788620007.367014:0:11397:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1788620007.367037:0:11397:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d400/0x1cd7fe53687372f9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.367061:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d800/0x1cd7fe53687372f2 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0a4 expref: 20 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.367079:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d400/0x1cd7fe53687372f9 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0a4 expref: 19 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.367086:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78cfedf000) 00010000:00010000:0.0:1788620007.367089:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1788620007.367092:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c777d400, rc 0) 00000100:00100000:0.0:1788620007.367163:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa41f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+19:18233:x1875503571782144:12345-192.168.201.5@tcp:101:chmod.0 Request processed in 339us (538us total) trans 0 rc 0/0 00000100:00100000:0.0:1788620007.367174:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1692 00000100:00100000:0.0:1788620007.370281:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788620007.370444:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571782400 00000100:00100000:2.0:1788620007.370453:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571782400 00000100:00100000:2.0:1788620007.370514:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1693 00000100:00100000:2.0:1788620007.370523:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f27480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+19:18233:x1875503571782400:12345-192.168.201.5@tcp:36:chmod.0 00010000:00010000:2.0:1788620007.370579:0:5846:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788620007.370605:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d400/0x1cd7fe53687372f9 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0a4 expref: 19 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.370628:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78cfedf000) 00010000:00010000:2.0:1788620007.370639:0:5846:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788620007.370661:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3fe00/0x1cd7fe5368737300 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.370675:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3fe00/0x1cd7fe5368737300 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.370684:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3fe00/0x1cd7fe5368737300 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1788620007.370987:0:5846:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000008c9d0d25 reply state 00000000c68aae86 transno 4294967898 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3fe00/0x1cd7fe5368737300 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788620007.370998:0:5846:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:2.0:1788620007.371006:0:5846:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000100:00100000:2.0:1788620007.371294:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f27480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+20:18233:x1875503571782400:12345-192.168.201.5@tcp:36:chmod.0 Request processed in 770us (1022us total) trans 4294967898 rc 0/0 00000100:00100000:2.0:1788620007.371302:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1693 00010000:00010000:0.0:1788620007.375682:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3fe00/0x1cd7fe5368737300 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.375700:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3fe00/0x1cd7fe5368737300 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.375707:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3fe00/0x1cd7fe5368737300 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.375712:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c6a3fe00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788620007.375717:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3fe00/0x1cd7fe5368737300 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.375724:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3fe00/0x1cd7fe5368737300 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.375739:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78c7cbb600) 00010000:00010000:0.0:1788620007.375743:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1788620007.375748:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3fe00/0x1cd7fe5368737300 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.375755:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3fe00/0x1cd7fe5368737300 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788620007.393264:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788620007.393418:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571782528 00000100:00100000:2.0:1788620007.393429:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571782528 00000100:00100000:2.0:1788620007.393498:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1694 00000100:00100000:2.0:1788620007.393508:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f25f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+19:18234:x1875503571782528:12345-192.168.201.5@tcp:101:chown.0 00010000:00010000:2.0:1788620007.393532:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788620007.393554:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3fe00/0x1cd7fe5368737307 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1e0ab expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.393617:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3fc00/0x1cd7fe536873730e lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.393632:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3fc00/0x1cd7fe536873730e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.393644:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3fc00/0x1cd7fe536873730e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788620007.393680:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:2.0:1788620007.393683:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1788620007.393701:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3fc00/0x1cd7fe536873730e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.393726:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3fe00/0x1cd7fe5368737307 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0ab expref: 21 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.393742:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3fc00/0x1cd7fe536873730e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0ab expref: 20 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.393750:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78c7cbba00) 00010000:00010000:2.0:1788620007.393753:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1788620007.393756:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6a3fc00, rc 0) 00000100:00100000:2.0:1788620007.393822:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f25f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+20:18234:x1875503571782528:12345-192.168.201.5@tcp:101:chown.0 Request processed in 314us (566us total) trans 0 rc 0/0 00000100:00100000:2.0:1788620007.393829:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1694 00000100:00100000:2.0:1788620007.397514:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788620007.397647:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571782656 00000100:00100000:1.0:1788620007.397657:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571782656 00000100:00100000:1.0:1788620007.397712:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1695 00000100:00100000:1.0:1788620007.397721:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78dfa41880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+20:18234:x1875503571782656:12345-192.168.201.5@tcp:101:chown.0 00010000:00010000:1.0:1788620007.397742:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788620007.397763:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368737315 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1e0b2 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.397824:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe536873731c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.397837:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe536873731c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.397847:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe536873731c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.397864:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368737323 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.397870:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368737323 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.397875:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368737323 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.397982:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe536873732a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x11:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.397990:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe536873732a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x11:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.397996:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe536873732a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x11:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788620007.398026:0:9417:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x11:0x0]: returning size 16384 00000004:00200000:1.0:1788620007.398408:0:9417:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1788620007.398432:0:9417:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe536873732a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x11:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.398444:0:9417:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe536873731c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.398450:0:9417:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe536873731c lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.398457:0:9417:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe536873731c lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.398465:0:9417:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c683d600 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:1.0:1788620007.398470:0:9417:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe536873731c lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.398488:0:9417:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe536873731c lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.398500:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78c7cbba00) 00010000:00010000:1.0:1788620007.398503:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788620007.398508:0:9417:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe536873731c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.398515:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe536873731c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.398528:0:9417:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368737323 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.398534:0:9417:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368737323 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.398539:0:9417:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368737323 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.398544:0:9417:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c683d000 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:1.0:1788620007.398546:0:9417:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368737323 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.398552:0:9417:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368737323 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.398562:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x792b1d9 (ffff8e78c67bc300) 00010000:00010000:1.0:1788620007.398564:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788620007.398567:0:9417:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368737323 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.398573:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368737323 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.398604:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368737315 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0b2 expref: 22 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.398617:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe536873732a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x11:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0b2 expref: 21 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.398624:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x11:0x0].0x0 (ffff8e78c67bc500) 00010000:00010000:1.0:1788620007.398626:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1788620007.398628:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683da00, rc 0) 00000100:00100000:1.0:1788620007.398698:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78dfa41880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+21:18234:x1875503571782656:12345-192.168.201.5@tcp:101:chown.0 Request processed in 977us (1191us total) trans 0 rc 0/0 00000100:00100000:1.0:1788620007.398705:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1695 00000100:00100000:1.0:1788620007.402621:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788620007.402856:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571782912 00000100:00100000:3.0:1788620007.402868:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571782912 00000100:00100000:3.0:1788620007.403655:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1696 00000100:00100000:3.0:1788620007.403670:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df965c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+21:18234:x1875503571782912:12345-192.168.201.5@tcp:36:chown.0 00010000:00010000:3.0:1788620007.403728:0:5844:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788620007.403755:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe536873732a lrc: 0/0,0 mode: --/PR res: [0x200000401:0x11:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0b2 expref: 21 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.403772:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x11:0x0].0x0 (ffff8e78c67bc500) 00010000:00010000:3.0:1788620007.403783:0:5844:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788620007.403809:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64a00/0x1cd7fe5368737331 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x11:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.403822:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64a00/0x1cd7fe5368737331 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x11:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.403831:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64a00/0x1cd7fe5368737331 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x11:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1788620007.445049:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967897 is committed 00000004:00080000:2.0:1788620007.445070:0:5867:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 4 committed 00000001:00080000:2.0:1788620007.445094:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967898 is committed 00000001:00080000:2.0:1788620007.445101:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967899 is committed 00000004:00080000:2.0:1788620007.445113:0:5867:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 2 committed 00000004:00010000:3.0:1788620007.445343:0:5844:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000715fd509 reply state 0000000047b313ee transno 4294967899 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64a00/0x1cd7fe5368737331 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x11:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788620007.445356:0:5844:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:3.0:1788620007.445371:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x11:0x0]: returning size 16384 00010000:00080000:3.0:1788620007.445483:0:5844:0:(ldlm_lib.c:3482:target_send_reply()) Schedule reply immediately 00000100:00100000:3.0:1788620007.445504:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df965c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:18234:x1875503571782912:12345-192.168.201.5@tcp:36:chown.0 Request processed in 41833us (42888us total) trans 4294967899 rc 0/0 00000100:00100000:3.0:1788620007.445513:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1696 00000100:00100000:3.0:1788620007.445659:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df19f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589300864:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1788620007.445716:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788620007.445788:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589300864 00000100:00100000:3.0:1788620007.445795:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589300864 00000100:00100000:3.0:1788620007.445860:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1378 00000100:00100000:3.0:1788620007.445872:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503589300864:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1788620007.445896:0:6696:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x280000400:2 count 1 00010000:00010000:3.0:1788620007.445917:0:6696:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c69aba9f/0x1cd7fe5368737338 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x2:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.445934:0:6696:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005cb4402d/0x1cd7fe5368737116 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e03b expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.445943:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x2:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1788620007.445947:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1788620007.445951:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c69aba9f/0x1cd7fe5368737338 lrc: 4/0,1 mode: --/PW res: [0x280000400:0x2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.445978:0:6696:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005cb4402d/0x1cd7fe5368737116 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e03b expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.445991:0:6696:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005cb4402d/0x1cd7fe5368737116 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e03b expref: 7 pid: 6696 timeout: 600 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788620007.446001:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df19df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6696:1875503589300992:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788620007.446373:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c56eb540 going to sleep for 11 seconds 00000100:00080000:3.0:1788620007.446386:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:3.0:1788620007.446403:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64a00/0x1cd7fe5368737331 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x11:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.446415:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64a00/0x1cd7fe5368737331 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x11:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.446422:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64a00/0x1cd7fe5368737331 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x11:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.446427:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78e6f64a00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:3.0:1788620007.446432:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64a00/0x1cd7fe5368737331 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x11:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.446440:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64a00/0x1cd7fe5368737331 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x11:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.446456:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x11:0x0].0x0 (ffff8e78c5f50f00) 00010000:00010000:3.0:1788620007.446459:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1788620007.446464:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64a00/0x1cd7fe5368737331 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x11:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.446470:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64a00/0x1cd7fe5368737331 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x11:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788620007.447467:0:3316:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1788620007.447493:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df965f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589301120:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1788620007.447545:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788620007.447585:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589301120 00000100:00100000:1.0:1788620007.447591:0:6697:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589301120 00000100:00100000:1.0:1788620007.447636:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1379 00000100:00100000:1.0:1788620007.447644:0:6697:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df964000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3316:x1875503589301120:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1788620007.447666:0:6697:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:3 count 1 00010000:00010000:1.0:1788620007.447688:0:6697:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b84bbc9/0x1cd7fe536873733f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.447707:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000000a9a342/0x1cd7fe53687371b0 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e05e expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.447715:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x3:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1788620007.447719:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788620007.447723:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003b84bbc9/0x1cd7fe536873733f lrc: 4/0,1 mode: --/PW res: [0x240000400:0x3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.447750:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000000a9a342/0x1cd7fe53687371b0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e05e expref: 6 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.447763:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000000a9a342/0x1cd7fe53687371b0 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e05e expref: 6 pid: 6696 timeout: 600 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788620007.447773:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df967480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589301248:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788620007.447815:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78cda62900 going to sleep for 11 seconds 00000100:00080000:1.0:1788620007.447829:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1788620007.463251:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788620007.463396:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df19df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6696:1875503589300992:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788620007.463426:0:6696:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000c69aba9f/0x1cd7fe5368737338 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788620007.463775:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571783040 00000100:00100000:1.0:1788620007.463782:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571783040 00010000:00010000:1.0:1788620007.463822:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000005cb4402d/0x1cd7fe5368737116 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e03b expref: 8 pid: 6696 timeout: 600 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788620007.463866:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 564 00000100:00100000:1.0:1788620007.463877:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64ae300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+8:2352:x1875503571783040:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1788620007.463894:0:13330:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788620007.463907:0:13330:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000005cb4402d/0x1cd7fe5368737116 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e03b expref: 8 pid: 6696 timeout: 600 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.463920:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000005cb4402d/0x1cd7fe5368737116 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e03b expref: 8 pid: 6696 timeout: 600 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.463946:0:13330:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005cb4402d/0x1cd7fe5368737116 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e03b expref: 8 pid: 6696 timeout: 600 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.463971:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788620007.463976:0:13330:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788620007.463981:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c69aba9f/0x1cd7fe5368737338 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.463992:0:13330:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c69aba9f/0x1cd7fe5368737338 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.464018:0:13330:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788620007.464072:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64ae300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2352:x1875503571783040:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 195us (830us total) trans 0 rc 0/0 00000100:00100000:1.0:1788620007.464079:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 564 00010000:00010000:1.0:1788620007.464113:0:6696:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000c69aba9f/0x1cd7fe5368737338 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.464126:0:6696:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c69aba9f/0x1cd7fe5368737338 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.464136:0:6696:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c69aba9f/0x1cd7fe5368737338 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.464144:0:6696:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000c69aba9f/0x1cd7fe5368737338 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.464163:0:6696:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1788620007.470602:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3316:x1875503589300864:12345-0@lo:6:osp-syn-1-0.0 Request processed in 24724us (24875us total) trans 4294967303 rc 0/0 00000100:00100000:1.0:1788620007.470617:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1378 00010000:00010000:1.0:1788620007.470664:0:5832:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1788620007.470672:0:5832:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000c69aba9f/0x1cd7fe5368737338 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.470684:0:5832:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78e6f65a00 already unused, calling callback (ffffffffc17c24f0) 00010000:00010000:1.0:1788620007.470689:0:5832:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000c69aba9f/0x1cd7fe5368737338 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.470700:0:5832:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000c69aba9f/0x1cd7fe5368737338 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.470718:0:5832:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000c69aba9f/0x1cd7fe5368737338 lrc: 1/0,0 mode: --/PW res: [0x280000400:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.470739:0:5832:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c69aba9f/0x1cd7fe5368737338 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.470761:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788620007.470763:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788620007.470765:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788620007.470808:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df967480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6697:1875503589301248:192.168.201.5@tcp:104: 00010000:00010000:1.0:1788620007.470832:0:6697:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000003b84bbc9/0x1cd7fe536873733f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1788620007.476821:0:3316:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff8e78df19f480/2, rc 0, transno 4294967303 00000100:00100000:3.0:1788620007.476881:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df19f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589300864:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1788620007.477435:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788620007.477521:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571783168 00000100:00100000:2.0:1788620007.477529:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571783168 00010000:00010000:2.0:1788620007.477568:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000000a9a342/0x1cd7fe53687371b0 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e05e expref: 7 pid: 6696 timeout: 600 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788620007.477607:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 565 00000100:00100000:2.0:1788620007.477616:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df967480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2353:x1875503571783168:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1788620007.477631:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1788620007.477637:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000000a9a342/0x1cd7fe53687371b0 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e05e expref: 7 pid: 6696 timeout: 600 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.477650:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000000a9a342/0x1cd7fe53687371b0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e05e expref: 7 pid: 6696 timeout: 600 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.477675:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000a9a342/0x1cd7fe53687371b0 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e05e expref: 7 pid: 6696 timeout: 600 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.477690:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788620007.477695:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1788620007.477700:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000003b84bbc9/0x1cd7fe536873733f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.477711:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003b84bbc9/0x1cd7fe536873733f lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.477762:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1788620007.477812:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df967480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2353:x1875503571783168:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 196us (381us total) trans 0 rc 0/0 00000100:00100000:2.0:1788620007.477818:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 565 00010000:00010000:1.0:1788620007.478036:0:6697:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000003b84bbc9/0x1cd7fe536873733f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.478047:0:6697:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003b84bbc9/0x1cd7fe536873733f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.478056:0:6697:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003b84bbc9/0x1cd7fe536873733f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.478064:0:6697:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003b84bbc9/0x1cd7fe536873733f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.478088:0:6697:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1788620007.478747:0:6697:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df964000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3316:x1875503589301120:12345-0@lo:6:osp-syn-0-0.0 Request processed in 31100us (31197us total) trans 4294967538 rc 0/0 00000100:00100000:1.0:1788620007.478757:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1379 00000004:00080000:1.0:1788620007.478806:0:3316:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff8e78df965f80/2, rc 0, transno 4294967538 00000100:00100000:1.0:1788620007.478811:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df965f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589301120:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1788620007.479852:0:5833:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1788620007.479863:0:5833:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003b84bbc9/0x1cd7fe536873733f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.479875:0:5833:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c683de00 already unused, calling callback (ffffffffc17c24f0) 00010000:00010000:2.0:1788620007.479879:0:5833:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003b84bbc9/0x1cd7fe536873733f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.479890:0:5833:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003b84bbc9/0x1cd7fe536873733f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.479910:0:5833:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003b84bbc9/0x1cd7fe536873733f lrc: 1/0,0 mode: --/PW res: [0x240000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.479917:0:5833:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b84bbc9/0x1cd7fe536873733f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.479939:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788620007.479941:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788620007.479943:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788620007.492937:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788620007.493027:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571783296 00000100:00100000:0.0:1788620007.493045:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571783296 00000100:00100000:0.0:1788620007.493134:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1697 00000100:00100000:0.0:1788620007.493146:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6001500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+11:18235:x1875503571783296:12345-192.168.201.5@tcp:101:touch.500 00010000:00010000:0.0:1788620007.493182:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788620007.493228:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d800/0x1cd7fe5368737346 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1e0b9 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.493303:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d200/0x1cd7fe536873734d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.493317:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d200/0x1cd7fe536873734d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.493368:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d200/0x1cd7fe536873734d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.493386:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777de00/0x1cd7fe5368737354 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.493395:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777de00/0x1cd7fe5368737354 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.493406:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777de00/0x1cd7fe5368737354 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.493737:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d400/0x1cd7fe536873735b lrc: 3/1,0 mode: --/PR res: [0x200000401:0x11:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.493747:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d400/0x1cd7fe536873735b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x11:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.493754:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d400/0x1cd7fe536873735b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x11:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788620007.493775:0:11397:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x11:0x0]: returning size 16384 00000004:00200000:0.0:1788620007.493779:0:11397:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1788620007.493798:0:11397:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d400/0x1cd7fe536873735b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x11:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.493809:0:11397:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d200/0x1cd7fe536873734d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.493815:0:11397:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d200/0x1cd7fe536873734d lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.493821:0:11397:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d200/0x1cd7fe536873734d lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.493827:0:11397:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c777d200 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788620007.493831:0:11397:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d200/0x1cd7fe536873734d lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.493839:0:11397:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d200/0x1cd7fe536873734d lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.493851:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78c7cbba00) 00010000:00010000:0.0:1788620007.493854:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788620007.493859:0:11397:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d200/0x1cd7fe536873734d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.493865:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d200/0x1cd7fe536873734d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.493877:0:11397:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777de00/0x1cd7fe5368737354 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.493882:0:11397:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777de00/0x1cd7fe5368737354 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.493891:0:11397:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777de00/0x1cd7fe5368737354 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.493895:0:11397:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c777de00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788620007.493898:0:11397:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777de00/0x1cd7fe5368737354 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.493915:0:11397:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777de00/0x1cd7fe5368737354 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.493921:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x792b1d9 (ffff8e78cfedf500) 00010000:00010000:0.0:1788620007.493923:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788620007.493926:0:11397:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777de00/0x1cd7fe5368737354 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.493931:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777de00/0x1cd7fe5368737354 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.493961:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d800/0x1cd7fe5368737346 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0b9 expref: 13 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.493974:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d400/0x1cd7fe536873735b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x11:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0b9 expref: 12 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.493982:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x11:0x0].0x0 (ffff8e78cfedfa00) 00010000:00010000:0.0:1788620007.493984:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:0.0:1788620007.493986:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c777d400, rc 0) 00000100:00100000:0.0:1788620007.494061:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6001500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+12:18235:x1875503571783296:12345-192.168.201.5@tcp:101:touch.500 Request processed in 917us (1136us total) trans 0 rc 0/0 00000100:00100000:0.0:1788620007.494071:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1697 00000100:00100000:2.0:1788620007.496642:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788620007.496793:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571783552 00000100:00100000:1.0:1788620007.496807:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571783552 00000100:00100000:1.0:1788620007.497011:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1698 00000100:00100000:1.0:1788620007.497022:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd93ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+12:18235:x1875503571783552:12345-192.168.201.5@tcp:101:touch.500 00010000:00010000:1.0:1788620007.497041:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788620007.497063:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368737362 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x11:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1e0c0 expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.497435:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368737369 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x11:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.497451:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368737369 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x11:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.497463:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368737369 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x11:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.497543:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368737370 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x11:0x0].0xf59c8806 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.497550:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368737370 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x11:0x0].0xf59c8806 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.497557:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368737370 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x11:0x0].0xf59c8806 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.501562:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368737377 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x12:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.501586:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368737377 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.501598:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368737377 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1788620007.501944:0:9417:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x12:0x0] updating lvb size from disk: 0 -> 0 00000004:00010000:3.0:1788620007.501963:0:9417:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x12:0x0] updating lvb mtime from disk: 0 -> 1788620007 00000004:00010000:3.0:1788620007.501967:0:9417:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x12:0x0] updating lvb atime from disk: 0 -> 1788620007 00000004:00010000:3.0:1788620007.501969:0:9417:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x12:0x0] updating lvb ctime from disk: 0 -> 1788620007 00000004:00010000:3.0:1788620007.501972:0:9417:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x12:0x0] updating lvb blocks from disk: 18427282363563114435 -> 2 00002000:00010000:3.0:1788620007.501977:0:9417:0:(mdt_io.c:1550:mdt_lvb2reply()) size 0 00010000:00010000:3.0:1788620007.501987:0:9417:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368737369 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x11:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.501997:0:9417:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368737369 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x11:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.502005:0:9417:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368737369 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x11:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.502011:0:9417:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c683ce00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:3.0:1788620007.502017:0:9417:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368737369 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x11:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.502026:0:9417:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368737369 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x11:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.502046:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x11:0x0].0x0 (ffff8e78cfedfa00) 00010000:00010000:3.0:1788620007.502050:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788620007.502056:0:9417:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368737369 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x11:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.502063:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ce00/0x1cd7fe5368737369 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x11:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1788620007.502155:0:9417:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000003625e1ce reply state 0000000012a54f42 transno 4294967900 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368737370 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x11:0x0].0xf59c8806 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788620007.502172:0:9417:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1788620007.502245:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe5368737362 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x11:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0c0 expref: 15 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.502264:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368737377 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0c0 expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.502272:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x12:0x0].0x0 (ffff8e78c67bc100) 00010000:00010000:3.0:1788620007.502274:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5b 00010000:00010000:3.0:1788620007.502277:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683ca00, rc 0) 00000100:00100000:3.0:1788620007.502412:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd93ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:18235:x1875503571783552:12345-192.168.201.5@tcp:101:touch.500 Request processed in 5390us (5781us total) trans 4294967900 rc 0/0 00000100:00100000:3.0:1788620007.502423:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1698 00010000:00010000:0.0:1788620007.508886:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368737370 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x11:0x0].0xf59c8806 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.508905:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368737370 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x11:0x0].0xf59c8806 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.508913:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368737370 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x11:0x0].0xf59c8806 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.508920:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c683c800 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788620007.508925:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368737370 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x11:0x0].0xf59c8806 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.508934:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368737370 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x11:0x0].0xf59c8806 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.508952:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x11:0x0].0xf59c8806 (ffff8e78c67bc300) 00010000:00010000:0.0:1788620007.508955:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788620007.508961:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368737370 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x11:0x0].0xf59c8806 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.508967:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe5368737370 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x11:0x0].0xf59c8806 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788620007.512795:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788620007.514390:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571783808 00000100:00100000:3.0:1788620007.514402:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571783808 00000100:00100000:3.0:1788620007.514458:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1699 00000100:00100000:3.0:1788620007.514468:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd93b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:18235:x1875503571783808:12345-192.168.201.5@tcp:36:touch.500 00010000:00010000:3.0:1788620007.514560:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64a00/0x1cd7fe5368737385 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x12:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.514577:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368737377 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0c0 expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.514586:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64a00/0x1cd7fe5368737385 lrc: 4/0,1 mode: --/PW res: [0x200000401:0x12:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.514617:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368737377 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0c0 expref: 14 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.514631:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368737377 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0c0 expref: 14 pid: 9417 timeout: 600 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788620007.514644:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df19df80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589301376:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788620007.514714:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c56eb000 going to sleep for 11 seconds 00000100:00100000:1.0:1788620007.520767:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788620007.520908:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df19df80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5844:1875503589301376:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788620007.520937:0:5844:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64a00/0x1cd7fe5368737385 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x12:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788620007.521035:0:13330:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571783936 00000100:00100000:2.0:1788620007.521043:0:13330:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571783936 00010000:00010000:2.0:1788620007.521080:0:13330:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368737377 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0c0 expref: 15 pid: 9417 timeout: 600 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788620007.521159:0:13330:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 566 00000100:00100000:2.0:1788620007.521170:0:13330:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6000000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571783936:12345-192.168.201.5@tcp:102:ldlm_bl.0 00010000:00010000:2.0:1788620007.521188:0:13330:0:(ldlm_lockd.c:1655:ldlm_handle_convert0()) ### server-side convert handler START ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368737377 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0c0 expref: 15 pid: 9417 timeout: 600 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.521198:0:13330:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368737377 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0c0 expref: 15 pid: 9417 timeout: 600 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.521228:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368737377 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x59/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0c0 expref: 15 pid: 9417 timeout: 600 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.521236:0:13330:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x12:0x0].0x0 (ffff8e78c67bc100) 00010000:00010000:2.0:1788620007.521239:0:13330:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5b 00010000:00010000:2.0:1788620007.521242:0:13330:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64a00/0x1cd7fe5368737385 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x12:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.521252:0:13330:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64a00/0x1cd7fe5368737385 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.521260:0:13330:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64a00/0x1cd7fe5368737385 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.521374:0:13330:0:(ldlm_lockd.c:1704:ldlm_handle_convert0()) ### server-side convert handler END, rc = 0 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368737377 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x59/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0c0 expref: 15 pid: 9417 timeout: 600 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1788620007.521429:0:13330:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6000000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2355:x1875503571783936:12345-192.168.201.5@tcp:102:ldlm_bl.0 Request processed in 261us (671us total) trans 0 rc 0/0 00000100:00100000:2.0:1788620007.521436:0:13330:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 566 00010000:00010000:1.0:1788620007.523010:0:5844:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64a00/0x1cd7fe5368737385 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.523024:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64a00/0x1cd7fe5368737385 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1788620007.523445:0:5844:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x12:0x0] updating lvb size from disk: 0 -> 0 00000004:00010000:1.0:1788620007.523451:0:5844:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x12:0x0] updating lvb mtime from disk: 1788620007 -> 1788620007 00000004:00010000:1.0:1788620007.523454:0:5844:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x12:0x0] updating lvb atime from disk: 1788620007 -> 1788620007 00000004:00010000:1.0:1788620007.523456:0:5844:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x12:0x0] updating lvb ctime from disk: 1788620007 -> 1788620007 00000004:00010000:1.0:1788620007.523458:0:5844:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x12:0x0] updating lvb blocks from disk: 2 -> 2 00000004:00010000:1.0:1788620007.523466:0:5844:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000c31ff23c reply state 000000007cb3748a transno 4294967901 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64a00/0x1cd7fe5368737385 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788620007.523538:0:5844:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788620007.523721:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd93b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:18235:x1875503571783808:12345-192.168.201.5@tcp:36:touch.500 Request processed in 9252us (10932us total) trans 4294967901 rc 0/0 00000100:00100000:1.0:1788620007.523729:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1699 00000100:00100000:0.0:1788620007.524500:0:3319:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1788620007.524519:0:3319:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6002300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3319:1875503589301504:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1788620007.524574:0:3319:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788620007.524729:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589301504 00000100:00100000:1.0:1788620007.524737:0:6699:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589301504 00000100:00100000:1.0:1788620007.524785:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 123 00000100:00100000:1.0:1788620007.524795:0:6699:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f24380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3319:x1875503589301504:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1788620007.524817:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 44236800 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1788620007.524864:0:6699:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f24380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3319:x1875503589301504:12345-0@lo:13:osp-pre-0-0.0 Request processed in 70us (288us total) trans 0 rc 0/0 00000100:00100000:1.0:1788620007.524870:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 123 00000100:00100000:1.0:1788620007.524912:0:3319:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6002300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3319:1875503589301504:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1788620007.529045:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788620007.529120:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571784192 00000100:00100000:0.0:1788620007.529131:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571784192 00000100:00100000:0.0:1788620007.529195:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 43 00000100:00100000:0.0:1788620007.529858:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:18235:x1875503571784192:12345-192.168.201.5@tcp:35:touch.500 00010000:00010000:2.0:1788620007.530176:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64a00/0x1cd7fe5368737385 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.530187:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64a00/0x1cd7fe5368737385 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.530195:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64a00/0x1cd7fe5368737385 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.530200:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78e6f64a00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:2.0:1788620007.530220:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64a00/0x1cd7fe5368737385 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.530229:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64a00/0x1cd7fe5368737385 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.530245:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x12:0x0].0x0 (ffff8e78c67bc100) 00010000:00010000:2.0:1788620007.530248:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788620007.530253:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64a00/0x1cd7fe5368737385 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x12:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.530259:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64a00/0x1cd7fe5368737385 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x12:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788620007.530896:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+16:18235:x1875503571784192:12345-192.168.201.5@tcp:35:touch.500 Request processed in 1688us (1868us total) trans 4294967902 rc 0/0 00000100:00100000:0.0:1788620007.530905:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 43 00000100:00100000:3.0:1788620007.549507:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788620007.549655:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571784320 00000100:00100000:2.0:1788620007.549666:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571784320 00000100:00100000:2.0:1788620007.549720:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1700 00000100:00100000:2.0:1788620007.549730:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+16:18236:x1875503571784320:12345-192.168.201.5@tcp:101:rm.0 00010000:00010000:2.0:1788620007.549754:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788620007.549775:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e200/0x1cd7fe536873738c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1e0c7 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.549839:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3f000/0x1cd7fe5368737393 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x11:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.549853:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3f000/0x1cd7fe5368737393 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x11:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.549863:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3f000/0x1cd7fe5368737393 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x11:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788620007.549896:0:5846:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x11:0x0]: returning size 16384 00000004:00200000:2.0:1788620007.549899:0:5846:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1788620007.550060:0:5846:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3f000/0x1cd7fe5368737393 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x11:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.550086:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e200/0x1cd7fe536873738c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0c7 expref: 18 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.550174:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3f000/0x1cd7fe5368737393 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x11:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0c7 expref: 17 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.550184:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x11:0x0].0x0 (ffff8e78cfedfa00) 00010000:00010000:2.0:1788620007.550187:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1788620007.550190:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6a3f000, rc 0) 00000100:00100000:2.0:1788620007.550442:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+17:18236:x1875503571784320:12345-192.168.201.5@tcp:101:rm.0 Request processed in 711us (943us total) trans 0 rc 0/0 00000100:00100000:2.0:1788620007.550452:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1700 00000100:00100000:1.0:1788620007.555446:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788620007.555702:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571784576 00000100:00100000:3.0:1788620007.555714:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571784576 00000100:00100000:3.0:1788620007.555774:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1701 00000100:00100000:3.0:1788620007.555785:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6002300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+17:18236:x1875503571784576:12345-192.168.201.5@tcp:101:rm.0 00010000:00010000:3.0:1788620007.555806:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788620007.555827:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65000/0x1cd7fe536873739a lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1e0ce expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788620007.556084:0:9417:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x11:0x0]: returning size 16384 00010000:00010000:3.0:1788620007.556798:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65000/0x1cd7fe536873739a lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0ce expref: 19 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.556815:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78c7cbba00) 00010000:00010000:3.0:1788620007.556818:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1788620007.556825:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65000/0x1cd7fe536873739a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0ce expref: 19 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.556837:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e6f65000, rc 0) 00000100:00100000:3.0:1788620007.556898:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6002300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+18:18236:x1875503571784576:12345-192.168.201.5@tcp:101:rm.0 Request processed in 1114us (1461us total) trans 4294967903 rc 301/301 00000100:00100000:3.0:1788620007.559308:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1701 00000100:00100000:3.0:1788620007.564665:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788620007.564791:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571784704 00000100:00100000:0.0:1788620007.564801:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571784704 00000100:00100000:0.0:1788620007.564860:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 44 00000100:00100000:0.0:1788620007.564871:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+18:18236:x1875503571784704:12345-192.168.201.5@tcp:37:rm.0 00000100:00100000:1.0:1788620007.572612:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+18:18236:x1875503571784704:12345-192.168.201.5@tcp:37:rm.0 Request processed in 7731us (7946us total) trans 0 rc 0/0 00000100:00100000:1.0:1788620007.572627:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 44 00000100:00100000:3.0:1788620007.576617:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788620007.576733:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571784960 00000100:00100000:2.0:1788620007.576742:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571784960 00000100:00100000:2.0:1788620007.576801:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 45 00000100:00100000:2.0:1788620007.576812:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78f1d9d850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+18:18236:x1875503571784960:12345-192.168.201.5@tcp:35:rm.0 00000100:00100000:2.0:1788620007.577423:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78f1d9d850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+19:18236:x1875503571784960:12345-192.168.201.5@tcp:35:rm.0 Request processed in 609us (812us total) trans 4294967904 rc 0/0 00000100:00100000:2.0:1788620007.577433:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 45 00000100:00100000:3.0:1788620007.580836:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788620007.581019:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571785216 00000100:00100000:1.0:1788620007.581038:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571785216 00000100:00100000:1.0:1788620007.581108:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1702 00000100:00100000:1.0:1788620007.581119:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+19:18236:x1875503571785216:12345-192.168.201.5@tcp:101:rm.0 00010000:00010000:1.0:1788620007.581143:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788620007.581174:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687373a8 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1e0d5 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.581593:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe53687373af lrc: 3/1,0 mode: --/CR res: [0x200000401:0x11:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.581609:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe53687373af lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x11:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.581619:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe53687373af lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x11:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788620007.581634:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x11:0x0]: returning size 16384 00010000:00010000:1.0:1788620007.581755:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687373a8 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0d5 expref: 22 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.581772:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe53687373af lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x11:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0d5 expref: 21 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.581781:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x11:0x0].0x0 (ffff8e78cfedfa00) 00010000:00010000:1.0:1788620007.581784:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:1.0:1788620007.581787:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683c200, rc 0) 00000100:00100000:1.0:1788620007.581956:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+21:18236:x1875503571785216:12345-192.168.201.5@tcp:101:rm.0 Request processed in 837us (1128us total) trans 4294967905 rc 0/0 00000100:00100000:1.0:1788620007.581964:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1702 00000100:00100000:2.0:1788620007.585918:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788620007.586386:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571785344 00000100:00100000:0.0:1788620007.586396:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571785344 00000100:00100000:0.0:1788620007.586458:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1703 00000100:00100000:0.0:1788620007.586595:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd93bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+21:18236:x1875503571785344:12345-192.168.201.5@tcp:41:rm.0 00000004:00000024:0.0:1788620007.586662:0:5845:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 143523840 pending 0 free 7716470784 avail 7712276480 00000100:00100000:0.0:1788620007.586739:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd93bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+21:18236:x1875503571785344:12345-192.168.201.5@tcp:41:rm.0 Request processed in 271us (842us total) trans 0 rc 0/0 00000100:00100000:0.0:1788620007.586746:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1703 00000100:00100000:1.0:1788620007.589806:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788620007.590360:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571785600 00000100:00100000:3.0:1788620007.590371:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571785600 00000100:00100000:3.0:1788620007.590433:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1704 00000100:00100000:3.0:1788620007.590444:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6003b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+21:18236:x1875503571785600:12345-192.168.201.5@tcp:36:rm.0 00010000:00010000:3.0:1788620007.590582:0:9417:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:3.0:1788620007.590628:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683ca00/0x1cd7fe5368737377 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x12:0x0].0x0 bits 0x59/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0c0 expref: 21 pid: 9417 timeout: 600 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.590647:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x12:0x0].0x0 (ffff8e78c67bc100) 00010000:00010000:3.0:1788620007.590663:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3f000/0x1cd7fe5368737393 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x11:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0c7 expref: 20 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.590672:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x11:0x0].0x0 (ffff8e78cfedfa00) 00010000:00010000:3.0:1788620007.590676:0:9417:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788620007.590692:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65000/0x1cd7fe53687373bd lrc: 3/0,1 mode: --/CW res: [0x200000401:0x11:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.590705:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65000/0x1cd7fe53687373bd lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x11:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.590716:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65000/0x1cd7fe53687373bd lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x11:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.590730:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64400/0x1cd7fe53687373c4 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x11:0x0].0xf59c8806 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.590738:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64400/0x1cd7fe53687373c4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x11:0x0].0xf59c8806 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.590744:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64400/0x1cd7fe53687373c4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x11:0x0].0xf59c8806 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.590843:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe53687373cb lrc: 3/0,1 mode: --/EX res: [0x200000401:0x12:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.590851:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe53687373cb lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x12:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.590858:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe53687373cb lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x12:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.592596:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64200/0x1cd7fe53687373d2 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x12:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.592613:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64200/0x1cd7fe53687373d2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.592622:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64200/0x1cd7fe53687373d2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.592629:0:9417:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64200/0x1cd7fe53687373d2 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.592639:0:9417:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64200/0x1cd7fe53687373d2 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.592645:0:9417:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64200/0x1cd7fe53687373d2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.592700:0:9417:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1788620007.592722:0:9417:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe53687373cb lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x12:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.592727:0:9417:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe53687373cb lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x12:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.592733:0:9417:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe53687373cb lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x12:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.592738:0:9417:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78e6f65e00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:3.0:1788620007.592742:0:9417:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe53687373cb lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x12:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.592748:0:9417:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe53687373cb lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x12:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.592759:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x12:0x0].0x0 (ffff8e78c5f50a00) 00010000:00010000:3.0:1788620007.592761:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1788620007.592765:0:9417:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe53687373cb lrc: 2/0,0 mode: --/EX res: [0x200000401:0x12:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.592771:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe53687373cb lrc: 0/0,0 mode: --/EX res: [0x200000401:0x12:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.592788:0:9417:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65000/0x1cd7fe53687373bd lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x11:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.592794:0:9417:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65000/0x1cd7fe53687373bd lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x11:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.592799:0:9417:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65000/0x1cd7fe53687373bd lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x11:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.592804:0:9417:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78e6f65000 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:3.0:1788620007.592807:0:9417:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65000/0x1cd7fe53687373bd lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x11:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.592813:0:9417:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65000/0x1cd7fe53687373bd lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x11:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.592820:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x11:0x0].0x0 (ffff8e78cfedfa00) 00010000:00010000:3.0:1788620007.592822:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788620007.592825:0:9417:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65000/0x1cd7fe53687373bd lrc: 2/0,0 mode: --/CW res: [0x200000401:0x11:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620007.592830:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65000/0x1cd7fe53687373bd lrc: 0/0,0 mode: --/CW res: [0x200000401:0x11:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1788620007.592839:0:9417:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000007b4295e6 reply state 000000004da29b0d transno 4294967906 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64400/0x1cd7fe53687373c4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x11:0x0].0xf59c8806 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788620007.592847:0:9417:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1788620007.592945:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6003b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+21:18236:x1875503571785600:12345-192.168.201.5@tcp:36:rm.0 Request processed in 2500us (3145us total) trans 4294967906 rc 0/0 00000100:00100000:3.0:1788620007.592955:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1704 00010000:00010000:0.0:1788620007.596701:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64400/0x1cd7fe53687373c4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x11:0x0].0xf59c8806 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.596717:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64400/0x1cd7fe53687373c4 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x11:0x0].0xf59c8806 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.596725:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64400/0x1cd7fe53687373c4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x11:0x0].0xf59c8806 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.596730:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78e6f64400 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788620007.596736:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64400/0x1cd7fe53687373c4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x11:0x0].0xf59c8806 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.596744:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64400/0x1cd7fe53687373c4 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x11:0x0].0xf59c8806 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.596762:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x11:0x0].0xf59c8806 (ffff8e78c5f50f00) 00010000:00010000:0.0:1788620007.596765:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788620007.596769:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64400/0x1cd7fe53687373c4 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x11:0x0].0xf59c8806 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620007.596776:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64400/0x1cd7fe53687373c4 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x11:0x0].0xf59c8806 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.598108:0:5832:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1788620007.598120:0:5832:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64200/0x1cd7fe53687373d2 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.598129:0:5832:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78e6f64200 already unused, calling callback (ffffffffc17c24f0) 00010000:00010000:1.0:1788620007.598132:0:5832:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64200/0x1cd7fe53687373d2 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.598141:0:5832:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64200/0x1cd7fe53687373d2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x12:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.598165:0:5832:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x12:0x0].0x0 (ffff8e78c5f50a00) 00010000:00010000:1.0:1788620007.598169:0:5832:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1788620007.598172:0:5832:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64200/0x1cd7fe53687373d2 lrc: 1/0,0 mode: --/PW res: [0x200000401:0x12:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.598180:0:5832:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64200/0x1cd7fe53687373d2 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x12:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.598214:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788620007.598216:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788620007.598218:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788620007.599608:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788620007.599733:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571785856 00000100:00100000:1.0:1788620007.599742:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571785856 00000100:00100000:1.0:1788620007.599798:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 46 00000100:00100000:1.0:1788620007.599809:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd93b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+20:18236:x1875503571785856:12345-192.168.201.5@tcp:35:rm.0 00000100:00100000:1.0:1788620007.600050:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd93b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+21:18236:x1875503571785856:12345-192.168.201.5@tcp:35:rm.0 Request processed in 242us (449us total) trans 4294967907 rc 0/0 00000100:00100000:1.0:1788620007.600058:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 46 00000100:00100000:3.0:1788620007.604713:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788620007.605072:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571786112 00000100:00100000:1.0:1788620007.605082:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571786112 00000100:00100000:1.0:1788620007.605152:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1705 00000100:00100000:1.0:1788620007.605162:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+21:18236:x1875503571786112:12345-192.168.201.5@tcp:36:rm.0 00010000:00010000:1.0:1788620007.605229:0:5844:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:1.0:1788620007.605256:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c200/0x1cd7fe53687373af lrc: 0/0,0 mode: --/CR res: [0x200000401:0x11:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0d5 expref: 21 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.605274:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d400/0x1cd7fe536873735b lrc: 0/0,0 mode: --/PR res: [0x200000401:0x11:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0b9 expref: 20 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.605286:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x11:0x0].0x0 (ffff8e78cfedfa00) 00010000:00010000:1.0:1788620007.605302:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3fc00/0x1cd7fe536873730e lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0ab expref: 19 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.605312:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78c7cbba00) 00010000:00010000:1.0:1788620007.605319:0:5844:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788620007.605372:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687373d9 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.605386:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687373d9 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.605395:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687373d9 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.605406:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe53687373e0 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.605413:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe53687373e0 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.605419:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe53687373e0 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.605665:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687373e7 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x11:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.605673:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687373e7 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.605680:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687373e7 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788620007.606198:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x11:0x0]: returning size 16384 00010000:00010000:1.0:1788620007.606228:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687373e7 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.606235:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687373e7 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.606241:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687373e7 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.606246:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c683c000 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:1.0:1788620007.606250:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687373e7 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.606256:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687373e7 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.606265:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x11:0x0].0x0 (ffff8e78c67bc700) 00010000:00010000:1.0:1788620007.606268:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1788620007.606272:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687373e7 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.606277:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c000/0x1cd7fe53687373e7 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.606296:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687373d9 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.606302:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687373d9 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.606307:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687373d9 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.606312:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c683d800 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:1.0:1788620007.606315:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687373d9 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.606320:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687373d9 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.606370:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78c67bc800) 00010000:00010000:1.0:1788620007.606374:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788620007.606377:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687373d9 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620007.606383:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d800/0x1cd7fe53687373d9 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1788620007.606394:0:5844:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000003455d40f reply state 0000000087d9426a transno 4294967908 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe53687373e0 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788620007.606402:0:5844:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788620007.606472:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+20:18236:x1875503571786112:12345-192.168.201.5@tcp:36:rm.0 Request processed in 1310us (1769us total) trans 4294967908 rc 0/0 00000100:00100000:1.0:1788620007.606480:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1705 00010000:00010000:2.0:1788620007.611895:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe53687373e0 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.611920:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe53687373e0 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.611929:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe53687373e0 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.611936:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c683c800 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:2.0:1788620007.611941:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe53687373e0 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.611950:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe53687373e0 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.611969:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x792b1d9 (ffff8e78c67bcc00) 00010000:00010000:2.0:1788620007.611972:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788620007.611978:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe53687373e0 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620007.611984:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683c800/0x1cd7fe53687373e0 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620008.227036:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788620009.252001:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788620010.274511:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788620011.107505:0:3317:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f75880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3317:1875503589301888:0@lo:400:kworker.0 00000100:00100000:3.0:1788620011.107577:0:3317:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788620011.107642:0:3317:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1788620011.107648:0:3317:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df19d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3317:1875503589301760:0@lo:400:kworker.0 00000100:00100000:3.0:1788620011.107674:0:3317:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788620011.107693:0:3317:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64a8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3317:1875503589302272:0@lo:400:kworker.0 00000100:00100000:3.0:1788620011.107716:0:3317:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788620011.107745:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589301888 00000100:00100000:3.0:1788620011.107752:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589301888 00000100:00100000:3.0:1788620011.107798:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589301760 00000100:00100000:3.0:1788620011.107801:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589301760 00000100:00100000:3.0:1788620011.107820:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 261 00000100:00100000:3.0:1788620011.107828:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c64a9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3317:x1875503589301888:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788620011.107878:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c64a9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3317:x1875503589301888:12345-0@lo:400:kworker.0 Request processed in 50us (301us total) trans 0 rc 0/0 00000100:00100000:3.0:1788620011.107885:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 261 00000100:00100000:3.0:1788620011.107901:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 262 00000100:00100000:3.0:1788620011.107907:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6533b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3317:x1875503589301760:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788620011.107933:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6533b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3317:x1875503589301760:12345-0@lo:400:kworker.0 Request processed in 26us (257us total) trans 0 rc 0/0 00000100:00100000:3.0:1788620011.107938:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 262 00000100:00100000:3.0:1788620011.107960:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589302272 00000100:00100000:3.0:1788620011.107963:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589302272 00000100:00100000:3.0:1788620011.107988:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 172 00000100:00100000:3.0:1788620011.107994:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6531880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+6:3317:x1875503589302272:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788620011.108024:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6531880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+6:3317:x1875503589302272:12345-0@lo:400:kworker.0 Request processed in 30us (307us total) trans 0 rc 0/0 00000100:00100000:3.0:1788620011.108029:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 172 00000100:00080000:3.0:1788620011.108045:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1788620011.108049:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788620011.108082:0:3317:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df19d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3317:1875503589301760:0@lo:400:kworker.0 00000100:00100000:3.0:1788620011.108092:0:3317:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64a8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3317:1875503589302272:0@lo:400:kworker.0 00000100:00100000:3.0:1788620011.108100:0:3317:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f75880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3317:1875503589301888:0@lo:400:kworker.0 00000100:00100000:0.0:1788620011.108351:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f77480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589302016:0@lo:400:kworker.0 00000100:00100000:0.0:1788620011.108381:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788620011.108412:0:3318:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:0.0:1788620011.108418:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78df19dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:3318:1875503589301632:0@lo:400:kworker.0 00000100:00100000:0.0:1788620011.108454:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788620011.108475:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f76a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3318:1875503589302144:0@lo:400:kworker.0 00000100:00100000:0.0:1788620011.108494:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788620011.108518:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589302144 00000100:00100000:0.0:1788620011.108521:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589302144 00000100:00100000:0.0:1788620011.108552:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 263 00000100:00100000:0.0:1788620011.108559:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f25880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3318:x1875503589302144:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788620011.108592:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f25880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3318:x1875503589302144:12345-0@lo:400:kworker.0 Request processed in 34us (97us total) trans 0 rc 0/0 00000100:00100000:0.0:1788620011.108598:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 263 00000100:00100000:0.0:1788620011.108632:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589301632 00000100:00100000:0.0:1788620011.108637:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589301632 00000100:00100000:0.0:1788620011.108658:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 337 00000100:00100000:0.0:1788620011.108664:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5ce5c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+19:3318:x1875503589301632:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788620011.108688:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5ce5c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+19:3318:x1875503589301632:12345-0@lo:400:kworker.0 Request processed in 25us (234us total) trans 0 rc 0/0 00000100:00100000:0.0:1788620011.108694:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 337 00000100:00080000:0.0:1788620011.108717:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1788620011.108730:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78df19dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:3318:1875503589301632:0@lo:400:kworker.0 00000100:00100000:0.0:1788620011.108739:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f76a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3318:1875503589302144:0@lo:400:kworker.0 00000100:00100000:3.0:1788620011.108877:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589302016 00000100:00100000:3.0:1788620011.108881:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589302016 00000100:00100000:3.0:1788620011.108916:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 173 00000100:00100000:3.0:1788620011.108922:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f27800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3318:x1875503589302016:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788620011.108950:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f27800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3318:x1875503589302016:12345-0@lo:400:kworker.0 Request processed in 29us (569us total) trans 0 rc 0/0 00000100:00100000:3.0:1788620011.108955:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 173 00000100:00080000:0.0:1788620011.108995:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1788620011.109011:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f77480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589302016:0@lo:400:kworker.0 00000001:02000400:3.0:1788620011.165750:0:15913:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity test 36a: MDS utime check (mknod, utime) ======= 10:53:29 (1788620009) 00010000:00010000:3.0:1788620011.299039:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788620011.320494:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788620011.320531:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571786240 00000100:00100000:3.0:1788620011.320543:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571786240 00000100:00100000:3.0:1788620011.320607:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1706 00000100:00100000:3.0:1788620011.320618:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6532d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+19:18609:x1875503571786240:12345-192.168.201.5@tcp:101:rm.0 00010000:00010000:3.0:1788620011.320641:0:9417:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788620011.320884:0:9417:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe53687373ee lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1e0dc expref: -99 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620011.320951:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65000/0x1cd7fe53687373f5 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620011.320967:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65000/0x1cd7fe53687373f5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620011.320979:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65000/0x1cd7fe53687373f5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788620011.321014:0:9417:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:3.0:1788620011.321018:0:9417:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1788620011.321036:0:9417:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65000/0x1cd7fe53687373f5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620011.321059:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe53687373ee lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0dc expref: 21 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620011.321074:0:9417:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65000/0x1cd7fe53687373f5 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0dc expref: 20 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620011.321082:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78c5f50400) 00010000:00010000:3.0:1788620011.321085:0:9417:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788620011.321089:0:9417:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e6f65000, rc 0) 00000100:00100000:3.0:1788620011.321157:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6532d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+20:18609:x1875503571786240:12345-192.168.201.5@tcp:101:rm.0 Request processed in 541us (671us total) trans 0 rc 0/0 00000100:00100000:3.0:1788620011.321163:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1706 00000100:00080000:0.0:1788620011.324581:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1788620011.326034:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788620011.326179:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571786368 00000100:00100000:0.0:1788620011.326189:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571786368 00000100:00100000:0.0:1788620011.326248:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1707 00000100:00100000:0.0:1788620011.326258:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd938a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+20:18609:x1875503571786368:12345-192.168.201.5@tcp:101:rm.0 00010000:00010000:0.0:1788620011.326277:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788620011.326298:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c400/0x1cd7fe53687373fc lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1e0e3 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.326558:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c800/0x1cd7fe5368737403 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.326574:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c800/0x1cd7fe5368737403 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.326587:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c800/0x1cd7fe5368737403 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.326601:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c600/0x1cd7fe536873740a lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.326608:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c600/0x1cd7fe536873740a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.326614:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c600/0x1cd7fe536873740a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.326663:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c800/0x1cd7fe5368737403 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.326671:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c800/0x1cd7fe5368737403 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.326678:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c800/0x1cd7fe5368737403 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.326684:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c777c800 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788620011.326688:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c800/0x1cd7fe5368737403 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.326696:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c800/0x1cd7fe5368737403 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.326707:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78c5f50400) 00010000:00010000:0.0:1788620011.326710:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788620011.326715:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c800/0x1cd7fe5368737403 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.326722:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c800/0x1cd7fe5368737403 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.326733:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c600/0x1cd7fe536873740a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.326739:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c600/0x1cd7fe536873740a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.326745:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c600/0x1cd7fe536873740a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.326751:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c777c600 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788620011.326753:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c600/0x1cd7fe536873740a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.326759:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c600/0x1cd7fe536873740a lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.326767:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xf7f6e39 (ffff8e78cfedf500) 00010000:00010000:0.0:1788620011.326769:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788620011.326772:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c600/0x1cd7fe536873740a lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.326779:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c600/0x1cd7fe536873740a lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.326808:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c400/0x1cd7fe53687373fc lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0e3 expref: 21 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.326819:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78c5f50400) 00010000:00010000:0.0:1788620011.326821:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788620011.326825:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c400/0x1cd7fe53687373fc lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0e3 expref: 21 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.326835:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c777c400, rc 0) 00000100:00100000:0.0:1788620011.326997:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd938a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+20:18609:x1875503571786368:12345-192.168.201.5@tcp:101:rm.0 Request processed in 739us (1386us total) trans 0 rc 301/301 00000100:00100000:0.0:1788620011.327004:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1707 00000100:00100000:3.0:1788620011.339355:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788620011.339392:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788620011.339470:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571786496 00000100:00100000:0.0:1788620011.339481:0:5838:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571786496 00000100:00100000:3.0:1788620011.339530:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788620011.339547:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 338 00000100:00100000:3.0:1788620011.339551:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571786752 00000100:00100000:3.0:1788620011.339555:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571786752 00000100:00100000:0.0:1788620011.339557:0:5838:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5ce7850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:597450a2-e90c-408c-b401-a0ce48eed204+10:2355:x1875503571786496:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:3.0:1788620011.339590:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571786624 00000100:00100000:3.0:1788620011.339593:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571786624 00000100:00100000:3.0:1788620011.339610:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 174 00000100:00100000:3.0:1788620011.339617:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6533480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571786752:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:1.0:1788620011.339690:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 175 00000100:00100000:1.0:1788620011.339699:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6532300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571786624:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:0.0:1788620011.343745:0:5838:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5ce7850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:597450a2-e90c-408c-b401-a0ce48eed204+10:2355:x1875503571786496:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 4178us (4418us total) trans 0 rc 0/0 00000100:00100000:0.0:1788620011.343759:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 338 00000100:00100000:1.0:1788620011.343815:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6532300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2355:x1875503571786624:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 4115us (4283us total) trans 0 rc 0/0 00000100:00100000:1.0:1788620011.343824:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 175 00000100:00080000:0.0:1788620011.343828:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1788620011.343833:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1788620011.344581:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6533480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+7:2355:x1875503571786752:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 4960us (5184us total) trans 0 rc 0/0 00000100:00100000:3.0:1788620011.344590:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 174 00000100:00100000:3.0:1788620011.349949:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788620011.350184:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571787008 00000100:00100000:0.0:1788620011.350194:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571787008 00000100:00100000:0.0:1788620011.350254:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1708 00000100:00100000:0.0:1788620011.350263:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6530700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+20:18611:x1875503571787008:12345-192.168.201.5@tcp:36:utime.0 00010000:00010000:0.0:1788620011.350312:0:5846:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788620011.350552:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65000/0x1cd7fe53687373f5 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0dc expref: 20 pid: 9417 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.350574:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78c5f50400) 00010000:00010000:0.0:1788620011.350585:0:5846:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788620011.352183:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368737411 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620011.352206:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368737411 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620011.352217:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368737411 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620011.352228:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368737418 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620011.352234:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368737418 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620011.352239:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368737418 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620011.353760:0:5846:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368737411 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620011.353773:0:5846:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368737411 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620011.353781:0:5846:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368737411 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620011.353787:0:5846:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c683cc00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:1.0:1788620011.353792:0:5846:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368737411 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620011.353800:0:5846:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368737411 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620011.353814:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78c67bc700) 00010000:00010000:1.0:1788620011.353818:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788620011.353823:0:5846:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368737411 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620011.353829:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368737411 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1788620011.353844:0:5846:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000111e03ce reply state 00000000872981f6 transno 4294967909 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368737418 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788620011.353852:0:5846:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788620011.353996:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6530700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+21:18611:x1875503571787008:12345-192.168.201.5@tcp:36:utime.0 Request processed in 3730us (4052us total) trans 4294967909 rc 0/0 00000100:00100000:1.0:1788620011.354005:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1708 00000100:00100000:2.0:1788620011.367676:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00010000:00010000:0.0:1788620011.367790:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368737418 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.367807:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368737418 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.367814:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368737418 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.367820:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c683de00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788620011.367824:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368737418 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.367832:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368737418 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.367848:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xf7f6e39 (ffff8e78c67bc900) 00010000:00010000:0.0:1788620011.367852:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788620011.367857:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368737418 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.367863:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683de00/0x1cd7fe5368737418 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788620011.368984:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571787136 00000100:00100000:3.0:1788620011.368996:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571787136 00000100:00100000:3.0:1788620011.369084:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1709 00000100:00100000:3.0:1788620011.369095:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd93a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+20:18611:x1875503571787136:12345-192.168.201.5@tcp:101:utime.0 00010000:00010000:3.0:1788620011.369117:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788620011.369135:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe536873741f lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1e0ea expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620011.369206:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64600/0x1cd7fe5368737426 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620011.369219:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64600/0x1cd7fe5368737426 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620011.369230:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64600/0x1cd7fe5368737426 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788620011.369263:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:3.0:1788620011.369267:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1788620011.369289:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64600/0x1cd7fe5368737426 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620011.369323:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe536873741f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0ea expref: 22 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620011.369362:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64600/0x1cd7fe5368737426 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0ea expref: 21 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620011.369370:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78c5f50900) 00010000:00010000:3.0:1788620011.369372:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788620011.369375:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e6f64600, rc 0) 00000100:00100000:3.0:1788620011.369578:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd93a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+21:18611:x1875503571787136:12345-192.168.201.5@tcp:101:utime.0 Request processed in 484us (1911us total) trans 0 rc 0/0 00000100:00100000:3.0:1788620011.369585:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1709 00000100:00100000:0.0:1788620011.376471:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788620011.376829:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571787264 00000100:00100000:1.0:1788620011.376839:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571787264 00000100:00100000:1.0:1788620011.376901:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1710 00000100:00100000:1.0:1788620011.376922:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f26d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+21:18611:x1875503571787264:12345-192.168.201.5@tcp:101:utime.0 00010000:00010000:1.0:1788620011.376942:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788620011.376962:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe536873742d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1e0f1 expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620011.377023:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368737434 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620011.377038:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368737434 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620011.377050:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368737434 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620011.377068:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe536873743b lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620011.377074:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe536873743b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620011.377080:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe536873743b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620011.377185:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe5368737442 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x13:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620011.377193:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe5368737442 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620011.377200:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe5368737442 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1788620011.377229:0:11397:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x13:0x0]: returning size 0 00000004:00200000:1.0:1788620011.377233:0:11397:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1788620011.377252:0:11397:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe5368737442 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620011.377263:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe5368737442 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620011.377271:0:11397:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368737434 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620011.377278:0:11397:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368737434 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620011.377290:0:11397:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368737434 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620011.377296:0:11397:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c683d600 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:1.0:1788620011.377301:0:11397:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368737434 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620011.377310:0:11397:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368737434 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620011.377321:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78c5f50900) 00010000:00010000:1.0:1788620011.377362:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788620011.377368:0:11397:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368737434 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620011.377374:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d600/0x1cd7fe5368737434 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620011.377387:0:11397:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe536873743b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620011.377394:0:11397:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe536873743b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620011.377400:0:11397:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe536873743b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620011.377877:0:11397:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c683d000 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:1.0:1788620011.377881:0:11397:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe536873743b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620011.377890:0:11397:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe536873743b lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620011.377900:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xf7f6e39 (ffff8e78c67bc700) 00010000:00010000:1.0:1788620011.378094:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788620011.378099:0:11397:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe536873743b lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620011.378106:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe536873743b lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620011.378152:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe536873742d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0f1 expref: 23 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620011.378166:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe5368737442 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0f1 expref: 22 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620011.378172:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x13:0x0].0x0 (ffff8e78c67bcb00) 00010000:00010000:1.0:1788620011.378174:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1788620011.378177:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683da00, rc 0) 00000100:00100000:1.0:1788620011.378286:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f26d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+22:18611:x1875503571787264:12345-192.168.201.5@tcp:101:utime.0 Request processed in 1366us (1824us total) trans 0 rc 0/0 00000100:00100000:1.0:1788620011.378294:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1710 00000100:00100000:2.0:1788620011.388788:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788620011.392527:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571787520 00000100:00100000:1.0:1788620011.392544:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571787520 00000100:00100000:1.0:1788620011.392687:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1711 00000100:00100000:1.0:1788620011.392698:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78fd93aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+22:18611:x1875503571787520:12345-192.168.201.5@tcp:36:utime.0 00010000:00010000:1.0:1788620011.392754:0:5846:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788620011.392779:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683da00/0x1cd7fe5368737442 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0f1 expref: 22 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620011.392798:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x13:0x0].0x0 (ffff8e78c67bcb00) 00010000:00010000:1.0:1788620011.392808:0:5846:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1788620011.392845:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368737449 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x13:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620011.392858:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368737449 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620011.392868:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368737449 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1788620011.393509:0:5846:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000009649d8bd reply state 000000008fafe36a transno 4294967910 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368737449 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788620011.393521:0:5846:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1788620011.393626:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78fd93aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+23:18611:x1875503571787520:12345-192.168.201.5@tcp:36:utime.0 Request processed in 929us (4847us total) trans 4294967910 rc 0/0 00000100:00100000:1.0:1788620011.393632:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1711 00000100:00100000:3.0:1788620011.396864:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788620011.396943:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571787648 00000100:00100000:1.0:1788620011.396953:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571787648 00000100:00100000:1.0:1788620011.397024:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1712 00000100:00100000:1.0:1788620011.397034:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f77480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+23:18611:x1875503571787648:12345-192.168.201.5@tcp:101:utime.0 00010000:00010000:1.0:1788620011.397056:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1788620011.397081:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368737450 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x13:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1e0f8 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.397119:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368737449 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.397129:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368737449 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.397145:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368737449 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620011.397158:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368737450 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x13:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0f8 expref: 24 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.401018:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c683cc00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788620011.401032:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368737449 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.401048:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368737449 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.401069:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x13:0x0].0x0 (ffff8e78c67bc700) 00010000:00010000:0.0:1788620011.401073:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788620011.401078:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368737449 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.401086:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683cc00/0x1cd7fe5368737449 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620011.401781:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368737450 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x13:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0f8 expref: 23 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620011.401800:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x13:0x0].0x0 (ffff8e78c67bc700) 00010000:00010000:1.0:1788620011.401803:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1788620011.401806:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c683d000, rc 0) 00000100:00100000:1.0:1788620011.402134:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f77480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+23:18611:x1875503571787648:12345-192.168.201.5@tcp:101:utime.0 Request processed in 5098us (5276us total) trans 0 rc 0/0 00000100:00100000:1.0:1788620011.402153:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1712 00000100:00100000:0.0:1788620011.424012:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788620011.424128:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571787776 00000100:00100000:1.0:1788620011.424148:0:5848:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571787776 00000100:00100000:1.0:1788620011.424216:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 47 00000100:00100000:1.0:1788620011.424226:0:5848:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f26300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+23:18611:x1875503571787776:12345-192.168.201.5@tcp:49:utime.0 00000100:00100000:1.0:1788620011.424921:0:5848:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f26300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+23:18611:x1875503571787776:12345-192.168.201.5@tcp:49:utime.0 Request processed in 174us (398us total) trans 0 rc 0/0 00000100:00100000:1.0:1788620011.424934:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 47 00000100:00100000:3.0:1788620011.435697:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788620011.435857:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571787904 00000100:00100000:1.0:1788620011.435868:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571787904 00000100:00100000:0.0:1788620011.436043:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1713 00000100:00100000:0.0:1788620011.436057:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f76a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+23:18611:x1875503571787904:12345-192.168.201.5@tcp:101:utime.0 00010000:00010000:0.0:1788620011.436078:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788620011.436098:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c600/0x1cd7fe5368737457 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1e0ff expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.436174:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c800/0x1cd7fe536873745e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.436187:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c800/0x1cd7fe536873745e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.436198:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c800/0x1cd7fe536873745e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.436212:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777da00/0x1cd7fe5368737465 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.436218:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777da00/0x1cd7fe5368737465 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.436224:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777da00/0x1cd7fe5368737465 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.436309:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c000/0x1cd7fe536873746c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x13:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.436318:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c000/0x1cd7fe536873746c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.436368:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c000/0x1cd7fe536873746c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788620011.436400:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x13:0x0]: returning size 0 00000004:00200000:0.0:1788620011.437676:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788620011.437720:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c000/0x1cd7fe536873746c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.437738:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c000/0x1cd7fe536873746c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.437748:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c800/0x1cd7fe536873745e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.437755:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c800/0x1cd7fe536873745e lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.437767:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c800/0x1cd7fe536873745e lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.437772:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c777c800 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788620011.437776:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c800/0x1cd7fe536873745e lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.437784:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c800/0x1cd7fe536873745e lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.437797:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78c5f50900) 00010000:00010000:0.0:1788620011.437800:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788620011.437805:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c800/0x1cd7fe536873745e lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.437811:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c800/0x1cd7fe536873745e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.437826:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777da00/0x1cd7fe5368737465 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.437832:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777da00/0x1cd7fe5368737465 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.437837:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777da00/0x1cd7fe5368737465 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.437843:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c777da00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788620011.437846:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777da00/0x1cd7fe5368737465 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.437855:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777da00/0x1cd7fe5368737465 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.437862:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xf7f6e39 (ffff8e78cfedf500) 00010000:00010000:0.0:1788620011.437864:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788620011.437866:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777da00/0x1cd7fe5368737465 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.437872:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777da00/0x1cd7fe5368737465 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.437920:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c600/0x1cd7fe5368737457 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0ff expref: 25 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.437934:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c000/0x1cd7fe536873746c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0ff expref: 24 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620011.438132:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x13:0x0].0x0 (ffff8e78c67bc700) 00010000:00010000:0.0:1788620011.438240:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788620011.438245:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c777c000, rc 0) 00000100:00100000:0.0:1788620011.438371:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f76a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+24:18611:x1875503571787904:12345-192.168.201.5@tcp:101:utime.0 Request processed in 2316us (2817us total) trans 0 rc 0/0 00000100:00100000:0.0:1788620011.438379:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1713 00000004:00080000:0.0:1788620011.618586:0:7993:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 504288256821ns, next at 505288254166ns, rc = 0 00000100:00100000:3.0:1788620011.618618:0:3317:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f25180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3317:1875503589302400:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1788620011.618697:0:3317:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788620011.627684:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589302400 00000100:00100000:0.0:1788620011.627699:0:6699:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589302400 00000100:00100000:0.0:1788620011.627852:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 124 00000100:00100000:0.0:1788620011.627862:0:6699:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f75880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3317:x1875503589302400:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1788620011.627903:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 44236800 pending 0 free 3858759680 avail 3856662528 00000100:00100000:0.0:1788620011.628064:0:6699:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f75880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3317:x1875503589302400:12345-0@lo:13:osp-pre-1-0.0 Request processed in 203us (9366us total) trans 0 rc 0/0 00000100:00100000:0.0:1788620011.628071:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 124 00000100:00100000:0.0:1788620011.628124:0:3317:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f25180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3317:1875503589302400:0@lo:13:osp-pre-1-0.0 00000001:00080000:2.0:1788620011.686256:0:6721:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967538 is committed 00000001:00100000:2.0:1788620011.686271:0:6721:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0000: committing for initial connect of 92dda994-8428-47f5-a731-0b06f1aba2e4 00000001:00080000:1.0:1788620011.707983:0:7657:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967303 is committed 00000001:00100000:1.0:1788620011.708002:0:7657:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0001: committing for initial connect of 92dda994-8428-47f5-a731-0b06f1aba2e4 00010000:00010000:1.0:1788620012.327425:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788620012.646105:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c64a8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589302528:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1788620012.646188:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788620012.646338:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589302528 00000100:00100000:0.0:1788620012.646348:0:6699:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589302528 00000100:00100000:0.0:1788620012.646408:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 125 00000100:00100000:0.0:1788620012.646418:0:6699:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589302528:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1788620012.646452:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 44236800 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1788620012.646503:0:6699:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3318:x1875503589302528:12345-0@lo:13:osp-pre-0-0.0 Request processed in 86us (314us total) trans 0 rc 0/0 00000100:00100000:0.0:1788620012.646511:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 125 00000100:00100000:0.0:1788620012.646549:0:3318:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967538 gen 2 00000004:00080000:0.0:1788620012.646555:0:3318:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff8e78df965f80, transno 4294967538 00000100:00100000:0.0:1788620012.646575:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c64a8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589302528:0@lo:13:osp-pre-0-0.0 00000004:00080000:1.0:1788620012.726647:0:5867:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 3 committed 00000001:00080000:1.0:1788620012.726672:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967902 is committed 00000001:00080000:1.0:1788620012.726681:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967910 is committed 00000100:00100000:1.0:1788620012.726766:0:3319:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6003480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3319:1875503589302656:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1788620012.726828:0:3319:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788620012.726896:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589302656 00000100:00100000:1.0:1788620012.726914:0:6698:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589302656 00000100:00100000:1.0:1788620012.726959:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1380 00000100:00100000:1.0:1788620012.726968:0:6698:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6001180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3319:x1875503589302656:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1788620012.727547:0:6698:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x280000400:3 count 1 00010000:00010000:1.0:1788620012.727579:0:6698:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002ab297cb/0x1cd7fe5368737473 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620012.727603:0:6698:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000081ae252b/0x1cd7fe53687371fd lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e073 expref: 5 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620012.727612:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x3:0x0].0x0 since=6s v=(0 0) 00010000:00010000:1.0:1788620012.727617:0:6698:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1788620012.727622:0:6698:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002ab297cb/0x1cd7fe5368737473 lrc: 4/0,1 mode: --/PW res: [0x280000400:0x3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620012.727651:0:6698:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000081ae252b/0x1cd7fe53687371fd lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e073 expref: 5 pid: 7654 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620012.727666:0:6698:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000081ae252b/0x1cd7fe53687371fd lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e073 expref: 5 pid: 7654 timeout: 605 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788620012.727679:0:6698:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6000380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6698:1875503589302784:192.168.201.5@tcp:104: 00000100:00100000:1.0:1788620012.727746:0:6698:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78c5d2d9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1788620012.738520:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788620012.739031:0:6698:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6000380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6698:1875503589302784:192.168.201.5@tcp:104: 00010000:00010000:2.0:1788620012.739055:0:6698:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000002ab297cb/0x1cd7fe5368737473 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788620012.740879:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571788032 00000100:00100000:0.0:1788620012.740894:0:5830:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571788032 00010000:00010000:0.0:1788620012.740938:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000081ae252b/0x1cd7fe53687371fd lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e073 expref: 6 pid: 7654 timeout: 605 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788620012.740973:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 567 00000100:00100000:0.0:1788620012.740983:0:5830:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19d880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+6:2352:x1875503571788032:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1788620012.740999:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788620012.741005:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000081ae252b/0x1cd7fe53687371fd lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e073 expref: 6 pid: 7654 timeout: 605 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620012.741018:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000081ae252b/0x1cd7fe53687371fd lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e073 expref: 6 pid: 7654 timeout: 605 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620012.741044:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000081ae252b/0x1cd7fe53687371fd lrc: 0/0,0 mode: --/PR res: [0x280000400:0x3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e073 expref: 6 pid: 7654 timeout: 605 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620012.741059:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788620012.741064:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1788620012.741070:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000002ab297cb/0x1cd7fe5368737473 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620012.741080:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002ab297cb/0x1cd7fe5368737473 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620012.741123:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1788620012.741141:0:6698:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000002ab297cb/0x1cd7fe5368737473 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620012.741154:0:6698:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002ab297cb/0x1cd7fe5368737473 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620012.741164:0:6698:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002ab297cb/0x1cd7fe5368737473 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620012.741171:0:6698:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000002ab297cb/0x1cd7fe5368737473 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620012.741196:0:6698:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1788620012.741362:0:5830:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19d880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2352:x1875503571788032:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 378us (2849us total) trans 0 rc 0/0 00000100:00100000:0.0:1788620012.741372:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 567 00000100:00100000:2.0:1788620012.741916:0:6698:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6001180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+6:3319:x1875503589302656:12345-0@lo:6:osp-syn-1-0.0 Request processed in 14947us (15086us total) trans 4294967304 rc 0/0 00000100:00100000:2.0:1788620012.741926:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1380 00010000:00010000:2.0:1788620012.741952:0:5832:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1788620012.741959:0:5832:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000002ab297cb/0x1cd7fe5368737473 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620012.741967:0:5832:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c683c200 already unused, calling callback (ffffffffc17c24f0) 00010000:00010000:2.0:1788620012.741972:0:5832:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000002ab297cb/0x1cd7fe5368737473 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620012.741991:0:5832:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000002ab297cb/0x1cd7fe5368737473 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620012.742004:0:5832:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000002ab297cb/0x1cd7fe5368737473 lrc: 1/0,0 mode: --/PW res: [0x280000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620012.742012:0:5832:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002ab297cb/0x1cd7fe5368737473 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6698 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620012.742029:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788620012.742031:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788620012.742033:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788620012.742153:0:3319:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967303 gen 2 00000004:00080000:0.0:1788620012.742161:0:3319:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff8e78df19f480, transno 4294967303 00000100:00100000:0.0:1788620012.742187:0:3319:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff8e77c6003480 x1875503589302656/t4294967304(4294967304) o6->lustre-OST0001-osc-MDT0000@0@lo:28/4 lens 544/432 e 0 to 0 dl 1788620028 ref 2 fl Rpc:RQU/204/0 rc 0/0 job:'osp-syn-1-0.0' uid:0 gid:0 projid:4294967295 00000004:00080000:0.0:1788620012.742197:0:3319:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff8e77c6003480/2, rc 0, transno 4294967304 00000100:00100000:0.0:1788620012.742202:0:3319:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6003480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3319:1875503589302656:0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1788620013.348182:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788620014.370524:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788620015.398719:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788620016.227643:0:3317:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f25c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3317:1875503589303040:0@lo:400:kworker.0 00000100:00100000:0.0:1788620016.227669:0:3319:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f26a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3319:1875503589303296:0@lo:400:kworker.0 00000100:00100000:1.0:1788620016.227749:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f25180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:3316:1875503589302912:0@lo:400:kworker.0 00000100:00100000:2.0:1788620016.227911:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f24e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3318:1875503589303168:0@lo:400:kworker.0 00000100:00100000:1.0:1788620016.237922:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788620016.237988:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f27b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3316:1875503589303424:0@lo:400:kworker.0 00000100:00100000:1.0:1788620016.238024:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788620016.238072:0:3316:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1788620016.238081:0:3316:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c5f26680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589303552:0@lo:400:kworker.0 00000100:00100000:1.0:1788620016.238108:0:3316:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788620016.239033:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589302912 00000100:00100000:1.0:1788620016.239045:0:5836:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589302912 00000100:00100000:1.0:1788620016.239115:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 339 00000100:00100000:1.0:1788620016.239124:0:5836:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5ce5050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+19:3316:x1875503589302912:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788620016.239452:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788620016.239522:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589303552 00000100:00100000:2.0:1788620016.239527:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589303552 00000100:00100000:2.0:1788620016.239569:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 176 00000100:00100000:2.0:1788620016.239589:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6002a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3316:x1875503589303552:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788620016.239656:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6002a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3316:x1875503589303552:12345-0@lo:400:kworker.0 Request processed in 79us (1546us total) trans 0 rc 0/0 00000100:00100000:2.0:1788620016.239663:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 176 00000100:00100000:2.0:1788620016.239898:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589303424 00000100:00100000:2.0:1788620016.239904:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589303424 00000100:00100000:2.0:1788620016.239932:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589303168 00000100:00100000:2.0:1788620016.239935:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589303168 00000100:00100000:2.0:1788620016.239954:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 264 00000100:00100000:2.0:1788620016.239960:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6001880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3316:x1875503589303424:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788620016.239996:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6001880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3316:x1875503589303424:12345-0@lo:400:kworker.0 Request processed in 36us (1971us total) trans 0 rc 0/0 00000100:00100000:2.0:1788620016.240002:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 264 00000100:00100000:2.0:1788620016.240013:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 265 00000100:00100000:2.0:1788620016.240019:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5cdb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3318:x1875503589303168:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788620016.240063:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5cdb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3318:x1875503589303168:12345-0@lo:400:kworker.0 Request processed in 32us (600us total) trans 0 rc 0/0 00000100:00100000:2.0:1788620016.240069:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 265 00000100:00100000:2.0:1788620016.240106:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f24e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3318:1875503589303168:0@lo:400:kworker.0 00000100:00100000:3.0:1788620016.240942:0:3317:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:3.0:1788620016.241011:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1788620016.241016:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1788620016.241045:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589303040 00000100:00100000:3.0:1788620016.241059:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589303040 00000100:00100000:3.0:1788620016.241094:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 266 00000100:00100000:3.0:1788620016.241102:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df19f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3317:x1875503589303040:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1788620016.241149:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df19f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3317:x1875503589303040:12345-0@lo:400:kworker.0 Request processed in 49us (210us total) trans 0 rc 0/0 00000100:00100000:3.0:1788620016.241156:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 266 00000100:00100000:3.0:1788620016.241190:0:3317:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f25c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3317:1875503589303040:0@lo:400:kworker.0 00000100:00100000:2.0:1788620016.241554:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f26680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3316:1875503589303552:0@lo:400:kworker.0 00000100:00100000:2.0:1788620016.241572:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f27b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3316:1875503589303424:0@lo:400:kworker.0 00000100:00100000:1.0:1788620016.241686:0:5836:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5ce5050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ad6fde96-7e4e-4f30-b71c-a421068ce9c0+19:3316:x1875503589302912:12345-0@lo:400:kworker.0 Request processed in 2561us (3770us total) trans 0 rc 0/0 00000100:00100000:1.0:1788620016.241695:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 339 00000100:00100000:3.0:1788620016.241737:0:3316:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f25180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ad6fde96-7e4e-4f30-b71c-a421068ce9c0:3316:1875503589302912:0@lo:400:kworker.0 00000100:00100000:0.0:1788620016.248638:0:3319:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788620016.249894:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589303296 00000100:00100000:1.0:1788620016.249908:0:6696:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589303296 00000100:00100000:1.0:1788620016.249979:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 177 00000100:00100000:1.0:1788620016.249990:0:6696:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f25180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3319:x1875503589303296:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788620016.250109:0:6696:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f25180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3319:x1875503589303296:12345-0@lo:400:kworker.0 Request processed in 122us (1478us total) trans 0 rc 0/0 00000100:00100000:1.0:1788620016.250116:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 177 00000100:00100000:0.0:1788620016.250309:0:3319:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c5f26a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3319:1875503589303296:0@lo:400:kworker.0 00000100:00080000:3.0:1788620016.251763:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:2.0:1788620016.418489:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788620016.466631:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788620016.466685:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788620016.466718:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788620016.466746:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788620016.466780:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571788160 00000100:00100000:1.0:1788620016.466791:0:5837:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571788160 00000100:00100000:1.0:1788620016.466845:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 340 00000100:00100000:1.0:1788620016.466855:0:5837:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c5ce6c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:597450a2-e90c-408c-b401-a0ce48eed204+10:2354:x1875503571788160:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:1.0:1788620016.466931:0:5837:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c5ce6c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:597450a2-e90c-408c-b401-a0ce48eed204+10:2354:x1875503571788160:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 77us (311us total) trans 0 rc 0/0 00000100:00100000:1.0:1788620016.466939:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 340 00000100:00100000:1.0:1788620016.466974:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571788416 00000100:00100000:1.0:1788620016.466978:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571788416 00000100:00100000:1.0:1788620016.467013:0:7654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571788544 00000100:00100000:1.0:1788620016.467016:0:7654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571788544 00000100:00100000:1.0:1788620016.467038:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 178 00000100:00100000:1.0:1788620016.467045:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6000380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571788416:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:1.0:1788620016.467095:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6000380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571788416:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 50us (409us total) trans 0 rc 0/0 00000100:00100000:1.0:1788620016.467101:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 178 00000100:00100000:1.0:1788620016.467114:0:7654:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 179 00000100:00100000:1.0:1788620016.467122:0:7654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571788544:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:1.0:1788620016.467227:0:7654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+5:2354:x1875503571788544:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 106us (480us total) trans 0 rc 0/0 00000100:00100000:1.0:1788620016.467233:0:7654:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 179 00000100:00100000:1.0:1788620016.467255:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571788288 00000100:00100000:1.0:1788620016.467259:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571788288 00000100:00100000:1.0:1788620016.471767:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 267 00000100:00100000:1.0:1788620016.471797:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571788288:12345-192.168.201.5@tcp:400:kworker.0 00000100:00100000:1.0:1788620016.471902:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571788288:12345-192.168.201.5@tcp:400:kworker.0 Request processed in 119us (5182us total) trans 0 rc 0/0 00000100:00100000:1.0:1788620016.471910:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 267 00000100:00080000:1.0:1788620016.471944:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1788620016.471950:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1788620016.471953:0:5840:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1788620016.742093:0:3318:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e78c74f5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589303680:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1788620016.743163:0:3318:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788620016.743287:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589303680 00000100:00100000:3.0:1788620016.743298:0:6699:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589303680 00000100:00100000:3.0:1788620016.743365:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 126 00000100:00100000:3.0:1788620016.743376:0:6699:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b16d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3318:x1875503589303680:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1788620016.743423:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 44236800 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1788620016.743485:0:6699:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b16d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3318:x1875503589303680:12345-0@lo:13:osp-pre-1-0.0 Request processed in 112us (328us total) trans 0 rc 0/0 00000100:00100000:3.0:1788620016.743493:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 126 00000100:00100000:3.0:1788620016.743552:0:3318:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e78c74f5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3318:1875503589303680:0@lo:13:osp-pre-1-0.0 00000001:00080000:1.0:1788620016.817922:0:7657:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967304 is committed 00000001:02000400:3.0:1788620017.242627:0:16114:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity test 36b: OST utime check (open, utime) ======== 10:53:35 (1788620015) 00000100:00100000:1.0:1788620017.293887:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788620017.293974:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571788800 00000100:00100000:3.0:1788620017.293988:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571788800 00000100:00100000:3.0:1788620017.294356:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1714 00000100:00100000:3.0:1788620017.294371:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:19044:x1875503571788800:12345-192.168.201.5@tcp:101:bash.0 00010000:00010000:3.0:1788620017.294400:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788620017.294405:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1788620017.294442:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64600/0x1cd7fe5368737426 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0ea expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620017.294472:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78c5f50900) 00010000:00010000:3.0:1788620017.294487:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1788620017.294507:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe536873747a lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1e106 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620017.294633:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64c00/0x1cd7fe5368737481 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x13:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620017.294651:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368737450 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x13:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0f8 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620017.294660:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c000/0x1cd7fe536873746c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0ff expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620017.294669:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64c00/0x1cd7fe5368737481 lrc: 5/0,1 mode: --/EX res: [0x200000401:0x13:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620017.294712:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c000/0x1cd7fe536873746c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0ff expref: 13 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620017.294729:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c000/0x1cd7fe536873746c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0ff expref: 13 pid: 5845 timeout: 609 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788620017.294746:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b14700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589303808:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788620017.294869:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368737450 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x13:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0f8 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620017.294884:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368737450 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x13:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0f8 expref: 13 pid: 5844 timeout: 609 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788620017.294893:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b14000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589303936:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788620017.294917:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff8e78df029480 going to sleep for 11 seconds 00000100:00100000:3.0:1788620017.299854:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b14700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589303808:192.168.201.5@tcp:104: 00000100:00100000:3.0:1788620017.299881:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b14000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5845:1875503589303936:192.168.201.5@tcp:104: 00010000:00010000:3.0:1788620017.299921:0:5845:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64c00/0x1cd7fe5368737481 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x13:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1788620017.306462:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788620017.306514:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788620017.306802:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571788928 00000100:00100000:3.0:1788620017.306819:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571788928 00010000:00010000:3.0:1788620017.306887:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c000/0x1cd7fe536873746c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0ff expref: 14 pid: 5845 timeout: 609 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788620017.306947:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571789056 00000100:00100000:3.0:1788620017.306951:0:5829:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571789056 00010000:00010000:3.0:1788620017.306965:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368737450 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x13:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0f8 expref: 15 pid: 5844 timeout: 609 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788620017.306997:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 568 00000100:00100000:3.0:1788620017.307008:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c80ea850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571788928:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788620017.307026:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788620017.307052:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c000/0x1cd7fe536873746c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0ff expref: 15 pid: 5845 timeout: 609 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620017.307074:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c000/0x1cd7fe536873746c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0ff expref: 15 pid: 5845 timeout: 609 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620017.307103:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c000/0x1cd7fe536873746c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0ff expref: 15 pid: 5845 timeout: 609 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620017.307125:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x13:0x0].0x0 (ffff8e78c67bc700) 00010000:00010000:3.0:1788620017.307132:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64c00/0x1cd7fe5368737481 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x13:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620017.307145:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788620017.307215:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c80ea850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571788928:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 209us (771us total) trans 0 rc 0/0 00000100:00100000:3.0:1788620017.307224:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 568 00000100:00100000:3.0:1788620017.307289:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 569 00000100:00100000:3.0:1788620017.307296:0:5829:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f7480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+13:2354:x1875503571789056:12345-192.168.201.5@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1788620017.307317:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1788620017.307323:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368737450 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x13:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0f8 expref: 13 pid: 5844 timeout: 609 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620017.307377:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368737450 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x13:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0f8 expref: 13 pid: 5844 timeout: 609 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620017.307390:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c683d000/0x1cd7fe5368737450 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x13:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e0f8 expref: 13 pid: 5844 timeout: 609 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620017.307406:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x13:0x0].0x0 (ffff8e78c67bc700) 00010000:00010000:3.0:1788620017.307410:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64c00/0x1cd7fe5368737481 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x13:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620017.307419:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64c00/0x1cd7fe5368737481 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x13:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620017.307428:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64c00/0x1cd7fe5368737481 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x13:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620017.307462:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1788620017.307484:0:5829:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f7480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+12:2354:x1875503571789056:12345-192.168.201.5@tcp:103:ldlm_bl.0 Request processed in 188us (970us total) trans 0 rc 0/0 00000100:00100000:3.0:1788620017.307490:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 569 00010000:00010000:3.0:1788620017.307654:0:5845:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64c00/0x1cd7fe5368737481 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x13:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620017.307664:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64c00/0x1cd7fe5368737481 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x13:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788620017.307689:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x13:0x0]: returning size 0 00010000:00010000:3.0:1788620017.308121:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64c00/0x1cd7fe5368737481 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x13:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620017.308131:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64c00/0x1cd7fe5368737481 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x13:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620017.308138:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64c00/0x1cd7fe5368737481 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x13:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620017.308148:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e78e6f64c00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:3.0:1788620017.308154:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64c00/0x1cd7fe5368737481 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x13:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620017.308162:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64c00/0x1cd7fe5368737481 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x13:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620017.308173:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x13:0x0].0x0 (ffff8e78c67bc700) 00010000:00010000:3.0:1788620017.308176:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1788620017.308180:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64c00/0x1cd7fe5368737481 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x13:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620017.308186:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64c00/0x1cd7fe5368737481 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x13:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620017.308269:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe536873747a lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e106 expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620017.308279:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78c5f50300) 00010000:00010000:3.0:1788620017.308281:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1788620017.308286:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe536873747a lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e106 expref: 12 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620017.308298:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e6f65e00, rc 0) 00000100:00100000:3.0:1788620017.308374:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+11:19044:x1875503571788800:12345-192.168.201.5@tcp:101:bash.0 Request processed in 14007us (14499us total) trans 4294967911 rc 301/301 00000100:00100000:3.0:1788620017.308382:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1714 00000100:00100000:3.0:1788620017.321317:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788620017.321582:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571789312 00000100:00100000:0.0:1788620017.321598:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571789312 00000100:00100000:0.0:1788620017.321684:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1715 00000100:00100000:0.0:1788620017.321699:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b14000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+11:19044:x1875503571789312:12345-192.168.201.5@tcp:36:bash.0 00010000:00010000:0.0:1788620017.321998:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777ce00/0x1cd7fe536873748f lrc: 3/0,1 mode: --/PW res: [0x200000401:0x13:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620017.322024:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777ce00/0x1cd7fe536873748f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620017.322051:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777ce00/0x1cd7fe536873748f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1788620017.322476:0:11397:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000e4a78163 reply state 000000008bf00493 transno 4294967912 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777ce00/0x1cd7fe536873748f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788620017.322490:0:11397:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788620017.326215:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b14000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+13:19044:x1875503571789312:12345-192.168.201.5@tcp:36:bash.0 Request processed in 4503us (4905us total) trans 4294967912 rc 0/0 00000100:00100000:0.0:1788620017.326232:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1715 00010000:00010000:0.0:1788620017.336566:0:3303:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777ce00/0x1cd7fe536873748f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620017.336591:0:3303:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777ce00/0x1cd7fe536873748f lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620017.336604:0:3303:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777ce00/0x1cd7fe536873748f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620017.336610:0:3303:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c777ce00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788620017.336617:0:3303:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777ce00/0x1cd7fe536873748f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620017.336626:0:3303:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777ce00/0x1cd7fe536873748f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620017.336651:0:3303:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x13:0x0].0x0 (ffff8e78cfedf000) 00010000:00010000:0.0:1788620017.336655:0:3303:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788620017.336661:0:3303:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777ce00/0x1cd7fe536873748f lrc: 2/0,0 mode: --/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620017.336669:0:3303:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777ce00/0x1cd7fe536873748f lrc: 0/0,0 mode: --/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1788620017.337830:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788620017.350594:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571789440 00000100:00100000:2.0:1788620017.350616:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571789440 00000100:00100000:2.0:1788620017.350713:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1716 00000100:00100000:2.0:1788620017.350727:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b14700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+12:19044:x1875503571789440:12345-192.168.201.5@tcp:101:bash.0 00010000:00010000:2.0:1788620017.350763:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788620017.350796:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3f000/0x1cd7fe5368737496 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x13:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1e10d expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620017.350885:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3f000/0x1cd7fe5368737496 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x13:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e10d expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620017.350904:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3f000/0x1cd7fe5368737496 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x13:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e10d expref: 13 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620017.350928:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x13:0x0].0x0 (ffff8e78c7cbba00) 00010000:00010000:2.0:1788620017.350931:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1788620017.350937:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6a3f000, rc 0) 00000100:00100000:2.0:1788620017.351044:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b14700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+13:19044:x1875503571789440:12345-192.168.201.5@tcp:101:bash.0 Request processed in 310us (13217us total) trans 0 rc 0/0 00000100:00100000:2.0:1788620017.351052:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1716 00000100:00100000:1.0:1788620017.356436:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788620017.356663:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571789568 00000100:00100000:3.0:1788620017.356677:0:5845:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571789568 00000100:00100000:3.0:1788620017.356754:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1717 00000100:00100000:3.0:1788620017.356767:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+13:19044:x1875503571789568:12345-192.168.201.5@tcp:101:bash.0 00010000:00010000:3.0:1788620017.356797:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788620017.356829:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe536873749d lrc: 2/0,0 mode: --/PW res: [0x200000401:0x13:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1e114 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620017.356871:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe536873749d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000080000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e114 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620017.356888:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f65e00/0x1cd7fe536873749d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000080000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e114 expref: 14 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1788620017.356919:0:5845:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x13:0x0] updating lvb size from disk: 0 -> 0 00000004:00010000:3.0:1788620017.356923:0:5845:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x13:0x0] updating lvb mtime from disk: 0 -> 1788620017 00000004:00010000:3.0:1788620017.356926:0:5845:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x13:0x0] updating lvb atime from disk: 0 -> 200000 00000004:00010000:3.0:1788620017.356929:0:5845:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x13:0x0] updating lvb ctime from disk: 0 -> 1788620017 00000004:00010000:3.0:1788620017.356931:0:5845:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x13:0x0] updating lvb blocks from disk: 18427282363563114435 -> 3 00010000:00010000:3.0:1788620017.356940:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x13:0x0].0x0 (ffff8e78c7cbba00) 00010000:00010000:3.0:1788620017.356943:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1788620017.356948:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e6f65e00, rc 0) 00000100:00100000:3.0:1788620017.357070:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+14:19044:x1875503571789568:12345-192.168.201.5@tcp:101:bash.0 Request processed in 304us (649us total) trans 0 rc 0/0 00000100:00100000:3.0:1788620017.357077:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1717 00000100:00100000:0.0:1788620017.362228:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788620017.362355:0:5859:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571789696 00000100:00100000:1.0:1788620017.362368:0:5859:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571789696 00002000:00010000:1.0:1788620017.362432:0:5859:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0x13:0x0] 00002000:00100000:1.0:1788620017.362438:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5f26a00 with x1875503571789696 00000100:00100000:1.0:1788620017.362478:0:5859:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1027 00000100:00100000:1.0:1788620017.362489:0:5859:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f26a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2354:x1875503571789696:12345-192.168.201.5@tcp:10:bash.0 00000004:00010000:1.0:1788620017.362862:0:5859:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x13:0x0] updating lvb size from disk: 0 -> 0 00000004:00010000:1.0:1788620017.362867:0:5859:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x13:0x0] updating lvb mtime from disk: 1788620017 -> 1788620017 00000004:00010000:1.0:1788620017.362870:0:5859:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x13:0x0] updating lvb atime from disk: 200000 -> 200000 00000004:00010000:1.0:1788620017.362872:0:5859:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x13:0x0] updating lvb ctime from disk: 1788620017 -> 1788620017 00000004:00010000:1.0:1788620017.362874:0:5859:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x13:0x0] updating lvb blocks from disk: 3 -> 3 00000100:00100000:1.0:1788620017.362968:0:5859:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f26a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+15:2354:x1875503571789696:12345-192.168.201.5@tcp:10:bash.0 Request processed in 479us (755us total) trans 4294967913 rc 0/0 00000100:00100000:1.0:1788620017.362976:0:5859:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1027 00002000:00010000:1.0:1788620017.362982:0:5859:0:(mdt_io.c:271:mdt_punch_hpreq_check()) lustre-MDT0000: refresh DOM lock for [0x200000401:0x13:0x0] 00002000:00100000:1.0:1788620017.362985:0:5859:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5f26a00 with x1875503571789696 00000100:00100000:0.0:1788620017.378618:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788620017.382746:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571789952 00000100:00100000:2.0:1788620017.382770:0:5847:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571789952 00000100:00100000:2.0:1788620017.382898:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 48 00000100:00100000:2.0:1788620017.382910:0:5847:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f27b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+15:19044:x1875503571789952:12345-192.168.201.5@tcp:35:bash.0 00000100:00100000:2.0:1788620017.383278:0:5847:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f27b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:19044:x1875503571789952:12345-192.168.201.5@tcp:35:bash.0 Request processed in 369us (4670us total) trans 4294967914 rc 0/0 00000100:00100000:2.0:1788620017.383290:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 48 00000100:00100000:1.0:1788620017.427455:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788620017.430597:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571790080 00000100:00100000:2.0:1788620017.430618:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571790080 00000100:00100000:2.0:1788620017.430695:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1718 00000100:00100000:2.0:1788620017.430708:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+16:19187:x1875503571790080:12345-192.168.201.5@tcp:101:utime.0 00010000:00010000:2.0:1788620017.430741:0:11397:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788620017.430777:0:11397:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3fa00/0x1cd7fe53687374a4 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1e11b expref: -99 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620017.430870:0:11397:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3ee00/0x1cd7fe53687374ab lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620017.430890:0:11397:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3ee00/0x1cd7fe53687374ab lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620017.430906:0:11397:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3ee00/0x1cd7fe53687374ab lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788620017.430959:0:11397:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:2.0:1788620017.430964:0:11397:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1788620017.430997:0:11397:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3ee00/0x1cd7fe53687374ab lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 11397 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620017.431038:0:11397:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3fa00/0x1cd7fe53687374a4 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e11b expref: 18 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620017.431066:0:11397:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3ee00/0x1cd7fe53687374ab lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e11b expref: 17 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620017.431082:0:11397:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78c7cbb200) 00010000:00010000:2.0:1788620017.431085:0:11397:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1788620017.431091:0:11397:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6a3ee00, rc 0) 00000100:00100000:2.0:1788620017.431181:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+17:19187:x1875503571790080:12345-192.168.201.5@tcp:101:utime.0 Request processed in 476us (3774us total) trans 0 rc 0/0 00000100:00100000:2.0:1788620017.431189:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1718 00000100:00100000:3.0:1788620017.438188:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:1.0:1788620017.438353:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571790336 00000100:00100000:1.0:1788620017.438367:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571790336 00000100:00100000:1.0:1788620017.438449:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1719 00000100:00100000:1.0:1788620017.438462:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b17100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+17:19187:x1875503571790336:12345-192.168.201.5@tcp:36:utime.0 00010000:00010000:1.0:1788620017.438543:0:9417:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1788620017.438581:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3ee00/0x1cd7fe53687374ab lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e11b expref: 17 pid: 11397 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620017.438606:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78c7cbb200) 00010000:00010000:1.0:1788620017.438622:0:9417:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1788620017.440356:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b17100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+16:19187:x1875503571790336:12345-192.168.201.5@tcp:36:utime.0 Request processed in 1886us (4020us total) trans 0 rc -17/-17 00000100:00100000:1.0:1788620017.440371:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1719 00010000:00010000:1.0:1788620017.444392:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788620017.444875:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:3.0:1788620017.444957:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571790464 00000100:00100000:3.0:1788620017.444972:0:5844:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571790464 00000100:00100000:3.0:1788620017.445059:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1720 00000100:00100000:3.0:1788620017.445072:0:5844:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78c74f5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+16:19187:x1875503571790464:12345-192.168.201.5@tcp:101:utime.0 00010000:00010000:3.0:1788620017.445099:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1788620017.445131:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64c00/0x1cd7fe53687374b2 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1e122 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620017.445222:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64600/0x1cd7fe53687374b9 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620017.445240:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64600/0x1cd7fe53687374b9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620017.445255:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64600/0x1cd7fe53687374b9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1788620017.445311:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:3.0:1788620017.445317:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1788620017.445378:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64600/0x1cd7fe53687374b9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620017.445408:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64c00/0x1cd7fe53687374b2 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e122 expref: 18 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620017.445428:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e78e6f64600/0x1cd7fe53687374b9 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e122 expref: 17 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1788620017.445438:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78c5f50300) 00010000:00010000:3.0:1788620017.445441:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1788620017.445446:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e78e6f64600, rc 0) 00000100:00100000:3.0:1788620017.445518:0:5844:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78c74f5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+17:19187:x1875503571790464:12345-192.168.201.5@tcp:101:utime.0 Request processed in 450us (653us total) trans 0 rc 0/0 00000100:00100000:3.0:1788620017.445526:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1720 00000100:00100000:3.0:1788620017.455068:0:3310:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:2.0:1788620017.455173:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571790592 00000100:00100000:2.0:1788620017.455186:0:5846:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571790592 00000100:00100000:2.0:1788620017.455258:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1721 00000100:00100000:2.0:1788620017.455271:0:5846:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b16a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+17:19187:x1875503571790592:12345-192.168.201.5@tcp:101:utime.0 00010000:00010000:2.0:1788620017.455309:0:5846:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1788620017.455365:0:5846:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3fa00/0x1cd7fe53687374c0 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1e129 expref: -99 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620017.455457:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3f200/0x1cd7fe53687374c7 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620017.455477:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3f200/0x1cd7fe53687374c7 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620017.455491:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3f200/0x1cd7fe53687374c7 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620017.455510:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e000/0x1cd7fe53687374ce lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620017.455518:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e000/0x1cd7fe53687374ce lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620017.455531:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e000/0x1cd7fe53687374ce lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620017.455929:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e400/0x1cd7fe53687374d5 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x13:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620017.455939:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e400/0x1cd7fe53687374d5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620017.455947:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e400/0x1cd7fe53687374d5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1788620017.455992:0:5846:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1788620017.456017:0:5846:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e400/0x1cd7fe53687374d5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620017.456047:0:5846:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3f200/0x1cd7fe53687374c7 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620017.456055:0:5846:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3f200/0x1cd7fe53687374c7 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620017.456063:0:5846:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3f200/0x1cd7fe53687374c7 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620017.456070:0:5846:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c6a3f200 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:2.0:1788620017.456075:0:5846:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3f200/0x1cd7fe53687374c7 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620017.456084:0:5846:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3f200/0x1cd7fe53687374c7 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620017.456103:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78c5f50300) 00010000:00010000:2.0:1788620017.456107:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788620017.456113:0:5846:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3f200/0x1cd7fe53687374c7 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620017.456121:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3f200/0x1cd7fe53687374c7 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620017.456136:0:5846:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e000/0x1cd7fe53687374ce lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620017.456143:0:5846:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e000/0x1cd7fe53687374ce lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620017.456149:0:5846:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e000/0x1cd7fe53687374ce lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620017.456154:0:5846:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c6a3e000 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:2.0:1788620017.456157:0:5846:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e000/0x1cd7fe53687374ce lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620017.456164:0:5846:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e000/0x1cd7fe53687374ce lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620017.456172:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xf7f6e39 (ffff8e78c7cbb800) 00010000:00010000:2.0:1788620017.456175:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1788620017.456178:0:5846:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e000/0x1cd7fe53687374ce lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620017.456184:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e000/0x1cd7fe53687374ce lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620017.456232:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3fa00/0x1cd7fe53687374c0 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e129 expref: 19 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620017.456248:0:5846:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e400/0x1cd7fe53687374d5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e129 expref: 18 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1788620017.456255:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x13:0x0].0x0 (ffff8e78c7cbba00) 00010000:00010000:2.0:1788620017.456258:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1788620017.456261:0:5846:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c6a3e400, rc 0) 00000100:00100000:2.0:1788620017.456383:0:5846:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b16a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92dda994-8428-47f5-a731-0b06f1aba2e4+18:19187:x1875503571790592:12345-192.168.201.5@tcp:101:utime.0 Request processed in 1114us (1330us total) trans 0 rc 0/0 00000100:00100000:2.0:1788620017.456392:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1721 00000100:00100000:3.0:1788620017.472047:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788620017.472141:0:9417:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571790848 00000100:00100000:0.0:1788620017.472156:0:9417:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571790848 00000100:00100000:0.0:1788620017.472236:0:9417:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1722 00000100:00100000:0.0:1788620017.472248:0:9417:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c6b16680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+18:19187:x1875503571790848:12345-192.168.201.5@tcp:36:utime.0 00010000:00010000:0.0:1788620017.472370:0:9417:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1788620017.472412:0:9417:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c6a3e400/0x1cd7fe53687374d5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e129 expref: 18 pid: 5846 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620017.472441:0:9417:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x13:0x0].0x0 (ffff8e78c7cbba00) 00010000:00010000:0.0:1788620017.472449:0:9417:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1788620017.472516:0:9417:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777ce00/0x1cd7fe53687374dc lrc: 3/0,1 mode: --/PW res: [0x200000401:0x13:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620017.472536:0:9417:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777ce00/0x1cd7fe53687374dc lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620017.472549:0:9417:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777ce00/0x1cd7fe53687374dc lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1788620017.472751:0:9417:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x13:0x0] updating lvb size from disk: 0 -> 0 00000004:00010000:0.0:1788620017.472756:0:9417:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x13:0x0] updating lvb mtime from disk: 1788620017 -> 100000 00000004:00010000:0.0:1788620017.472758:0:9417:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x13:0x0] updating lvb atime from disk: 200000 -> 200000 00000004:00010000:0.0:1788620017.472761:0:9417:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x13:0x0] updating lvb ctime from disk: 1788620017 -> 1788620017 00000004:00010000:0.0:1788620017.472763:0:9417:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x13:0x0] updating lvb blocks from disk: 3 -> 3 00000004:00010000:0.0:1788620017.472771:0:9417:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000014e173a5 reply state 0000000089b03f96 transno 4294967915 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777ce00/0x1cd7fe53687374dc lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1788620017.472780:0:9417:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1788620017.472909:0:9417:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c6b16680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92dda994-8428-47f5-a731-0b06f1aba2e4+19:19187:x1875503571790848:12345-192.168.201.5@tcp:36:utime.0 Request processed in 661us (889us total) trans 4294967915 rc 0/0 00000100:00100000:0.0:1788620017.472918:0:9417:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1722 00000100:00100000:0.0:1788620017.494566:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788620017.494606:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788620017.494634:0:3311:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.5@tcp (source: 12345-192.168.201.5@tcp) 00000100:00100000:0.0:1788620017.494680:0:5857:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571790976 00000100:00100000:0.0:1788620017.494693:0:5857:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571790976 00002000:00100000:0.0:1788620017.494776:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0x13:0x0] req@ffff8e77c5f26680 x1875503571790976/t0(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:113/0 lens 496/0 e 0 to 0 dl 1788620028 ref 1 fl New:/600/ffffffff rc 0/-1 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788620017.494789:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5f26680 with x1875503571790976 00000100:00100000:0.0:1788620017.494831:0:5857:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1028 00000100:00100000:0.0:1788620017.494843:0:5857:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f26680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+19:2353:x1875503571790976:12345-192.168.201.5@tcp:4:bash.0 00000004:00010000:0.0:1788620017.495376:0:5857:0:(mdt_lvb.c:89:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x13:0x0] updating lvb size from disk: 0 -> 1 00000004:00010000:0.0:1788620017.495382:0:5857:0:(mdt_lvb.c:96:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x13:0x0] updating lvb mtime from disk: 100000 -> 1788620017 00000004:00010000:0.0:1788620017.495386:0:5857:0:(mdt_lvb.c:102:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x13:0x0] updating lvb atime from disk: 200000 -> 200000 00000004:00010000:0.0:1788620017.495389:0:5857:0:(mdt_lvb.c:108:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x13:0x0] updating lvb ctime from disk: 1788620017 -> 1788620017 00000004:00010000:0.0:1788620017.495391:0:5857:0:(mdt_lvb.c:114:mdt_dom_disk_lvbo_update()) res: [0x200000401:0x13:0x0] updating lvb blocks from disk: 3 -> 3 00000100:00100000:0.0:1788620017.495481:0:5857:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f26680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_io00_000:92dda994-8428-47f5-a731-0b06f1aba2e4+21:2353:x1875503571790976:12345-192.168.201.5@tcp:4:bash.0 Request processed in 638us (929us total) trans 4294967916 rc 0/0 00000100:00100000:0.0:1788620017.495491:0:5857:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1028 00002000:00100000:0.0:1788620017.495500:0:5857:0:(mdt_io.c:150:mdt_rw_hpreq_check()) @@@ lustre-MDT0000 mdt_io00_000: refresh rw locks for [0x200000401:0x13:0x0] req@ffff8e77c5f26680 x1875503571790976/t4294967916(0) o4->92dda994-8428-47f5-a731-0b06f1aba2e4@192.168.201.5@tcp:113/0 lens 496/448 e 0 to 0 dl 1788620028 ref 1 fl Complete:/600/0 rc 0/0 job:'bash.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1788620017.495509:0:5857:0:(mdt_io.c:66:mdt_prolong_dom_lock()) Prolong DOM lock for req ffff8e77c5f26680 with x1875503571790976 00000100:00100000:0.0:1788620017.495650:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571791104 00000100:00100000:0.0:1788620017.495657:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571791104 00000100:00100000:0.0:1788620017.495691:0:11397:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503571791232 00000100:00100000:0.0:1788620017.495695:0:11397:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503571791232 00000100:00100000:0.0:1788620017.495717:0:11397:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1723 00000100:00100000:0.0:1788620017.495725:0:11397:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f25c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+22:2353:x1875503571791104:12345-192.168.201.5@tcp:16:kworker.0 00000100:00100000:0.0:1788620017.495788:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.5@tcp, seq: 1724 00000100:00100000:0.0:1788620017.495796:0:5845:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e77c5f24e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+22:19187:x1875503571791232:12345-192.168.201.5@tcp:101:utime.0 00010000:00010000:0.0:1788620017.495806:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1788620017.495829:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c000/0x1cd7fe53687374e3 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x5ab1062ad0a1e130 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620017.495905:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c200/0x1cd7fe53687374ea lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620017.495923:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c200/0x1cd7fe53687374ea lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620017.495937:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c200/0x1cd7fe53687374ea lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620017.495952:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777cc00/0x1cd7fe53687374f1 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620017.495959:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777cc00/0x1cd7fe53687374f1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620017.495966:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777cc00/0x1cd7fe53687374f1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620017.496369:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d600/0x1cd7fe53687374f8 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x13:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620017.496383:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777ce00/0x1cd7fe53687374dc lrc: 2/0,1 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620017.496392:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d600/0x1cd7fe53687374f8 lrc: 4/1,0 mode: --/PR res: [0x200000401:0x13:0x0].0x0 bits 0x13/0x8 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620017.496416:0:5845:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777ce00/0x1cd7fe53687374dc lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000020 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620017.496429:0:5845:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d600/0x1cd7fe53687374f8 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x13:0x0].0x0 bits 0x13/0x8 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620017.496634:0:3302:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777ce00/0x1cd7fe53687374dc lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000020 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620017.496645:0:3302:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777ce00/0x1cd7fe53687374dc lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000020 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620017.496652:0:3302:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777ce00/0x1cd7fe53687374dc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000020 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620017.496659:0:3302:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c777ce00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:1.0:1788620017.496663:0:3302:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777ce00/0x1cd7fe53687374dc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000020 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620017.496673:0:3302:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777ce00/0x1cd7fe53687374dc lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000020 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620017.496689:0:3302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x13:0x0].0x0 (ffff8e78c7cbba00) 00010000:00010000:1.0:1788620017.496692:0:3302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1788620017.496695:0:3302:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d600/0x1cd7fe53687374f8 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x13:0x0].0x0 bits 0x13/0x8 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620017.496704:0:3302:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d600/0x1cd7fe53687374f8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620017.496712:0:3302:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d600/0x1cd7fe53687374f8 lrc: 4/1,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620017.496755:0:3302:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777ce00/0x1cd7fe53687374dc lrc: 2/0,0 mode: --/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000020 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1788620017.496763:0:3302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777ce00/0x1cd7fe53687374dc lrc: 0/0,0 mode: --/PW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000020 pid: 9417 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620017.505360:0:5845:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d600/0x1cd7fe53687374f8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620017.505384:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d600/0x1cd7fe53687374f8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1788620017.505458:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1788620017.505490:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d600/0x1cd7fe53687374f8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620017.505510:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c200/0x1cd7fe53687374ea lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620017.505517:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c200/0x1cd7fe53687374ea lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620017.505525:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c200/0x1cd7fe53687374ea lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620017.505530:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c777c200 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788620017.505536:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c200/0x1cd7fe53687374ea lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620017.505544:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c200/0x1cd7fe53687374ea lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620017.505566:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff8e78c5f50300) 00010000:00010000:0.0:1788620017.505569:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788620017.505575:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c200/0x1cd7fe53687374ea lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620017.505586:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c200/0x1cd7fe53687374ea lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620017.505608:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777cc00/0x1cd7fe53687374f1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620017.505614:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777cc00/0x1cd7fe53687374f1 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620017.505620:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777cc00/0x1cd7fe53687374f1 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620017.505624:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8e77c777cc00 already unused, calling callback (ffffffffc22064f0) 00010000:00010000:0.0:1788620017.505627:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777cc00/0x1cd7fe53687374f1 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620017.505632:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777cc00/0x1cd7fe53687374f1 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620017.505640:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xf7f6e39 (ffff8e78cfedf000) 00010000:00010000:0.0:1788620017.505642:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1788620017.505645:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777cc00/0x1cd7fe53687374f1 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620017.505651:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777cc00/0x1cd7fe53687374f1 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xf7f6e39 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620017.505712:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777c000/0x1cd7fe53687374e3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e130 expref: 23 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620017.505730:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8e77c777d600/0x1cd7fe53687374f8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.201.5@tcp remote: 0x5ab1062ad0a1e130 expref: 22 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1788620017.505738:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x13:0x0].0x0 (ffff8e78c7cbba00) 00010000:00010000:0.0:1788620017.508977:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1788620017.508989:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8e77c777d600, rc 0) 00000100:00100000:0.0:1788620017.518934:0:5845:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f24e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92dda994-8428-47f5-a731-0b06f1aba2e4+22:19187:x1875503571791232:12345-192.168.201.5@tcp:101:utime.0 Request processed in 23120us (24279us total) trans 0 rc 0/0 00000100:00100000:0.0:1788620017.518952:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1724 00000001:00080000:3.0:1788620017.610753:0:5867:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967916 is committed 00000100:00100000:3.0:1788620017.611011:0:11397:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e77c5f25c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92dda994-8428-47f5-a731-0b06f1aba2e4+14:2353:x1875503571791104:12345-192.168.201.5@tcp:16:kworker.0 Request processed in 115277us (116394us total) trans 0 rc 0/0 00000100:00100000:3.0:1788620017.611040:0:11397:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.5@tcp, seq: 1723 00000100:00100000:0.0:1788620017.764095:0:3319:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff8e77c6b16300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3319:1875503589304064:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1788620017.764207:0:3319:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788620017.764276:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875503589304064 00000100:00100000:3.0:1788620017.764295:0:6699:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875503589304064 00000100:00100000:3.0:1788620017.764398:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 127 00000100:00100000:3.0:1788620017.764409:0:6699:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff8e78df967480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1875503589304064:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1788620017.764462:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 44236800 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1788620017.764527:0:6699:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff8e78df967480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1875503589304064:12345-0@lo:13:osp-pre-0-0.0 Request processed in 119us (319us total) trans 0 rc 0/0 00000100:00100000:3.0:1788620017.764535:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 127 00000100:00100000:0.0:1788620017.764676:0:3319:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff8e77c6b16300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3319:1875503589304064:0@lo:13:osp-pre-0-0.0 00010000:00010000:2.0:1788620018.468776:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:2.0:1788620018.828839:0:16162:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: sanity test_36b: @@@@@@ FAIL: utime failed for OST 00010000:00010000:1.0:1788620019.491487:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) Debug log: 71765 lines, 71765 kept, 0 dropped, 0 bad.